An agent cross-modal decision chain link security monitoring and causal attribution system

Abstract

This application relates to the field of artificial intelligence security technology, and in particular to a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent. The system includes: a runtime dual detection module, used to detect anomalies in the current reasoning state of the intelligent agent and output a warning signal when the current reasoning state is determined to be an abnormal state; a full-link structured decision tracking module, used to collect multimodal interaction data corresponding to the abnormal moment and its preceding moments and construct a structured decision trajectory with causal dependencies; a structured causal model attribution module, used to perform counterfactual intervention processing on target variables based on the structured decision trajectory, obtaining the probability distribution of action outputs and corresponding responsibility contribution weights under different intervention conditions; and a minimum semantic perturbation extraction module, used to perform semantic unit-level perturbation analysis on target variables whose responsibility contribution weights exceed a preset threshold, and extract the minimum semantic perturbation set under the condition of maintaining the abnormal result.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence security technology, and in particular to a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent. Background Technology

[0002] As Large Language Models (LLMs) evolve into intelligent agent systems with autonomous planning, tool invocation, and environmental interaction capabilities, their decision-making processes have expanded from single-round text generation to complex chains involving prompts, Retrieval Augmentation (RAG), CoT reasoning, and external tool / API calls. However, existing security pre-monitoring mechanisms still have the following shortcomings. First, insufficient runtime monitoring capabilities: Existing solutions mainly rely on input-output filtering or anomaly detection based on statistical features, which can only identify known attack patterns and struggle to detect intent drift, cognitive shifts, or logical anomalies generated by the agent during long-term reasoning. They lack deep monitoring capabilities for hidden layer states and internal semantic evolution. Second, a lack of effective causal attribution mechanisms: Existing interpretive methods are mostly based on feature correlation analysis, which cannot accurately distinguish whether abnormal results originate from knowledge contamination, model defects, externally induced input, or tool feedback errors, making it difficult to reliably assign responsibility for security failures in complex decision-making chains. Third, there is a lack of structured decision tracing capabilities: existing auditing methods mainly rely on discrete log records, lacking a structured decision tracing mechanism with temporal correlation and logical dependencies. This makes it difficult to reconstruct the complete reasoning path after a security failure, and also makes it difficult to achieve high-fidelity process reproduction and vulnerability localization. Fourth, there is a lack of formal verification methods for complex logic vulnerabilities: existing security assessments mainly rely on static test set scoring, making it difficult to verify and warn of runtime logic vulnerabilities such as recursive logic out of control and reasoning loop collapse in real time. Traditional formal verification methods suffer from state space explosion and semantic mapping difficulties when dealing with the probabilistic output and continuous semantic states of large language models. Therefore, existing technologies lack a security monitoring and attribution scheme for intelligent agent decision-making links that integrates runtime cognitive monitoring, structured decision tracing, causal intervention analysis, and logical verification. Summary of the Invention

[0003] This disclosure is provided to briefly introduce the concepts, which will be described in detail in the subsequent Detailed Description section. This disclosure is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] To address the aforementioned technical issues, this invention provides a security monitoring and causal attribution system for cross-modal decision-making links of intelligent agents. By integrating multi-dimensional detection during operation, structured decision tracking, and causal attribution analysis, a closed-loop assurance system covering the entire chain of "monitoring-tracking-attribution-verification" is constructed.

[0005] This invention provides a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent, comprising:

[0006] The runtime dual detection module is used to obtain runtime representation information during the execution of the tested intelligent agent according to the input original task instructions, and calculate geometric anomaly index and semantic offset index based on the runtime representation information to detect anomalies in the current reasoning state of the intelligent agent, and output a warning signal when the current reasoning state is determined to be an abnormal state.

[0007] The runtime representation information includes at least the hidden layer activation representation, the Logits vector output by the hidden layer, and the probability distribution generated based on the Logits vector;

[0008] The end-to-end structured decision tracking module is used to collect multimodal interaction data corresponding to the abnormal moment and its preceding moment when the warning signal is received, and to construct a structured decision trajectory with causal dependency based on time-series correlation index and state machine mirroring technology.

[0009] The multimodal interaction data includes at least: user instruction information, knowledge retrieval information, intermediate reasoning information, and API interaction information;

[0010] The structural causal model attribution module is used to convert the multimodal interaction data into a set of structured variables based on the structured decision trajectory, construct a structural causal model based on a directed acyclic graph representation, and perform counterfactual intervention processing on the target variable based on the structural causal model. While keeping other variables unchanged, the target variable is deleted, masked, or replaced to obtain the action output probability distribution corresponding to different intervention conditions, and the responsibility contribution weight corresponding to each state variable is calculated based on the difference between the action output probability distribution before and after the intervention.

[0011] The minimum semantic perturbation extraction module is used to perform semantic unit-level perturbation analysis on target variables whose responsibility contribution weight exceeds a preset threshold under a fixed random number seed, and extract the minimum semantic perturbation set under the condition of maintaining the abnormal results through replay verification and screening mechanisms.

[0012] Furthermore, the geometric anomaly index includes at least one of reconstruction error based on hidden layer activation representation or manifold distance calculated based on latent space distribution; the semantic offset index includes at least one of semantic mutual information, semantic similarity or target consistency score between the current inference state and the original task instruction.

[0013] Furthermore, the structural causal model is represented by a directed acyclic graph, including nodes for characterizing state variables in the agent's reasoning process and directed edges for characterizing the causal relationships and causal propagation directions between variables. The state variables include at least user instruction variables, knowledge retrieval variables, intermediate reasoning variables, tool invocation variables, and action output variables.

[0014] Furthermore, the runtime dual detection module includes at least: a geometric anomaly detection unit, used to reconstruct the hidden layer activation representation using a pre-trained variational autoencoder (VAE), and determine the degree of deviation of the current inference state based on the reconstruction error; or used to calculate the manifold distance d_m of the current inference trajectory relative to the normal inference trajectory based on the latent space distribution of the pre-trained variational autoencoder, and determine the degree of deviation of the current inference state based on the manifold distance; when the reconstruction error or the manifold distance exceeds or equals a preset dynamic threshold, it is determined that the current inference state has deviated;

[0015] The semantic deviation detection unit is used to analyze the probability distribution of the hidden layer Logits generated by the agent in real time and calculate the current inference state S. t Semantic mutual information between the original task objective and the objective, and based on this semantic mutual information, determine whether the current reasoning state has deviated;

[0016] When the geometric anomaly detection unit and the semantic deviation detection unit simultaneously determine that the current inference state has deviated, the current inference state is determined to be an abnormal state.

[0017] Furthermore, the reconstruction error formula is:

[0018]

[0019] Wherein, x represents the hidden layer activation tensor representation of the agent extracted in real time by the runtime dual detection module. This represents the estimated value of the expected tensor distribution after reconstruction via VAE encoding and decoding; This represents the set of standard behavioral characteristics of a normal task flow in the manifold space. The set of observed features representing the current state of disturbance or anomalous condition; As a denominator term, it represents the baseline variance under normal execution logic and is used to normalize the reconstruction error.

[0020] The formula for the manifold distance d_m is:

[0021]

[0022] in, denoted as the manifold distance between the current inference trajectory point x and the target safe state (or reference point) y in the high-dimensional decision vector space; f(x) and f(y) represent state transition functions used to map semantic states to coordinate vectors in the decision space; This represents the discretized representation of the distance in actual calculations.

[0023] Furthermore, the semantic mutual information formula is as follows:

[0024]

[0025] in, G represents the set of probability distributions of the Agent hidden layer Logits extracted in real time by the auditor, characterizing its current execution state space. The auditor is the Sentinel agent, used to perform bypass security auditing on the agent's operation process; G is the task intent space. The core semantic target vector in the vector; p(x,g) is the joint probability distribution of the action to be performed and the intention target, and p(x) and p(g) are the marginal probability distributions of the two respectively. This represents the discretized approximate computational form of the system in its running state, where n is the size of the semantic sampling space.

[0026] Furthermore, the end-to-end structured decision tracking module includes a semantic hook capture unit and a temporal correlation index unit.

[0027] Furthermore, the structural causal model attribution module includes:

[0028] A directed graph construction unit is used to construct a structural causal model in the form of a directed acyclic graph based on the structured decision trajectory.

[0029] The counterfactual intervention execution unit is used to perform intervention operations on the target variables in the structural causal model based on the do operator, and generate the corresponding action output probability distribution;

[0030] The quantitative responsibility unit is used to calculate the responsibility contribution weight of each variable based on the difference in the probability distribution of action outputs before and after intervention, using the Shapley Value method.

[0031] Furthermore, the formula for the probability distribution of the action output is:

[0032]

[0033] Wherein, X represents candidate causal component variables, specifically referring to units that may induce failure, such as "model inherent weights", "RAG (Retrieval-Augmented Generation) retrieval knowledge fragments" or "toolchain return results"; Y represents system security failure outcome variables, representing the illegal state of the Agent, such as logical collapse, intent hijacking or unauthorized actions; Z represents the set of confounding factors, representing environmental variables in the system that, in addition to X, simultaneously affect X and Y.

[0034] The weight of total contribution of responsibility The formula is:

[0035]

[0036] Representation Component The causal contribution weight represents the marginal contribution of the i-th system component to the occurrence of the final safety failure event; n represents the total number of components participating in the attribution, i.e., the set. The number of elements in; The summation constraint term is represented by N, which is the complete set of system components; S represents the absence of components. Any subset of components; represents the number of components contained in subset S, used to calculate the permutation probability when component i is added under this size combination; v(S) represents the characteristic function, which is defined in this project as the probability that the component subset S triggers system failure; Represents marginal contribution, measuring contribution within an existing set of components. Based on this, the net value that increases the probability of failure after adding component i; This represents a weighted probability factor. It is used to fairly weight all possible component addition orders, ensuring the uniqueness and impartiality of the attribution results.

[0037] Furthermore, the construction process of the pre-trained variational autoencoder includes at least the following:

[0038] Obtain the hidden layer activation representation corresponding to the historical safe reasoning trajectory under normal reasoning conditions;

[0039] A training sample set is constructed based on the hidden layer activation representation;

[0040] The variational encoder is trained using the training sample set, enabling the variational encoder to learn the latent space distribution features of the normal inference trajectory.

[0041] Based on the latent space distribution characteristics, a safety boundary representing the normal reasoning state is constructed for the detection of abnormal states in subsequent reasoning processes.

[0042] The security monitoring and causal attribution system for the cross-modal decision-making link of intelligent agents provided by this invention constructs a closed-loop security protection system for intelligent agents consisting of "perception-tracking-attribution-verification", which significantly improves the security monitoring capability, anomaly location capability and responsibility attribution capability of the intelligent agent decision-making link.

[0043] (1) Realize dynamic feedforward early warning: By combining VAE manifold trajectory monitoring with semantic mutual information verification, it can perceive cognitive deviation, intention drift and logical anomaly in the reasoning process of intelligent agents in real time, and trigger early warning and intervention before dangerous actions occur. Compared with the traditional static filtering method, it has higher real-time performance and accuracy.

[0044] (2) Achieve high-fidelity decision link tracing: Through the full-link structured decision tracing mechanism, the prompt words, search results, intermediate reasoning states and tool call processes are uniformly associated and persistently recorded, transforming the originally unanalyzable black box reasoning process into a traceable and replayable structured evidence chain, which significantly improves the vulnerability location and security audit capabilities.

[0045] (3) Achieve accurate failure attribution with causal logic: By introducing structural causal model and counterfactual intervention mechanism, it is possible to effectively remove environmental noise and confounding factors, accurately distinguish the source of abnormal results, and quantify the responsibility contribution of different components to the failure results. Compared with traditional correlation analysis methods, it has higher accuracy in assigning responsibility and logical rigor.

[0046] (4) Forming a closed loop of anomaly verification and system repair: Based on the minimum semantic perturbation set, the vulnerability triggering conditions are reproduced and verified, which can accurately locate the key factors that cause the logic failure, provide a reliable basis for model repair, cognitive optimization and security hardening, thereby significantly improving the overall security and stability of the intelligent agent system. Attached Figure Description

[0047] The above and other features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0048] Figure 1 This is a schematic diagram of a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent provided in an embodiment of the present invention.

[0049] Figure 2 A schematic diagram of a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent provided in another embodiment of the present invention;

[0050] Figure 3This is a schematic diagram illustrating the principle of liability determination for counterfactual decision-making failure based on the do-operator intervention of an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the agent inference trajectory reconstruction error distribution and security boundary determination based on a security semantic manifold in a security monitoring and causal attribution system for cross-modal decision-making links of an agent provided in an embodiment of the present invention. Detailed Implementation

[0052] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0053] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0054] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0055] It should be noted that the concepts of "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0056] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0057] The modules and units in the specific embodiments are numbered as follows: runtime dual detection module 100, full-link structured decision tracking module 200, structural causal model attribution module 300, minimum semantic disturbance extraction module 400, geometric anomaly detection unit 110, semantic deviation detection unit 120, semantic hook capture unit 210, temporal correlation index unit 220, directed graph construction unit 310, counterfactual intervention execution unit 320, and quantitative accountability unit 330.

[0058] In one or more embodiments of this application, a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent is proposed, comprising four core modules. Specifically, it includes: (1) a runtime dual detection module based on a dual judgment mechanism of geometric deviation and semantic deviation, used for multi-dimensional real-time monitoring of nondeterministic deviations that may occur in the long-term task flow of the intelligent agent; (2) a full-link structured decision tracking module, which captures reasoning state data in real time and constructs it into a structured data stream to provide a high-fidelity controlled environment for subsequent intervention experiments; (3) a structural causal model attribution module, which quantitatively analyzes the essential contribution of each variable to the failure result during the reasoning process and achieves accurate responsibility determination through a counterfactual reasoning mechanism based on the structural causal model and combined with the Shapley Value algorithm; and (4) a minimum semantic perturbation extraction module, which reversely extracts the minimum semantic features that trigger the failure result by fixing random seeds and tensor rewriting, and accurately verifies the accuracy of the attribution result. The integrated design of the four modules aims to achieve the following objectives: First, before the logical failure of the intelligent agent system evolves into substantial damage, the system can trigger feedforward early warning and execute intervention strategies; second, it can transform the fragmented "black box" execution process of the intelligent agent system into a parsable and replayable structured data stream, providing a high-fidelity controlled environment for subsequent failure mechanism analysis; third, it can achieve accurate and quantitative failure attribution and responsibility determination in non-deterministic decision-making environments; and fourth, while verifying the attribution results, it can provide precise positioning support for subsequent logical repair and intelligent agent system repair.

[0059] Please refer to Figure 1 This illustration shows a schematic diagram of an embodiment of a security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent according to this application. Figure 1 As shown, the security monitoring and causal attribution system for the cross-modal decision-making link of the intelligent agent includes: a runtime dual detection module, a full-link structured decision tracking module, a structured causal model attribution module, and a minimum semantic perturbation extraction module.

[0060] In this embodiment, the runtime dual detection module is used to perform real-time security monitoring of the inference process of the tested agent during the execution of the original task instructions. Specifically, during the agent's inference process, the module acquires its corresponding runtime representation information in real time, and performs anomaly judgment and outputs warnings based on the runtime representation information. The runtime representation information includes at least the activation representations of each hidden layer of the agent model, the Logits vectors obtained by mapping from the hidden layers, and the probability distribution obtained after normalizing the Logits vectors.

[0061] As an example, the runtime dual detection module detects the current inference state from both geometric and semantic dimensions. On one hand, it calculates a geometric anomaly index based on the hidden layer activation representation to determine the degree of deviation of the current inference state from the normal execution trajectory in the hidden space distribution. On the other hand, it calculates a semantic offset index based on the Logits vector and its corresponding probability distribution to characterize the consistency changes between the current inference output and the original task instructions. When the geometric anomaly index and the semantic offset index meet preset anomaly judgment conditions, an anomaly is determined in the current inference state, and a warning signal is output, enabling real-time monitoring and early intervention of potential logical deviations or semantic instability during long-range inference of the agent.

[0062] The geometric anomaly index may include at least one of the reconstruction error based on the hidden layer activation representation or the manifold distance calculated based on the latent space distribution; the semantic offset index may include at least one of the semantic mutual information between the current inference state and the original task objective, semantic similarity, or objective consistency score.

[0063] This invention provides an embodiment of semantic parsing of original task instructions based on a large language model, and constructs an arbitrary intent graph using methods such as semantic role labeling. Specifically, based on the large language model, predicate logic semantic parsing is performed on the input natural language instructions, deconstructing the input instructions into semantic elements such as actions, objects, pre-constraints, and post-constraints. Causal and temporal relationship edges are established between these semantic elements based on a knowledge ontology, forming a task intent space that includes goals, constraints, and tool calls. The task intent space This is used to provide a structured representation of the agent's intention state and logical constraints. The original task objective G is the task intention space. The core semantic target vector in the text.

[0064] The user-inputted raw task instructions are converted into first-order predicate logic expressions using a large language model, and these expressions are then injected into the task intent space as logical constraints. Contextual features, memory state features, and tool state features are extracted using Transformer, and high-dimensional heterogeneous features are uniformly mapped using principal component analysis and an autoencoder to generate a state vector S in the decision vector space. t The state vector S t This is used to characterize the agent's global cognitive state at time t, so that the reasoning process is represented as a trajectory evolution process in the decision vector space. Define the state transition function:

[0065] S t+1 =f(S t G cog )

[0066] Among them, S t+1 The state vector represents the cognitive state at the next moment; the state transition function is used to characterize the state evolution relationship in the reasoning process of the thought chain.

[0067] Furthermore, the state transition probability corresponding to the agent's inference trajectory is expressed as:

[0068]

[0069] in, represents the cumulative cognitive deviation of the agent's reasoning trajectory relative to the safe baseline manifold during the observation period; t represents the reasoning step index or time variable, used to describe the temporal characteristics of decision-making behavior as it evolves along the thought chain; f(x) and f(y) represent the state mapping functions on the safe semantic manifold, used to characterize the geometric distribution of the normal decision path; p(x,y) represents the probability density kernel of the state transition, reflecting the intrinsic logical probability of the agent transitioning from one thought state to the next. represents the local logical information entropy, which measures the change in uncertainty caused by interference or contamination during the reasoning process; p(y) represents the marginal probability density function of the target state y in the agent's decision space.

[0070] By performing probability integration on the reasoning trajectory, the discrete semantic reasoning steps are mapped to a continuous probability trajectory space, thus quantifying the degree of trajectory deviation in long-range thought chains.

[0071] The end-to-end structured decision tracing module, upon receiving the warning signal output by the runtime dual detection module, performs data backtracking and collection on the execution process of the agent's abnormal behavior at the moment of occurrence and its preceding moments, acquiring multimodal interaction data within the corresponding time window. This multimodal interaction data includes at least user-inputted instructions, knowledge retrieval information during the agent's execution process, intermediate inference information generated during the inference phase, and API calls and feedback information generated from interactions with external tools or APIs (Application Programming Interfaces). This data collectively constitutes a complete source of interactive evidence for the agent in the decision-making chain. The end-to-end structured decision tracing module uses temporal correlation indexing technology to chronologically label the multimodal interaction data and combines fixed random number seeds and state machine mirroring technology to structurally reconstruct discrete interaction events, generating a structured decision trajectory with clear node dependencies and propagation paths. This structured decision trajectory characterizes the agent's complete inference chain and state evolution process before and after the anomaly occurs, providing a unified and traceable structured data foundation for subsequent causal analysis and responsibility attribution. The structured decision trajectory includes at least user instruction nodes, knowledge retrieval nodes, tool feedback nodes, intermediate reasoning nodes, and action output nodes, and records the timestamp, calling order, and dependencies of each node.

[0072] The structural causal model attribution module is used to perform causal modeling and responsibility attribution analysis on the abnormal decision-making process of the agent based on the obtained structured decision trajectory. Specifically, the module parses and variates the multimodal interaction data in the structured decision trajectory, converting it into a set of structured variables representing the agent's reasoning process, providing a unified data expression form for subsequent causal modeling. A structural causal model is constructed based on this set of structured variables. The structural causal model is represented by a directed acyclic graph (DAG), which includes multiple nodes and multiple directed edges. Nodes represent the state variables in the agent's reasoning process, and directed edges represent the dependencies and information propagation paths between state variables, representing the structured causal relationships in the agent's decision-making chain. The state variables include at least user instruction variables, knowledge retrieval variables, intermediate reasoning variables, tool invocation variables, and action output variables. Then, based on the structural causal model, counterfactual intervention is performed on the preset target variable. While keeping the structural relationships and value conditions of other variables unchanged, deletion, masking, or replacement operations are performed on the target variable to simulate the reasoning process under different causal conditions and obtain the corresponding action output probability distribution P.

[0073] The formula for calculating the probability distribution P of the action output is as follows:

[0074]

[0075] X represents candidate causal component variables, which can refer to units that may induce failure, such as "model inherent weights", "RAG retrieval knowledge fragments" or "toolchain return results"; Y represents system security failure outcome variables, which represent the illegal state of the Agent, such as logical collapse, intent hijacking or unauthorized actions. This represents the set of confounding factors, which are environmental variables (such as historical dialogue context) that simultaneously influence both X and Y in the system, in addition to X. This formula quantifies the direct impact of intervening in a specific component X on the failure outcome Y.

[0076] By comparing the differences in the probability distribution of action outputs before and after intervention, the influence of each state variable on the target action output is quantitatively analyzed, and the responsibility contribution weights corresponding to each state variable are calculated, thereby realizing causal attribution and responsibility determination for abnormal decisions of the agent.

[0077] Among them, the responsibility contribution weight The calculation formula is:

[0078]

[0079] Representation Component The causal contribution weight represents the marginal contribution of the i-th system component to the occurrence of the final safety failure event; n represents the total number of components participating in the attribution, i.e., the set. The number of elements in; This represents the summation constraint term, where N is the complete set of system components. This expression indicates that the formula must iterate over components that do not include the target component. A subset S of all possible components is used to fairly measure the incremental impact of component i under different background combinations; S represents a subset of components that do not contain components. Any subset of components; denoted by , represents the number of components contained in subset S, used to calculate the permutation probability when component i is added under this size combination; v(S) represents the characteristic function, defined as the probability that the component subset S triggers system failure; Represents marginal contribution, measuring contribution within an existing set of components. Based on this, the net value that increases the probability of failure after adding component i; This represents a weighted probability factor. It is used to fairly weight all possible component addition orders, ensuring the uniqueness and impartiality of the attribution results.

[0080] The minimum semantic perturbation extraction module is used to refine and minimize the key semantic factors that lead to abnormal agent results. Specifically, the module operates under a fixed random seed condition to ensure the reproducibility of the reasoning process and thus eliminate the influence of random sampling on the result analysis. For target variables whose responsibility contribution weight exceeds a preset threshold, semantic unit-level perturbation analysis is performed. By performing perturbation operations such as deletion or masking on the target semantic units, different semantic variants are generated to simulate the impact of local semantic changes on abnormal results. The semantic units include at least sentence-level, phrase-level, word-level, or vector representation-level semantic units. The minimum semantic perturbation extraction module, combined with a replay verification mechanism, repeatedly verifies the agent execution process under different semantic perturbation conditions. Through a screening mechanism, it retains the minimum semantic perturbation combination that can still trigger the target's abnormal behavior while maintaining the abnormal result, obtaining a minimum semantic perturbation set. This minimum semantic perturbation set is used to characterize the minimum necessary semantic conditions for triggering abnormal agent behavior, providing a precise basis for subsequent anomaly mechanism analysis and security hardening.

[0081] In one or more embodiments of this application, such as Figure 2 As shown, the running dual detection module may include a geometric anomaly detection unit and a semantic deviation detection unit.

[0082] Specifically, the geometric anomaly detection unit is used to reconstruct the hidden layer activation representations generated during the agent's inference process based on a pre-trained variational autoencoder, and uses the reconstruction error as an evaluation metric to quantify the degree of deviation of the current inference state from the latent space distribution; or, based on the latent space distribution characteristics of the pre-trained variational autoencoder, it calculates the manifold distance d_m of the current inference state relative to the manifold structure corresponding to the normal inference trajectory, and uses the manifold distance to characterize the degree of geometric deviation of the current inference state. When the reconstruction error exceeds or equals a preset first dynamic threshold, or the manifold distance exceeds or equals a preset second dynamic threshold, it is determined that the current inference state has deviated geometrically. The preset first dynamic threshold and the preset second dynamic threshold can be statistically determined based on historical normal inference trajectory datasets, respectively.

[0083] Specifically, a variational autoencoder is used to represent the manifold trajectory under normal reasoning logic based on the Manifold Evolution Theory (DMET). The training process of the variational autoencoder includes: acquiring a large number of safe reasoning trajectories under normal agent reasoning conditions, and extracting the hidden layer activation representations corresponding to these safe reasoning trajectories as the source of representational data for the agent's normal operating state. A training sample set is constructed based on these hidden layer activation representations to represent the distribution characteristics of the latent space representation during normal reasoning. The variational encoder is trained using the training sample set, enabling it to learn the probability distribution characteristics of the normal reasoning trajectory in the latent space, which represents the normal behavioral manifold. Based on the latent space distribution characteristics, the normal behavioral manifold is extracted using the diffusion geometry (DG) algorithm. The geometric structure is defined as the safety boundary. .

[0084] The reconstruction error formula is:

[0085]

[0086] Wherein, x represents the hidden layer activation tensor representation of the agent extracted in real time by the runtime dual detection module. This represents the estimated value of the expected tensor distribution after reconstruction via VAE encoding and decoding; This represents the set of standard behavioral characteristics of a normal task flow in the manifold space. The set of observed features representing the current state of disturbance or anomalous condition; As a denominator term, it represents the baseline variance under normal execution logic and is used to normalize the reconstruction error.

[0087] The formula for the manifold distance d_m is:

[0088]

[0089] in, denoted as the manifold distance between the current inference trajectory point x and the target safe state (or reference point) y in the high-dimensional decision vector space; f(x) and f(y) represent state transition functions used to map semantic states to coordinate vectors in the decision space; This represents the discretized expression of the distance in actual calculation, through... The state deviation norms in each reasoning step are accumulated to provide the auditor with a calculation basis that can be directly programmed.

[0090] The semantic deviation detection unit is used to perform real-time parsing of the hidden layer Logits probability distribution generated by the agent based on the Sentinel agent, and to calculate the current inference state S. t The semantic mutual information between the current reasoning process and the original task objective G characterizes the degree of change in semantic target consistency in the current reasoning process, and determines whether the current reasoning state has deviated at the semantic level based on the changes in the semantic mutual information.

[0091] The semantic mutual information formula is as follows:

[0092]

[0093] x represents the set of probability distributions of the Agent hidden layer Logits extracted in real time by the Sentinel agent, representing its current execution state space; G is the task intent space. The core semantic target vector in the equation; p(x,g) is the joint probability distribution of the action and the intention target, and p(x) and p(g) are the marginal probability distributions of the two, respectively. The right side of the equation... This represents the discretized approximate computational form of the system in its running state, where n is the size of the semantic sampling space. By accumulating the autocorrelation density of the semantic distribution, the system can quickly capture logical deviation signals.

[0094] The Sentinel agent is an independently configured, non-intrusive monitoring agent platform used for security monitoring and compliance assessment. During execution, it includes the intermediate reasoning process from the tested agent receiving input prompts to outputting a token. The agent's hidden layer activation representation and Logits probability distribution sequence, along with other operational status information, are mirrored and synchronized to the Sentinel agent for processing. The Sentinel agent continuously monitors the current reasoning trajectory by calculating specific reference values ​​such as semantic mutual information and triggers an early warning signal when a semantic deviation exceeds a preset security threshold. Based on this, counterfactual analysis, root cause diagnosis, and responsibility attribution are performed, completing a full-link bypass security audit and monitoring of the agent's operation.

[0095] Through the synergistic action of the geometric anomaly detection unit and the semantic deviation detection unit, joint monitoring of the agent's operational state is achieved from two dimensions: the latent space distribution structure and the semantic goal consistency. When both the geometric anomaly detection unit and the semantic deviation detection unit simultaneously determine that the current inference state has deviated, it is determined that the current inference state has significant anomalies in both the structural distribution features and the semantic goal consistency dimensions. This classifies the inference state as an abnormal state and triggers corresponding early warning processing, thereby improving the reliability and robustness of anomaly identification.

[0096] The full-link structured decision tracking module includes a semantic hook capture unit and a temporal correlation index unit.

[0097] The semantic hook capture unit is used to trigger real-time collection of multimodal interaction data during the execution of the agent when the dual detection module in the running state determines that the current inference state is an abnormal state. The semantic hooks are deployed in the agent kernel layer, which may include a user input layer, a retrieval enhancement generation interface layer, a tool call interface layer, and an inference execution layer, to obtain multimodal interaction data during the agent's inference process.

[0098] The temporal correlation index unit is used to timestamp the collected multimodal interaction data and perform correlation modeling based on the execution order of each interaction event and the dependency relationship between variables to generate a structured decision trajectory with temporal consistency and structural dependency relationship, thereby realizing the traceable expression of the abnormal reasoning process of the agent and supporting subsequent analysis.

[0099] As an example, the semantic hook capture unit collects multimodal interaction data in real time when the agent's current inference state is determined to be an abnormal state. The semantic hook capture unit uses preset semantic hook functions to intercept the token stream, RAG retrieval context, and tool call requests and / or tool call responses generated by the agent in real time, forming a multimodal interaction data stream with the basic nodes in the structured decision trajectory as the granularity. The semantic hook functions are embedded in the inference execution chain and may include a tool call interception function (interceptToolCall) and a dialogue response interception function (interceptChatResponse).

[0100] For the collected basic nodes, i.e., atomic operation events, the temporal association index unit assigns a globally monotonically increasing temporal fingerprint (TI) to each time step, and establishes temporal associations between corresponding inference steps based on the temporal fingerprint. Simultaneously, the temporal fingerprint is bound to the hidden layer state snapshot corresponding to the current inference step, thereby recording the latent space representation state of the agent at each inference moment. The hidden layer state snapshot includes at least the hidden layer activation representation, Logits vector, and corresponding latent space representation vector. Based on the atomic operation events, a temporal-augmented decision graph is constructed, with Step_ID as the primary key to form a key-value pair structure. Each node includes at least input prompt information, retrieval context information, tool call result information, latent space vector representation, and confidence score. A structured causal path from task input to final action output is formed through temporal dependencies and variable propagation relationships. Based on this, the system uniformly controls the fixed random seed, applying it to the sampling process of the large language model decoder (including Temperature and Top-p parameters), the Dropout random masking mechanism inside the neural network, and the random state evolution process of the environmental simulator, eliminating the impact of nondeterministic disturbances on the reconstruction process. Then, using state machine mirroring technology, a full snapshot of the agent's runtime memory stack and environmental variables is stored during the recording phase, and a state machine mirror instance is constructed during the reconstruction phase. The agent's input / output interfaces are hijacked and controlled through a redirection layer, and historical multimodal interaction data is gradually fed back in according to the order indicated by the Temporal Association Index (TI). This achieves deterministic synchronous replay and path-level reproduction of the failed reasoning process, resulting in a high-fidelity decision path backtracking flow, i.e., a structured decision trajectory. This trajectory uses a temporal enhanced decision graph as its carrier, gradually recording the feature distribution information, logical branch selection information, and latent space representation change information corresponding to each reasoning step in the time-sensitive process. This achieves a complete reconstruction of the entire causal evolution process leading to the agent's decision failure and provides a highly guaranteed structured evidence basis for subsequent structural causal model attribution analysis.

[0101] The structural causal model attribution module includes a directed graph construction unit, a counterfactual intervention execution unit, and a quantitative accountability unit.

[0102] The directed graph construction unit is used to perform variable processing on the multimodal interaction data in the reasoning process based on the structured decision trajectory, and construct a structured causal model represented in the form of a directed acyclic graph, where nodes are used to represent each state variable, and directed edges are used to represent the dependency relationship and influence direction between variables, thereby forming a causal structure representation for describing the decision-making link of the agent.

[0103] The counterfactual intervention execution unit is used to perform intervention operations on the target variable based on the do operator on the basis of the structural causal model. While keeping the structural relationship and value conditions of other variables unchanged, the target variable is deleted or replaced to simulate the decision evolution process under different causal conditions and generate the corresponding action output probability distribution P.

[0104] The quantitative responsibility assignment unit is used to compare and analyze the changes in the probability distribution P of the action output before and after intervention, and to quantify the marginal contribution of each state variable in the abnormal action output based on the Shapley Value method, thereby obtaining the responsibility contribution weight corresponding to each variable, and realizing the causal attribution and responsibility allocation of the agent's abnormal decision.

[0105] The minimum semantic perturbation extraction module is used to analyze sentence-level, phrase-level, word-level, or vector representation-level semantic units, performing perturbation operations such as deletion or masking on the target semantic units to construct different semantic variants and simulate the impact of local semantic changes on the agent's reasoning results. Then, by combining a replay verification mechanism, the agent's execution process under different semantic perturbation conditions is repeatedly inferred and the consistency of results is verified. A filtering mechanism retains the minimum semantic perturbation combinations that can still trigger abnormal target behavior while maintaining the abnormal result, thus extracting and forming the minimum semantic perturbation set.

[0106] As an example, based on the attribution module of the structural causal model, the ShapleyValue causal contribution weights corresponding to each semantic input variable are calculated. Initial region locking is performed on semantic segments that contribute significantly to the agent's failure. These semantic segments include at least cue word constraint information, knowledge retrieval results, or tool call information. Based on this, atomic masking is performed on the locked semantic segments. Under a controlled environment with a fixed random seed, the semantic segments are masked, replaced, or deleted token by token. Word-by-word masking is performed on the locked segments to eliminate the impact of random perturbations on the analysis results, and candidate semantic subsets based on different mask combinations are constructed. Then, an iterative greedy search strategy is used to continuously filter the smallest token combination that can maintain the agent's abnormal behavior within the candidate semantic subsets. The geometric deviation and semantic consistency of the current inference state are evaluated by combining the reconstruction error or manifold distance of the variational autoencoder and the semantic mutual information index. Simultaneously, an equivalent perturbation is injected into the latent space through a tensor rewriting mechanism to verify whether the causal constraint relationship of the token subset on the abnormal behavior holds, achieving dual consistency verification at both the upper and latent space layers. The semantic subset that can stably trigger agent logic anomalies or decision deviations under minimum scale conditions is defined as the Minimal Semantic Perturbation Set (MSP). The MSP is used to characterize the minimum necessary semantic conditions that cause the agent's cross-modal decision link to fail, and serves as structured evidence input for knowledge pollution detection, malicious inducement identification, and system robustness analysis.

[0107] As an example, the security monitoring and causal attribution system for the cross-modal decision-making link of the intelligent agent described in this embodiment of the invention uses a physical machine equipped with an Intel Core Ultra 7 155H processor and 64GB RAM, running the Windows 11 operating system; based on the OpenClaw intelligent agent framework, the core tested intelligent agent is selected as the qwen2.5:14b model, and local deployment is achieved through the Ollam engine to ensure the real-time performance and privacy of data processing.

[0108] A comprehensive test library of 6,798 trajectories was constructed, including:

[0109] Training / Benchmark Set (D_routine): Contains 4,999 regular operation trajectories (such as file indexing, system status retrieval) used to define the geometric boundaries of the safe manifold;

[0110] Utility Validation Set (D_complex): Contains 999 "expert-level" research tasks with high semantic entropy, specifically designed to test the false alarm rate (FPR) of the system when handling complex legitimate requests.

[0111] Attack Test Set (D_malicious): Contains 500 malicious tracks covering typical risks such as target hijacking and prompt injection;

[0112] The adversarial evolution set (D_adaptive) contains 300 "mind masquerading" samples generated by the red team model. These samples masquerade as compliant tasks in the inference logic, but attempt to trigger illegal operations such as privilege escalation at the execution layer, and are used to evaluate the system's extreme interception capabilities.

[0113] The determination of geometric deviations in the agent's reasoning process is achieved by generating a 384-dimensional semantic embedding through an all-MiniLM-L6-v2 converter and running a nonlinear autoencoder (VAE) in a PyTorch 3.14 environment. The determination of semantic deviations in the agent's reasoning process is achieved by calling the Qwen3-Max high-parameter model through an asynchronous API call, which acts as an independent security judge to perform axiomatic auditing.

[0114] The VAE is trained until the Mean Squared Error (MSE) converges to 0.000309; the 99th quantile of the MSE distribution in the training set is extracted to determine the safety boundary threshold of the system. =0.00086608.

[0115] During the execution of the agent, when the agent generates a tool call request, the request is intercepted through a preset `interceptToolCall` hook function, and the corresponding execution thread is paused, putting the tool call request into an auditable state. Then, Tier 1 (geometric audit) filtering is performed, and the reconstruction error is calculated on the hidden layer activation representation, context state vector, etc., corresponding to the current tool call request to obtain the corresponding MSE. When the MSE < The system determines that the current tool call request belongs to a safe and fast path, and directly generates a permission command to respond to the execution thread; when the MSE ≥ This triggers Tier 2 (cognitive auditing), which reconstructs the context of the current tool call request, including the task context, inference path, historical tool call records, retrieval enhancement content, and memory state. It also performs semantic determination on the tool call request. Based on the semantic determination result, it generates a corresponding execution decision file and, according to the execution decision file, executes the thread to resume tool call execution or terminates the current task flow.

[0116] Table 1: Core Technology Performance Indicators

[0117]

[0118] Experimental results show that the system described in this embodiment of the invention can effectively identify the hidden failure behaviors of intelligent agents caused by reasoning drift, knowledge pollution, tool misuse, or semantic deviation in complex interactive environments by detecting anomalies in the latent space manifold shift during the operation of intelligent agents and performing responsibility analysis on the abnormal decision-making links in combination with structural causal attribution. This solves the problem that existing technologies are unable to detect "silent failures" of intelligent agents and lack interpretability of anomaly sources, improves the cognitive safety monitoring capability, anomaly localization capability, and interpretability of safety decisions during the operation of intelligent agent systems, and makes up for the shortcomings of existing technologies in the cognitive safety control and operational causal analysis of intelligent agents.

[0119] like Figure 4 As shown in the figure, the MSE distribution of the inference trajectory of the tested agent in Tier 1 geometric deviation detection during the execution of different types of tasks is illustrated. Specifically, the reconstruction error of the benign routine operation dataset D_routine generally exhibits a low-value clustering distribution, indicating that the inference behavior corresponding to routine tasks has a high consistency with the safe semantic manifold obtained by the system's pre-training. The reconstruction errors of the expert research task dataset D_complex and the adaptive adversarial sample dataset D_adaptive show a distribution characteristic of diffusion towards high-value regions. Among them, due to the high semantic complexity of the expert research task, the geometric features of its inference trajectory in the latent space show a certain degree of divergence, while the adaptive attack samples show obvious structural shift features, indicating that malicious attack behavior can change the original cognitive manifold structure of the agent. The system sets a safety boundary threshold based on the reconstruction error. When the reconstruction error corresponding to the current inference trajectory is less than the safety boundary threshold, the inference trajectory is determined to be inside the safe manifold and is allowed to proceed directly; when the reconstruction error is greater than or equal to the safety boundary threshold, the current inference trajectory is determined to have a cognitive offset risk and the Tier 2 cognitive audit module is triggered to perform further security analysis.

[0120] It is important to emphasize that malicious attack trajectories and some highly complex legitimate task trajectories have local overlap areas in the latent space. Relying solely on geometric distribution features is insufficient to completely distinguish between covert attack behaviors and normal complex reasoning behaviors. Therefore, this invention further introduces a Tier 2 axiomatic cognitive auditing mechanism to perform context reconstruction, logical consistency verification, and causal responsibility analysis on reasoning trajectories within the overlapping areas. This reduces the false alarm rate and improves the accuracy of security judgments in complex scenarios while ensuring low system latency. Specifically, it can reduce the false alarm rate (FPR) to an extremely low level of 0.40%.

[0121] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0122] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain environments. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this application. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0123] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A security monitoring and causal attribution system for a cross-modal decision-making link of an intelligent agent, characterized in that, include: The runtime dual detection module is used to acquire runtime representation information corresponding to the reasoning process in real time during the execution of the tested intelligent agent according to the input original task instructions, and calculate geometric anomaly index and semantic offset index based on the runtime representation information to detect anomalies in the current reasoning state of the intelligent agent, and output a warning signal when the current reasoning state is determined to be an abnormal state. The runtime representation information includes at least the hidden layer activation representation, the Logits vector output by the hidden layer, and the probability distribution generated based on the Logits vector; The end-to-end structured decision tracking module is used to collect multimodal interaction data corresponding to the abnormal moment and its preceding moment when the warning signal is received, and to construct a structured decision trajectory with causal dependency based on the multimodal interaction data. The multimodal interaction data includes at least: user instruction information, knowledge retrieval information, intermediate reasoning information, and application programming interface interaction information; The structural causal model attribution module is used to convert the multimodal interaction data into a set of structured variables based on the structured decision trajectory, construct a structural causal model based on a directed acyclic graph representation, and perform counterfactual intervention processing on the target variable based on the structural causal model. While keeping other variables unchanged, the target variable is deleted, masked, or replaced to obtain the action output probability distribution corresponding to different intervention conditions, and the responsibility contribution weight corresponding to each state variable is calculated based on the difference between the action output probability distribution before and after the intervention. The minimum semantic perturbation extraction module is used to perform semantic unit-level perturbation analysis on target variables whose responsibility contribution weight exceeds a preset threshold under a fixed random number seed, and extract the minimum semantic perturbation set under the condition of maintaining the abnormal results through replay verification and screening mechanisms.

2. The system according to claim 1, characterized in that, The geometric anomaly index includes at least one of reconstruction error based on hidden layer activation representation or manifold distance calculated based on latent space distribution; the semantic offset index includes at least one of semantic mutual information, semantic similarity or target consistency score between the current inference state and the original task instruction.

3. The system according to claim 1 or 2, characterized in that, The structural causal model is represented by a directed acyclic graph, including nodes for characterizing state variables in the agent's reasoning process and directed edges for characterizing the causal relationships and causal propagation directions between variables. The state variables include at least user instruction variables, knowledge retrieval variables, intermediate reasoning variables, tool invocation variables, and action output variables.

4. The system according to claim 3, characterized in that, The operational dual detection module includes at least a geometric anomaly detection unit and a semantic deviation detection unit, wherein, A geometric anomaly detection unit is used to reconstruct the activation representation of the hidden layer using a pre-trained variational autoencoder and determine the degree of deviation of the current inference state based on the reconstruction error; or, it is used to calculate the manifold distance between the current inference trajectory and the normal inference trajectory based on the latent space distribution of the variational autoencoder and determine the degree of deviation of the current inference state based on the manifold distance; when the reconstruction error or the manifold distance exceeds or equals a preset dynamic threshold, it is determined that the current inference state has deviated. The semantic deviation detection unit is used to analyze the probability distribution of hidden layer Logits and the current inference state representation generated by the agent in real time, calculate the semantic mutual information between the current inference state and the original task instruction, and determine whether the current inference state has deviated based on the semantic mutual information. When the geometric anomaly detection unit and the semantic deviation detection unit simultaneously determine that the current inference state has deviated, the current inference state is determined to be an abnormal state.

5. The system according to claim 4, characterized in that, The reconstruction error formula is: Wherein, x represents the hidden layer activation tensor representation of the agent extracted in real time by the runtime dual detection module. This represents the expected tensor distribution estimate after encoding and decoding by a variational autoencoder; This represents the set of standard behavioral characteristics of a normal task flow in the manifold space. The set of observed features representing the current state of disturbance or anomalous condition; As a denominator term, it represents the baseline variance under normal execution logic and is used to normalize the reconstruction error. The formula for manifold distance is: ;in, denoted as the manifold distance between the current inference trajectory point x and the target safe state (or reference point) y in the high-dimensional decision vector space; f(x) and f(y) represent state transition functions used to map semantic states to coordinate vectors in the decision space; This represents the discretized representation of the distance in actual computation. The state deviation norms in each reasoning step are accumulated.

6. The system according to claim 5, characterized in that, The semantic mutual information formula is: ;in, G represents the set of probability distributions of the Agent hidden layer Logits extracted in real time, characterizing its current execution state space; G is the task intent space. The core semantic target vector in the equation; p(x,g) is the joint probability distribution of the action and the intention target, and p(x) and p(g) are the marginal probability distributions of the two, respectively; the right side of the equation... This represents the discretized approximate computational form of the system in its running state, where n is the size of the semantic sampling space.

7. The system according to claim 6, characterized in that, The end-to-end structured decision tracing module includes a semantic hook capture unit and a temporal correlation index unit, wherein, The semantic hook capture unit is used to collect corresponding multimodal interaction data in real time when the current reasoning state of the agent is determined to be an abnormal state. The temporal correlation index unit is used to timestamp the collected information and construct a structured decision trajectory based on execution order, variable dependency relationship and state transition relationship using a fixed random seed and state machine mirroring mechanism; wherein, the structured decision trajectory includes at least user instruction node, knowledge retrieval node, tool feedback node, intermediate inference node and action output node, and records the timestamp, calling order and dependency relationship of each node.

8. The system according to claim 7, characterized in that, The structural causal model attribution module includes a directed graph construction unit, a counterfactual intervention execution unit, and a quantitative accountability unit, wherein... A directed graph construction unit is used to construct a structural causal model in the form of a directed acyclic graph based on the structured decision trajectory. The counterfactual intervention execution unit is used to perform intervention operations on the target variables in the structural causal model based on the do operator, and generate the corresponding action output probability distribution; The quantitative responsibility unit is used to calculate the responsibility contribution weight of each state variable based on the difference in the probability distribution of action outputs before and after intervention, using the Shapley Value method.

9. The system according to claim 8, characterized in that, The formula for the probability distribution of the action output is: Where X represents candidate causal component variables; Y represents system security failure outcome variables, representing the violation state ultimately exhibited by the agent; This represents the set of confounding factors, which are environmental variables in the system that affect both X and Y, in addition to X. The formula for the weight of the total contribution of responsibility is: ;in, Representation Component The causal contribution weight represents the marginal contribution of the i-th system component to the occurrence of the final safety failure event; n represents the total number of components involved in the attribution. The summation constraint term is represented by N, which is the complete set of system components; S represents the absence of components. Any subset of components; represents the number of components contained in subset S, used to calculate the permutation probability when component i is added under this size combination; v(S) represents the characteristic function, which is defined in this project as the probability that the component subset S triggers system failure; Represents marginal contribution, measuring contribution within an existing set of components. Based on this, the net value that increases the probability of failure after adding component i; This represents the weighting probability factor, used to fairly weight all possible component addition orders.

10. The system according to claim 9, characterized in that, The construction process of the pre-trained variational autoencoder includes at least the following: Obtain the hidden layer activation representation corresponding to the historical safe reasoning trajectory under normal reasoning conditions; A training sample set is constructed based on the hidden layer activation representation; The variational encoder is trained using the training sample set, enabling the variational encoder to learn the latent space distribution features of the normal inference trajectory. Based on the latent space distribution characteristics, a safety boundary representing the normal reasoning state is constructed for the detection of abnormal states in subsequent reasoning processes.

Images