An agent running trajectory auditing method and system
Patent Information
- Application Number
- CN202611053205.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-15
- Publication Date
- 2026-10-30
AI Technical Summary
[0004]第一，多源运行事件缺少统一关联与因果链
[0028]本说明书一些实施例提供的技术方案带来的有益效果至少包括：
Smart Images
Figure CN123012394A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of information technology, specifically to a method and system for auditing the operational trajectory of an intelligent agent. Background Technology
[0002] In the fields of artificial intelligence software governance and information security technology, auditing the operational trajectory of intelligent agents based on large language models typically relies on traditional application log recording, general distributed tracing technology (such as Span propagation based on OpenTelemetry), and policy interception mechanisms that block execution based on rules before execution.
[0003] Compared to traditional application software, intelligent agents' decision-making paths are characterized by non-determinism, strong semantic dependencies, and dispersed external side effects, leading to the following auditing challenges:
[0004] First, multi-source events lack a unified correlation and causal chain. Various logs belong to different systems, making it difficult to reconstruct the complete decision-making chain afterward.
[0005] Second, there is a lack of automatic comparison between the published baseline and the operational trajectory. There is a lack of an automated "baseline-trajectory" diff mechanism to determine whether the actual trajectory during operation matches the published baseline. Violations often require manual review afterward, resulting in poor timeliness.
[0006] Third, audit logs are difficult to prove have not been tampered with. Distributed logs are easily deleted or tampered with, and anti-tampering measures such as hash chains are often independent of baseline comparisons and deviation annotations, making it difficult to verify both record integrity and behavioral compliance within the same view.
[0007] Fourth, cross-session anomalies are difficult to detect. Inconspicuous abnormal behaviors within a single session require multi-session windowing aggregation for identification, while single-point log retrieval or single trace alarms are insufficient to support such cross-session correlation analysis.
[0008] Fifth, the investigation process lacked a baseline comparison with the actual trajectory. Presenting the original Span or dialogue excerpts made it difficult for investigators to compare the baseline rules in effect at the time of the event with the actual behavior and deviations marked on the same decision-making timeline, significantly hindering compliance review. Summary of the Invention
[0009] This specification describes a method and system for auditing the trajectory of an intelligent agent through several embodiments.
[0010] Firstly, embodiments of this specification provide a method for auditing the operational trajectory of an intelligent agent, including the following steps:
[0011] During the operation of the intelligent agent, at least one type of running event is collected, and a global association identifier is generated for the same intelligent agent running instance. All running events within the same user task or the same session window are bound to the global association identifier.
[0012] The heterogeneous runtime events are converted into standard audit events conforming to a preset schema to obtain a standard audit event sequence;
[0013] Obtain baseline description information corresponding to the instance of the agent and the version number of the policy context currently used; match the standard audit event sequence with the baseline description information; and output the comparison results and categorical deviation items.
[0014] The standard audit events, deviations in the comparison results, and integrity verification values are written into the audit storage area in chronological order.
[0015] In response to an audit query request, the system retrieves and returns the standard audit event sequence and associated deviations from the audit storage area based on the global association identifier.
[0016] Secondly, embodiments of this specification provide an intelligent agent trajectory auditing system, including:
[0017] The acquisition module collects at least one type of running event during the operation of the intelligent agent, generates a global association identifier for the same intelligent agent running instance, and binds all running events within the same user task or the same session window to the global association identifier.
[0018] The conversion module converts the heterogeneous runtime events into standard audit events that conform to a preset schema, thereby obtaining a standard audit event sequence;
[0019] The matching module obtains baseline description information corresponding to the instance of the agent and the version number of the policy context currently used, matches the standard audit event sequence with the baseline description information, and outputs the comparison results and classification deviation items.
[0020] The storage module writes the standard audit events, deviations in the comparison results, and integrity verification values into the audit storage area in chronological order.
[0021] The audit module responds to audit query requests by retrieving and returning the standard audit event sequence and associated deviations from the audit storage area based on the global association identifier.
[0022] Thirdly, embodiments of this specification provide an electronic device, including a processor and a memory;
[0023] The processor is connected to the memory;
[0024] The memory is used to store executable program code;
[0025] The processor runs a program corresponding to the executable program code stored in the memory to perform the method described in any of the above aspects.
[0026] Fourthly, embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the above aspects.
[0027] Fifthly, embodiments of this specification provide a computer program product, including a computer program that, when executed by a processor, implements the methods described in any of the above aspects.
[0028] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:
[0029] In several embodiments of this specification, the provided intelligent agent operation trajectory auditing method and system, through global association identification, standard audit event schema with predecessor references, and decision timeline playback, connects heterogeneous operation records scattered in multiple systems into a reproducible complete decision chain of "reasoning -> tool -> feedback", which solves the problem of lack of causal relationship of multi-source events of intelligent agents and reduces the splicing cost of post-event investigation and regulatory evidence collection.
[0030] The baseline comparison module automatically matches the actual trajectory during runtime with the baseline description information corresponding to the released artifact version and outputs categorized deviation items, so that deviation behavior can be located during runtime or batch processing, making up for the inability of the pre-execution interception scheme to provide a complete post-execution Diff.
[0031] The correlation analysis module aggregates standard audit events corresponding to multiple global correlation identifiers within a preset time window. It can identify risk patterns that are not significant in a single session but recur across instances and output alarms. It also supports one-click drill-down from alarms to the corresponding decision timeline view, restoring attack signals scattered in noise into verifiable causal chains.
[0032] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description
[0033] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0034] Figure 1 This is a schematic diagram for auditing the operational trajectory of the intelligent agent provided in this specification.
[0035] Figure 2 This is a schematic diagram of the intelligent agent operation trajectory auditing method provided in this specification.
[0036] Figure 3 This is a flowchart illustrating the method for matching baseline description information provided in this specification.
[0037] Figure 4 This is a schematic diagram of the intelligent agent trajectory auditing system provided in this specification.
[0038] Figure 5 This is a schematic diagram of the electronic device provided in this manual. Detailed Implementation
[0039] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.
[0040] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.
[0041] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.
[0042] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0043] Before introducing the technical solutions described in this manual, the application scenarios and related technologies of the technical solutions will be introduced.
[0044] Large Language Model (LLM)-based agents represent one of the most transformative technological directions in the field of artificial intelligence. Unlike traditional software systems that execute deterministic operations based on pre-defined logic, agents possess the ability to autonomously understand goals, plan tasks, invoke tools, and dynamically interact with the environment. Currently, agents have moved from technological concepts to large-scale applications, with implementations in areas such as financial risk control, intelligent manufacturing, medical diagnosis, and enterprise process automation. IDC data shows that by 2025, the proportion of industrial enterprises using large models and agents will jump from 9.6% in 2024 to 47.5%. As agents become deeply integrated into core processes such as enterprise R&D, operations, data analysis, and business delivery, they have evolved from personal efficiency tools into a new type of productivity unit.
[0045] However, the autonomy and uncertainty of intelligent agents also bring unprecedented auditing and governance challenges.
[0046] Unlike traditional automated systems with predictable behavior and well-defined boundaries, the decision-making path of an intelligent agent is characterized by dynamic programming, multi-step reasoning, and tool combination. Its operation involves multi-source heterogeneous events such as model reasoning, tool invocation, retrieval augmentation generation (RAG), and memory reading and writing.
[0047] Currently, the industry generally relies on traditional methods such as application logs, distributed tracing (such as OpenTelemetry), and pre-execution policy interception. However, these technologies are not designed for the dynamic decision-making characteristics of intelligent agents, making it difficult to reconstruct the complete causal chain from user intent to final response, and they also lack the ability to automatically compare the running trajectory with the release baseline.
[0048] Against this backdrop, there is still a need to research dedicated auditing schemes for the operational trajectories of intelligent agents, in order to support controllable, trustworthy, and traceable auditing and management of intelligent agents.
[0049] Given that this application involves some technical terms, these terms will be introduced below.
[0050] Intelligent Agent:
[0051] Autonomous programs based on Large Language Models (LLMs) can understand user goals, perform multi-step reasoning, invoke external tools, and remember context, enabling them to complete complex tasks.
[0052] Large Language Model (LLM):
[0053] As the "brain" of an intelligent agent, it can understand natural language input and generate reasoning plans or final responses.
[0054] Multi-hop Reasoning:
[0055] When dealing with complex problems, intelligent agents need to go through multiple iterative processes of model inference and tool calls, with each step depending on the result of the previous step.
[0056] RAG (Retrieval Enhancement Generation):
[0057] Before generating an answer, the agent retrieves relevant document fragments from an external knowledge base (vector database) as a reference to enhance the accuracy and timeliness of the answer.
[0058] Baseline description information 30:
[0059] A set of compliance constraints defined when an intelligent agent is released includes a whitelist of allowed tools, prohibited data access patterns, a summary of cue constraints, and a maximum number of tool call steps.
[0060] This embodiment provides a method and system for auditing the operational trajectory of an intelligent agent. Please refer to the appendix. Figure 1 This involves constructing a complete audit pipeline from event collection to compliance verification. During the actual operation of the intelligent agent, at least one type of runtime event 21 is captured through the collection agent. For example, user questions, model inference outputs, tool call requests, or enhanced read / write operations are captured. A global association identifier 11 is generated for the current intelligent agent instance 10, and all events belonging to the same session or the same user task are uniformly bound under this identifier, thus providing a foundation for subsequent cross-system interconnection.
[0061] Subsequently, the heterogeneous raw log format is converted into standard audit events 22 that conform to the preset schema, forming a structured audit event sequence. Taking the e-commerce customer service scenario as an example, a task of "querying an order and initiating a refund" will be broken down into multiple standardized events such as user input, model inference, and two parallel tool calls, and the causal dependencies between them will be clarified by referencing predecessor events.
[0062] Based on this, according to the agent instance 10 identifier and the currently effective policy context version number, the corresponding baseline description information 30 is obtained. This specifies constraints such as the allowed set of tools, the maximum number of calls per session, and prohibited data patterns. The entire audit event sequence is automatically matched against the baseline, outputting a comparison result 31 containing specific deviation types. For example, if a call to the sensitive tool "admin.users.delete" outside the whitelist is detected, it will be marked as a "UNAUTHORIZED_TOOL" type deviation. Next, the method writes the standard audit event 22, the deviation, and the integrity verification value 32 calculated based on the event payload, timestamp, and preceding event hash in chronological order to the append-only audit storage area. This ensures that any tampering with historical records will disrupt the subsequent verification chain, thereby guaranteeing that the audit data is tamper-resistant in evidentiary scenarios.
[0063] When auditors initiate a query request, the method retrieves relevant event sequences and deviations from the storage area based on the global association identifier 11, and can further restore the complete decision timeline view by referencing preceding events, allowing investigators to check the difference between the actual trajectory and the baseline at that time on the same interface.
[0064] This embodiment provides a complete framework for auditing the operational trajectory of an intelligent agent, encompassing data acquisition, normalization, comparison, integrity protection, and playback. Specifically, this embodiment first provides a method for auditing the operational trajectory of an intelligent agent; please refer to the appendix. Figure 2 The steps include:
[0065] Step S1: When the agent is running, collect at least one type of running event 21, generate a global association identifier 11 for the same agent running instance, and bind all running events 21 in the same user task or the same session window to the global association identifier 11.
[0066] The runtime event 21 includes at least one of the following: user input event, model inference event, tool call request and response event, retrieval-enhanced read and write event, and memory read and write event;
[0067] The tool call request and response events include the tool identifier, a summary of the call parameters, and the call result status.
[0068] The data collection process is executed by a trajectory acquisition agent deployed in the agent's operating environment. This agent captures various events generated during the agent's operation by interception or subscription.
[0069] The collected runtime events 21 include at least one of the following types: user input events (recording the original questions or instructions initiated by the user to the agent), model reasoning events (recording the reasoning process performed by the large language model after receiving input, including the model identifier and the version of the prompt word template used), tool call request and response events (recording the entire process of the agent calling external tools, including the tool identifier, call parameter summary and call result status), retrieval enhancement reading and writing events (recording the context of the agent initiating a retrieval request to an external knowledge base and obtaining the results), and memory reading and writing events (recording the agent's access operations on long-term memory or short-term memory).
[0070] For ease of understanding, for example, a user sends the command "Help me check the status of order ORD-5567, and intercept the logistics if it has been shipped." During the operation of the intelligent agent, the trajectory acquisition agent will sequentially capture one user input event (corresponding to the above command), one model inference event (after the model analyzes the intent, it plans to call the "query order" and "intercept logistics" tools), two tool call request and response events (corresponding to the execution of the orders.get and logistics.block tools respectively), and one model inference event (after the model integrates the tool feedback, it generates the final response). All of the above events are bound to the same CID. This CID accompanies the entire task lifecycle, running through multiple stages such as model inference, tool calls, and RAG retrieval, enabling heterogeneous events that were originally scattered in different log files or storage locations to be traced back and aggregated based on the same identifier.
[0071] Step S2: Convert the heterogeneous format of the runtime event 21 into a standard audit event 22 that conforms to a preset schema to obtain a sequence of standard audit events 22.
[0072] The standard audit event 22 includes at least the event type, timestamp, global association identifier 11, agent instance 10 identifier, policy context version number, and a predecessor event reference representing causal order.
[0073] When responding to an audit query request, the decision timeline data is restored according to the preceding event references and used as the response content for the audit query request.
[0074] The raw events generated by the agent during operation originate from various components or services, with different log formats, field names, and data structures. A semantic normalization module is needed to parse, map, and transform these raw events, removing redundant fields and adding necessary context fields to ensure all events conform to the same schema specification.
[0075] The defined standard audit event 22 includes the following fields:
[0076] Event type: Mark the event as one of the following categories: USER_INPUT, LLM_INFER, TOOL_CALL, RAG_QUERY, or MEMORY_ACCESS.
[0077] Timestamp: Accurately records the moment an event occurred, used for subsequent timeline sorting.
[0078] Global Association Identifier 11: This is the global association identifier 11CID generated in step S1, which ensures that events within the same session can be aggregated.
[0079] Agent Instance 10 Identifier: Identifies which specific agent deployment instance the event originated from, facilitating retrieval and statistics at the instance level.
[0080] Policy Context Version Number: Records the baseline version of the policy followed by the agent when the event occurs, which is used to accurately load the corresponding constraint rules during post-event replay.
[0081] Precursor event references, representing causal order, are key fields for reconstructing the decision timeline. Each standard audit event 22 uses this field to point to its directly dependent predecessor event (i.e., the upstream event ID that triggered the current event), thus establishing a clear causal chain between events. For serial inference-tool call chains, this reference is presented as a singly linked list; for scenarios where a single round of model inference triggers multiple parallel tool calls, multiple tool call events can share the same predecessor event reference, forming a directed acyclic graph structure.
[0082] Step S3: Obtain baseline description information 30 corresponding to the instance of the agent and the version number of the policy context currently used; match the standard audit event 22 sequence with the baseline description information 30; and output the comparison result 31 and the category deviation item.
[0083] The baseline description information 30 includes at least one of the following: allowed tool set, prohibited data access mode, prompt constraint summary, or maximum number of autonomous call steps;
[0084] The categorized deviations include at least one of the following: the tool identifier is not in the allowed tool set, the number of tool calls in a single session exceeds the maximum number of autonomous call steps, and the call parameters match prohibited data access patterns.
[0085] When an agent is deployed, it typically defines a set of specifications or constraints to limit the boundaries of its executable behavior during runtime. These constraints, known as baseline description information 30, include at least one of the following types: a whitelist of permitted tools (restricting the agent to only call approved external tools), prohibited data access patterns (using regular expressions or pattern rules to restrict sensitive data that cannot be shared externally, such as ID card numbers and bank card numbers), a summary of cue constraint (boundary constraints on the content generated by the model, such as requiring the inclusion of disclaimers or prohibiting the generation of medical diagnostic conclusions), and a maximum number of autonomous call steps (limiting the maximum number of tool calls that the agent can execute within a single session or task). The baseline description information 30 is stored together with the agent's deployment artifact version (including model version, tool list, and cue word template) and identified and associated with it through a policy context version number.
[0086] During the matching process, the standard audit event 22 sequence is cross-validated against baseline constraints either individually or in batches. For events that do not meet the constraints, corresponding categorized deviations are output based on the specific rule type violated. Deviations include at least the following types: the tool identifier is not in the allowed tool set (e.g., the agent calls admin.users.delete, but the tool is not in the baseline whitelist), the number of tool calls in a single session exceeds the maximum number of autonomous call steps (e.g., the baseline specifies a maximum of 10 tool calls per session, but the current session has already triggered the 11th call), and the call parameters match prohibited data access patterns (e.g., the parameters of the tool call contain a string that matches an ID number regular expression, which is judged as an attempt to transmit sensitive data).
[0087] Please see the appendix Figure 3 Matching the standard audit event 22 sequence with the baseline description information 30 includes:
[0088] Step S31: Use a deterministic rule layer to perform threshold detection on the tool whitelist, the maximum number of calls, or the target host whitelist. This step handles constraints with clear boundary conditions, typically implemented using a HashSet or a numerical counter.
[0089] Step S32: Perform regular expression matching or sensitive information identification model matching on the tool parameter summary using the pattern rule layer. This process checks whether the tool call parameters contain sensitive data patterns. For example, in a corporate knowledge base agent scenario, the baseline specifies that ID card numbers are prohibited from being passed in parameters (matching the regular expression \d{17}[\dXx]). When the parameter summary of the tool call event contains a string that matches this regular expression pattern, the pattern rule layer identifies the deviation and outputs a deviation item of type "DENY_DATA_PATTERN_MATCHED", while recording the specific pattern and parameter location for subsequent review.
[0090] Step S33: Use a semantic rule layer to compare the input or output summary of the model reasoning event with the hint constraint summary.
[0091] This approach addresses detection scenarios involving whether the model's input and output content violates cue constraints. Since the input and output of model inference are typically large in volume and exist in natural language form, they are difficult to determine directly using deterministic rules. Therefore, summary comparison or integration with external content security models is employed for detection.
[0092] For example, in a medical consultation-assisted intelligent agent scenario, the baseline stipulates that the model output must include a disclaimer template, the hash value of which has been pre-entered into the baseline description information 30. When the semantic rule layer detects that the output digest hash of a certain LLM_INFER event does not match the pre-set disclaimer digest hash in the baseline, it is determined as a "PROMPT_CONSTRAINT_SUSPECTED" deviation.
[0093] Step S34: Use a time-series rule layer to maintain a cross-event state machine and detect cross-step time-series attack patterns.
[0094] To handle detection scenarios involving abnormal logical sequences between multiple events, it is necessary to maintain cross-event state information to identify complex attack patterns. For example, in an enterprise internal system intelligent agent scenario, the baseline defines a timing rule: prohibiting the immediate execution of a deletion operation after querying someone else's information. The timing rule layer maintains a lightweight state machine. When it detects that the same CID has consecutive `users.query` events (querying someone else's information) and `admin.user.delete` events (deleting a user), and both events' parameters point to the same target user, even if the individual tool calls themselves are within the whitelist, the state machine still outputs a deviation item of the type "TEMPORAL_PATTERN_ABUSE," marking this cross-step combination behavior as a suspicious attack pattern.
[0095] Through the hierarchical matching of the above four steps, various deviation behaviors in the standard audit event 22 sequence are structured into categorized deviation items and attached to the corresponding events. These deviation items, along with the event body, will be stored in the integrity protection section of step S4 to ensure that they cannot be tampered with or deleted during subsequent review.
[0096] Step S4: Write the standard audit event 22, the deviations in the comparison result 31, and the integrity verification value 32 into the audit storage area in chronological order.
[0097] The audit storage area employs an append-only write strategy, meaning that each new record is only appended to the end of existing data, and written records cannot be modified or deleted. During writing, the standard audit event 22 ontology and the corresponding deviation (if any) are committed as a complete record, and an integrity check value 32 is calculated and stored together for this record.
[0098] The integrity check value 32 is calculated based at least on the load, timestamp, and preceding events of the standard audit event 22.
[0099] The calculation of the integrity check value 32 is based on at least the following three types of inputs: the complete payload of the current standard audit event 22 (including all fields such as event type, timestamp, global association identifier 11, agent instance 10 identifier, policy context version number, and predecessor event reference; deviations, if any, are also included in the calculation), the timestamp of the current event, and the integrity check value 32 of the preceding event. Specifically, the integrity check value 32 of the first event is calculated using a pre-agreed initial value (such as an all-zero hash value) as the preceding input; for the Nth event, its integrity check value 32 is the result of performing a cryptographic hash operation (such as SHA-256) on the concatenation of the current event payload, the timestamp, and the integrity check value 32 of the (N-1)th event.
[0100] The chained hash structure gives the audit storage area tamper-proof properties. If the content of any written event is modified, the integrity check value 32 of that event will change, causing all subsequent events to calculate results based on the "integrity check value 32 of the preceding event" to be inconsistent with the original stored value.
[0101] Step S5: Respond to the audit query request, retrieve and return the standard audit event 22 sequence and associated deviations from the audit storage area according to the global association identifier 11.
[0102] Audit query requests are typically initiated by auditors, compliance review specialists, or security operations personnel during post-incident investigations or regulatory evidence gathering. Query criteria must include at least the target CID, and may also include filters such as time range, agent instance identifier, or event type to precisely locate the desired data.
[0103] The audit storage area supports efficient indexing and retrieval based on CIDs. The retrieval operation returns all standard audit events 22 bound to the CID, starting from the first user input event of the session, sequentially passing through each round of model inference events, tool call request and response events, RAG read / write events, and memory read / write events, until the final response event, forming a complete list of audit events. Simultaneously, deviations, because they are written to storage along with their corresponding events in step S4, are also returned along with their bound events during retrieval.
[0104] The returned content is not limited to a list of the original events. It can also be further reorganized into a causal tree or timeline structure at the retrieval end based on the preceding events recorded in step S2, and then returned. On this basis, auditors can directly obtain the complete audit log of the session corresponding to the CID, including the time distribution of normal and deviant behaviors, the order of tool calls, the causal dependencies between events, and the type and severity of deviations, without having to repeatedly switch between multi-source heterogeneous logs and manually piece them together.
[0105] On the other hand, in another implementation, standard audit events 22 corresponding to multiple global association identifiers 11 are aggregated within a preset time window, risk patterns that meet preset association rules are identified and risk alarms are output, and in response to the operation of the risk alarm, the decision timeline view of the corresponding global association identifier 11 is drilled down.
[0106] From a single-session or single-instance perspective, certain abnormal behaviors are often not significant. Calling an order query tool once within a session is a compliant operation, and a model output containing specific wording once within a session may not constitute a violation. However, when such behaviors occur frequently within a short period, are scattered across different sessions but point to the same data target, or originate from the same call source, they may constitute attack patterns such as data scraping, injection probing, or permission overreach attempts. Conventional single-point log retrieval or single-trace alert mechanisms can only examine the event sequence of each CID in isolation, failing to capture the aforementioned dispersed risks across sessions on a global scale. Therefore, based on the obtained normalized standard audit event 22, it is necessary to perform aggregated statistical analysis and correlation analysis on events from multiple CIDs within a preset time window.
[0107] Specifically, the aggregation analysis process uses a preset time window as the basic unit, such as 10 minutes, 30 minutes, or 1 hour, and performs multi-dimensional statistics on all standard audit events 22 falling within the window in a sliding or scrolling manner. The aggregation dimensions include, but are not limited to: event type (such as TOOL_CALL, LLM_INFER, RAG_QUERY), tool identifier (such as orders.get, admin.user.delete), agent instance 10 identifier, source IP range, or user identifier, etc. Preset association rules define the threshold conditions and combination logic for triggering alarms. For example, if the number of calls to a sensitive tool (such as admin.user.delete) by the same agent instance 10 within a 10-minute window exceeds a set threshold, or if calls to orders.get occur in different CIDs triggered within a 30-minute window from the same source IP range and the number of user IDs involved exceeds a set threshold, or if similar regular expression patterns (indicating injection characteristics) appear in the input summaries of LLM_INFER events across multiple CIDs, this indicates a potential injection vulnerability.
[0108] On the other hand, this embodiment provides an intelligent agent operation trajectory auditing system, please refer to the appendix. Figure 4 ,include:
[0109] The acquisition module 100, during the operation of the intelligent agent, acquires at least one type of running event 21, generates a global association identifier 11 for the same intelligent agent running instance, and binds all running events 21 in the same user task or the same session window to the global association identifier 11;
[0110] The conversion module 200 converts the heterogeneous format of the runtime event 21 into a standard audit event 22 that conforms to a preset schema, thereby obtaining a sequence of standard audit events 22.
[0111] The matching module 300 obtains baseline description information 30 corresponding to the instance of the agent and the version number of the policy context currently used, matches the standard audit event 22 sequence with the baseline description information 30, and outputs the comparison result 31 and the category deviation item.
[0112] The storage module 400 writes the standard audit event 22, the deviations in the comparison result 31, and the integrity verification value 32 into the audit storage area in chronological order.
[0113] The audit module 500 responds to the audit query request and retrieves and returns the standard audit event 22 sequence and associated deviations from the audit storage area based on the global association identifier 11.
[0114] Please see Figure 5 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.
[0115] like Figure 5 As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 can be used to connect and communicate with the various components mentioned above. The user interface 1103 may include buttons, and optionally may include standard wired or wireless interfaces. The network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, or a Wi-Fi module. The processor 1101 may include one or more processing cores. The processor 1101 connects to various parts within the electronic device 1100 using various interfaces and lines, and performs various functions of the routing device and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1105, and by calling data stored in the memory 1105. Optionally, the processor 1101 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or more combinations of CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content that the display screen needs to show; and the modem is used for wireless communication.
[0116] It is understandable that the aforementioned modem may not be integrated into the processor 1101, but may be implemented using a separate chip.
[0117] The memory 1105 may include RAM or ROM. Optionally, the memory 1105 may include a non-transitory computer-readable medium. The memory 1105 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1105 may also be at least one storage device located remotely from the aforementioned processor 1101. As a computer storage medium, the memory 1105 may include an operating system, a network communication module, a user interface module, and application programs. The processor 1101 may be used to call the application programs stored in the memory 1105 and execute the methods in the above-described embodiments.
[0118] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform multiple steps as described in the above embodiments. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.
[0119] This specification also provides a computer program product, including a computer program that, when executed by a processor, implements the multiple steps described in the above embodiments.
[0120] Where there is no conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.
[0121] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center integrating multiple available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., digital versatile discs (DVDs)), or semiconductor media (e.g., solid-state drives (SSDs)).
[0122] When implemented through hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and achieve the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, whose logic function is determined by the user programming the device. Designers can program a digital system onto a PLD themselves, eliminating the need for chip manufacturers to design and fabricate dedicated integrated circuit chips. Furthermore, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, similar to the software compiler used in program development. The original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There is not just one HDL, but many. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of the aforementioned hardware description languages and programming it into an integrated circuit, the hardware circuit implementing the logic method flow can be easily obtained.
[0123] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.
Claims
1. A method for auditing the operational trajectory of an intelligent agent, characterized in that, Including the following steps: During the operation of the intelligent agent, at least one type of running event is collected, and a global association identifier is generated for the same intelligent agent running instance. All running events within the same user task or the same session window are bound to the global association identifier. The heterogeneous runtime events are converted into standard audit events conforming to a preset schema to obtain a standard audit event sequence; Obtain baseline description information corresponding to the instance of the agent and the version number of the policy context currently used; match the standard audit event sequence with the baseline description information; and output the comparison results and categorical deviation items. The standard audit events, deviations in the comparison results, and integrity verification values are written into the audit storage area in chronological order. In response to an audit query request, the system retrieves and returns the standard audit event sequence and associated deviations from the audit storage area based on the global association identifier.
2. The method for auditing the trajectory of an intelligent agent according to claim 1, characterized in that, The standard audit event includes at least the event type, timestamp, global association identifier, agent instance identifier, policy context version number, and a reference to the preceding event representing the causal order. When responding to an audit query request, the decision timeline data is restored according to the preceding event references and used as the response content for the audit query request.
3. The method for auditing the trajectory of an intelligent agent according to claim 1, characterized in that, The integrity check value is calculated based at least on the standard audit event payload, timestamp, and preceding events.
4. The method for auditing the trajectory of an intelligent agent according to claim 1, characterized in that, The runtime events include at least one of user input events, model inference events, tool call request and response events, retrieval-enhanced read and write events, and memory read and write events; The tool call request and response events include the tool identifier, a summary of the call parameters, and the call result status.
5. The method for auditing the trajectory of an intelligent agent according to claim 1, characterized in that, The baseline description information includes at least one of the following: allowed toolset, prohibited data access modes, summary of cue constraints, or maximum number of autonomous call steps. The categorized deviations include at least one of the following: the tool identifier is not in the allowed tool set, the number of tool calls in a single session exceeds the maximum number of autonomous call steps, and the call parameters match prohibited data access patterns.
6. The method for auditing the trajectory of an intelligent agent according to claim 5, characterized in that, Matching the standard audit event sequence with the baseline description information includes: A deterministic rule layer is used to perform threshold detection on tool whitelists, maximum number of calls, or target host whitelists; A pattern rule layer is used to perform regular expression matching or sensitive information identification model matching on the tool parameter summary; A semantic rule layer is used to compare the input or output summary of the model's reasoning events with the hint constraint summary; A time-series rule layer is used to maintain a cross-event state machine to detect cross-step time-series attack patterns.
7. The method for auditing the trajectory of an intelligent agent according to claim 1, characterized in that, Within a preset time window, standard audit events corresponding to multiple globally associated identifiers are aggregated, risk patterns that meet preset association rules are identified and risk alarms are output, and in response to the operation of the risk alarm, the decision timeline view of the corresponding globally associated identifier is drilled down.
8. A system for auditing the trajectory of an intelligent agent, characterized in that, include: The acquisition module collects at least one type of running event during the operation of the intelligent agent, generates a global association identifier for the same intelligent agent running instance, and binds all running events within the same user task or the same session window to the global association identifier. The conversion module converts the heterogeneous runtime events into standard audit events that conform to a preset schema, thereby obtaining a standard audit event sequence; The matching module obtains baseline description information corresponding to the instance of the agent and the version number of the policy context currently used, matches the standard audit event sequence with the baseline description information, and outputs the comparison results and classification deviation items. The storage module writes the standard audit events, deviations in the comparison results, and integrity verification values into the audit storage area in chronological order. The audit module responds to audit query requests by retrieving and returning the standard audit event sequence and associated deviations from the audit storage area based on the global association identifier.
9. An electronic device, characterized in that, Including the processor and memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code stored in the memory to perform the method as described in any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.