A non-intrusive model behavior monitoring and guidance method and apparatus

Abstract

The application provides a non-intrusive model behavior monitoring and guiding method and device, belonging to the technical field of artificial intelligence; the method intercepts and analyzes tool call requests initiated by the model side in real time through an independent inspector deployed between the model context protocol client and the server, to determine whether the behavior of the model is unhealthy behavior; the determination method includes: generating an action fingerprint based on the tool call request, and comparing the historical action fingerprint sequence with a preset rule to identify patterns such as repeated calls or alternating deadlocks; or determining semantic stagnation by calculating the semantic similarity of the results of continuous tool calls. When it is determined to be unhealthy behavior, the independent inspector silently injects structured guiding information into the model, for example, the guiding information is used as a system prompt or is attached to the tool call result, to guide the model to correct its subsequent behavior.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a behavior monitoring and guidance technology for large language models interacting with external tools and data sources, which is particularly suitable for intelligent agent systems based on model context protocols. Background Technology

[0002] With the widespread adoption of large language model technology, its use in intelligent agent systems to invoke external tools to complete complex tasks has become a mainstream application paradigm. Model context protocols, as a standardized communication protocol, are commonly used to connect models with external tools or data sources. However, current behavior enhancement schemes for large language models with limited computing power and logical reasoning capabilities have significant limitations.

[0003] One mainstream approach is to strictly constrain the model's tool-calling behavior through predefined graph structures or workflows. However, this approach lacks flexibility, cannot adapt to dynamic behavior changes in complex scenarios, and has high development and maintenance costs. Another approach relies on the model's own powerful logical reasoning capabilities for self-verification and correction. However, this approach places extremely high demands on model performance, cannot adapt to large language models with weak logical capabilities, and the reflection process increases response latency, making real-time control difficult.

[0004] Furthermore, existing technologies employ non-intrusive monitoring approaches, such as using independent middleware or "sidecar" services to track and constrain the interaction between the model and the external environment. However, these solutions typically only provide a basic monitoring framework and lack specific, efficient, and lightweight mechanisms for identifying abnormal behavior. They struggle to identify, in real-time and accurately, models trapped in ineffective behavior patterns such as "repeated calls" or "standing still" without complex semantic analysis. Moreover, after anomalies are detected, there is a lack of a non-destructive intervention mechanism that is friendly to weak language models and does not cause logical interruption due to direct error reporting. Therefore, existing technologies for behavioral control of weak language models generally suffer from poor adaptability, lack of real-time performance, unstable control effects, and high system coupling. Summary of the Invention

[0005] The purpose of this application is to provide a non-intrusive model behavior monitoring and guidance method and device to solve the technical problem that existing technologies lack a technical solution that can balance real-time performance, non-intrusiveness, efficiency and adaptability when controlling the tool calling behavior of large language models with limited computing power and logic capabilities. In other words, it is impossible to intervene in advance and effectively guide the model's potentially invalid or erroneous behavior without modifying the model or tool itself.

[0006] To achieve the above objectives, this application provides a non-intrusive model behavior monitoring and guidance method. The method includes: intercepting Model Context Protocol (MTP) messages transmitted between the client and server in real time via an independent supervisor deployed between the client and server, wherein the MTP messages include at least a tool invocation request initiated by the model; the independent supervisor analyzing the tool invocation request to determine whether the model's behavior is unhealthy; and when the model's behavior is determined to be unhealthy, the independent supervisor injecting structured guidance information into the model to guide the model to correct its subsequent behavior.

[0007] Furthermore, the step of analyzing the tool call request to determine whether the model's behavior is unhealthy includes: generating an action fingerprint identifying the call based on the method name and parameters in the tool call request; adding the currently generated action fingerprint to a historical action fingerprint sequence; comparing the historical action fingerprint sequence with preset unhealthy behavior pattern rules to determine whether the model's behavior is unhealthy, and if it is unhealthy, determining its corresponding unhealthy behavior type.

[0008] Optionally, the method for generating the action fingerprint is to perform a hash operation on the method name and parameters.

[0009] Optionally, the unhealthy behavior pattern rule includes: detecting the occurrence of the same action fingerprint multiple times consecutively.

[0010] Optionally, the unhealthy behavior pattern rule includes: detecting two or more different action fingerprints that periodically alternate.

[0011] As one embodiment of this application, the step of analyzing the tool call request to determine whether the model's behavior is unhealthy further includes: intercepting the tool call result returned by the server corresponding to the tool call request; caching the intercepted multiple consecutive tool call results; determining their semantic similarity by calculating the cosine similarity between the text vectors of the cached multiple consecutive tool call results; and determining the unhealthy behavior type when the semantic similarity is consistently higher than a preset threshold.

[0012] Optionally, the step of injecting structured guidance information into the model specifically includes: attaching the structured guidance information as a system prompt to the next communication message sent to the model.

[0013] Optionally, the method further includes: intercepting the tool invocation result returned by the server corresponding to the tool invocation request; the step of injecting structured guidance information into the model specifically includes: appending the structured guidance information to the tool invocation result to form an enhanced message, and sending the enhanced message to the model.

[0014] Furthermore, the method also includes: pre-setting a strategy library, which stores the correspondence between unhealthy behaviors and structured guidance information; in the step of injecting structured guidance information into the model, according to the determined type of unhealthy behavior, the corresponding structured guidance information is retrieved from the strategy library and injected.

[0015] This application also provides a non-intrusive model behavior monitoring and guidance device, comprising: a capture module configured to intercept Model Context Protocol (MGP) messages transmitted between a Model Context Protocol client and a server in real time, wherein the MGP messages include at least a tool call request initiated by the model; a determination module connected to the capture module, configured to obtain the MGP messages from the capture module and analyze the tool call requests therein to determine whether the model's behavior is unhealthy; and an injection module connected to the determination module, configured to inject structured guidance information into communications sent to the model when the determination module determines that the model's behavior is unhealthy, to guide the correction of the model's behavior.

[0016] Compared with existing technologies, the technical solution provided in this application has the following advantages: First, this application has the advantages of being non-intrusive and highly adaptable. It achieves this by deploying an independent supervisor on the model context protocol communication channel, without requiring modification of existing large language models, agent logic, or tool interfaces, thus achieving plug-and-play functionality and universality across various models, reducing system coupling and maintenance costs. Second, this application achieves lightweight and high efficiency. By processing tool call requests with "action fingerprinting" and using rule-based state machines for behavior determination, it avoids complex full semantic analysis and long context recording, resulting in extremely low computational overhead and enabling real-time monitoring and intervention of model behavior. Finally, this application enables precise and real-time control, transforming the control of model behavior from "post-event remediation" to "in-process monitoring and early intervention." It identifies and guides models in the early stages of ineffective loops, injecting structured guidance information into communication to proactively guide the model towards the correct path, effectively avoiding waste of computational resources and business risks, and achieving precise and stable behavior control. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of the logical architecture of a non-intrusive model behavior monitoring and guidance method provided in an embodiment of this application;

[0019] Figure 2 A three-way interaction timing diagram for a non-intrusive model behavior monitoring and guidance method provided in an embodiment of this application;

[0020] Figure 3 A flowchart illustrating a non-intrusive model behavior monitoring and guidance method provided in this application embodiment;

[0021] Figure 4 This is a schematic diagram of a non-invasive model behavior monitoring and guidance device provided in an embodiment of this application.

[0022] The main reference numerals in the attached figures are explained as follows: 10-Capture module; 20-Judgment module; 21-Fingerprint generation unit; 22-Rule matching unit; 30-Injection module; S301-Intercepting MCP message step; S302-Generating action fingerprint step; S303-Judging whether the behavior is abnormal step; S304-Injecting guidance information step; S305-Passing message step. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] In current artificial intelligence technologies, large language models, especially weak language models with limited computing resources and logical reasoning capabilities, often exhibit uncertain behavior when performing complex tasks that require interaction with the external world. These models, as the decision-making core of an agent, invoke external tools (such as application programming interfaces or database queries) to obtain information or perform operations. Model context protocols, as a standardized communication framework, support data exchange between the model and external tools. However, when faced with unexpected situations such as permission errors, no results returned, or logical dilemmas, weak language models are prone to falling into ineffective behavioral patterns, such as repeatedly trying the same failed operation, looping between several ineffective steps, or stagnating in a state where the calling parameters change slightly but there is no substantial semantic progress.

[0025] To address the aforementioned issues, existing technical solutions have limitations. For example, pre-programmed fixed workflows restrict the flexibility of the model; while solutions relying on model self-reflection place excessive demands on the model's capabilities, making them unsuitable for weak language models. Therefore, this application provides a non-intrusive model behavior monitoring and guidance solution. This solution aims to monitor the model's behavior in real time through an independent middleware without modifying the model itself or the tools it calls. When the model exhibits unhealthy trends, it proactively and precisely injects guidance information to help correct the behavior, thereby improving the success rate and efficiency of task execution.

[0026] Please see Figure 1 This diagram illustrates the logical architecture of a non-intrusive model behavior monitoring and guidance method provided in this application. The core of this architecture is an independently deployed supervisor, which acts as middleware between the large language model and the model context protocol server. Specifically, the supervisor can be logically divided into multiple layers, including a behavior capture layer, a behavior state machine, and an intervention injection layer. The behavior capture layer is responsible for intercepting and parsing all communication messages between the two. The behavior state machine, as the core of analysis and judgment, receives data from the capture layer and performs anomaly judgment. It should be noted that this judgment process can be based on a preset rule base (e.g., detecting duplicate calls or alternating deadlocks) or implemented through lightweight semantic analysis. When the behavior state machine determines that the model behavior is abnormal, the intervention injection layer is activated. It generates or matches a corresponding guidance strategy and injects this strategy into the communication stream sent to the large language model, thereby influencing and guiding the model's subsequent behavior.

[0027] Please see Figure 4 The diagram illustrates a structural schematic of a non-invasive model behavior monitoring and guidance device provided in an embodiment of this application. This device can be considered as... Figure 1The physical or logical entity implementation of the illustrated logical architecture. Specifically, the device may include: a capture module 10, which functions as a behavior capture layer to intercept communication messages between the model and external tools in real time; a determination module 20 connected to the capture module 10, which functions as a behavior state machine and is responsible for analyzing the captured messages and determining the health of the model's behavior; and an injection module 30 connected to the determination module 20, which functions as an intervention injection layer. When the determination module 20 outputs an abnormal determination signal, the injection module 30 is responsible for performing the injection operation of guidance information. As an optional implementation, the determination module 20 may further include: a fingerprint generation unit 21 for characterizing model actions, and a rule matching unit 22 for comparing action feature sequences with preset abnormal patterns.

[0028] The technical solutions in the embodiments of this application will be described in more detail below with reference to the accompanying drawings.

[0029] Example 1

[0030] One embodiment of this application details how to monitor and guide a common unhealthy behavior: repeated invalid calls. In this scenario, a large language model with weak logical capabilities, after failing to execute a tool call, will repeatedly attempt the same operation because it fails to understand the root cause of the failure. This embodiment aims to demonstrate how to efficiently identify such behavior by generating "action fingerprints" and inject guiding information through system prompts to help the model escape this predicament.

[0031] Please combine Figure 2 and Figure 3 To understand. Figure 2 The interaction sequence among the weak language model, the independent supervisor, and the model context protocol server is demonstrated. Figure 3 This demonstrates a general process for the method described in this application.

[0032] In a specific application scenario, an agent is tasked with "reading and analyzing the contents of the configuration file config.json". This agent is driven by a weak language model. However, the user account currently running the agent does not have permission to read this file.

[0033] The execution process of the method is as follows:

[0034] Step 1: The model initiates its first tool call. Based on the task instructions, the weak language model internally plans the first step as reading a file. Accordingly, it constructs a tool call request conforming to the model context protocol. This request is a JSON-formatted message sent over the network to the model context protocol server, the content of which is shown in Table 1:

[0035]

[0036] Table 1 is a table of JSON format code blocks.

[0037] Step Two: The independent supervisor performs behavior capture and initial analysis. The tool's call request is intercepted by the independent supervisor deployed on the communication link before reaching the Model Context Protocol server; this corresponds to... Figure 3 Step S301 (Intercepting the MCP message). The capture module 10 inside the inspector parses the JSON message and identifies it as a request of type tools / call.

[0038] Subsequently, the determination module 20 begins its operation. Specifically, its internal fingerprint generation unit 21 extracts information that is crucial to identifying this call behavior, namely the method name `read_file` and the parameter `{"path": "config.json"}`. To achieve lightweight and efficient comparison, the fingerprint generation unit 21 performs a hash operation on this key information. For example, the method name and parameter (serialized into strings in a fixed order) can be concatenated and then a deterministic hash algorithm (such as SHA-256) can be applied to calculate: `hash_input = "read_file" + '{"path":"config.json"}'`

[0039] Action_ID = SHA256(hash_input). This generates a unique, fixed-length string, which serves as the "action fingerprint" for this call, denoted as Action_ID_A. This process corresponds to... Figure 3 Step S302 (generating action fingerprint).

[0040] The determination module 20 maintains a historical action fingerprint sequence, which can be a fixed-length queue (e.g., storing only the fingerprints of the most recent 10 calls). The determination module 20 adds the newly generated Action_ID_A to this sequence. At this time, the rule matching unit 22 checks whether the sequence matches any preset unhealthy behavior pattern rules (step S303, determining whether the behavior is abnormal). Since this is the first call, there is only one element in the historical sequence, which does not satisfy any rules such as "continuous repetition," therefore the determination module 20 determines the current behavior as "healthy."

[0041] Step 3: Message Transmission and Tool Execution. Since the behavior is deemed healthy, the supervisor performs a "transmission" operation, forwarding the original tool call request intact to the model context protocol server. This corresponds to... Figure 3Step S305 (transmit message) in the process.

[0042] After receiving the request, the Model Context Protocol (MCP) server executes the `read_file` method of the `file_system` tool. During execution, a permission check is performed at the operating system level, and it is found that the current user does not have permission to read the `config.json` file, so the operation fails. The MCP server then constructs a tool call result message (`tool_result`) containing error information and sends it back. An example of the `tool_result` message content is shown in Table 2.

[0043]

[0044] Table 2 shows the code for the tool_result message content.

[0045] The `tool_result` message was also intercepted by the independent supervisor on its way back to the weak language model. The supervisor recognized it as a `tool_result` message and, since it was not currently in an intervention state, also passed it directly to the weak language model.

[0046] Step 4: The model initiates a repeat request. The weak language model receives an error message containing "Permission Denied". Due to its limited logical reasoning ability, it may not be able to correctly understand the deeper meaning of "permission denied" or associate it with strategies such as "changing tools" or "checking its own identity". Driven by its inherent and potentially simple retry logic, the model decides to try the same operation again, and thus generates and sends a second tool call request with the exact same content as the first request.

[0047] Step 5: The supervisor detects unhealthy behavior. This duplicate request is intercepted again by the independent supervisor (step S301). The fingerprint generation unit 21 of the judgment module 20 performs the same hash operation on the method name and parameters in the request again. Since the input is exactly the same, it will inevitably generate the same action fingerprint Action_ID_A as the first time (step S302).

[0048] The determination module 20 compares this new Action_ID_A with the historical action fingerprint sequence. Assuming the previous Action_ID_A is already stored in the historical sequence, and the same fingerprint is detected for the second consecutive time, a "repeated call rule" in the rule matching unit 22 is triggered. This rule can be defined as: "When the same action fingerprint is detected N times consecutively (N≥2), the model behavior is determined to be unhealthy." In this example, N can be set to 2. Therefore, the determination module 20 determines the current model's behavior to be "unhealthy" and marks its type as "repeated call" (step S303 determines "yes").

[0049] Step Six: Intervention Injection. Once the determination module 20 determines that the behavior is unhealthy, it will activate the injection module 30, which corresponds to... Figure 3 Step S304 (Injecting Guidance Information). The injection module 30 first needs to determine the content to be injected. To do this, it accesses a preset guidance strategy library. This strategy library stores the correspondence between different types of unhealthy behaviors and structured guidance information.

[0050] In one specific embodiment of this application, the policy library can be a key-value pair storage structure, where the key is the behavior type and the value is a guiding text template. For example: {"repetitive_call": "Repetitive operation detected. It is recommended to check the parameters or change the tool.", "alternating_deadlock": "Looping between operation A and operation B detected. Please check if the preconditions are met."} Furthermore, the policy library can be more context-aware. That is, the injection module 30 can perform more precise policy matching not only based on the "repetitive call" type, but also by combining the content of the most recent failed tool_result (such as the keyword "Permission Denied"). For example, the policy library might contain a more specific rule like this: {"type":"repetitive_call","error_keyword":"PermissionDenied","hint": "It is recommended to first call get_user_info to check the current permission level, or use list_directory to find other readable files."}. The injection module 30 retrieves the most matching structured bootstrap information from the policy library based on the current situation (type: "repetitive call", error message context: "Permission Denied").

[0051] Next, injection module 30 performs "silent injection." In this embodiment, it chooses to package the guidance information into a system prompt. It intercepts the next communication message about to be sent to the weak language model, which may be the tool_result returned by the second failed call. Injection module 30 modifies this message or creates a new enhancement message to which the guidance information is appended. For example, it appends the guidance information as a special text to the original tool call result to form an enhancement message, and then sends the enhancement message to the model, as shown in Table 3:

[0052]

[0053] Table 3 shows the model code table.

[0054] Step 7: Model Behavior Correction. The weak language model receives this tool_result injected with guiding information. While it may not be able to infer the correct course of action from "Permission Denied," the text in the [System Hint] provides it with explicit, specific, and actionable suggestions for the next step. When processing this text containing explicit instructions, the model's internal planner is highly likely to adopt these suggestions. Therefore, in its next planning step, the model will abandon continuing to call read_file and instead generate a new tools / call request to invoke get_user_info or list_directory.

[0055] Through the above steps, this embodiment successfully monitored the model's repetitive and ineffective behavior in real time without changing the model and tools, and proactively provided effective guidance to help the model break the deadlock, allowing the task to continue. This fully demonstrates the beneficial effects of the non-intrusive, lightweight, and precise control of the proposed solution.

[0056] Example 2

[0057] This embodiment demonstrates a more advanced capability for detecting unhealthy behaviors: "semantic stagnation" detection. This situation is more insidious than simple repeated calls. The model may use slightly different parameters each time the tool is invoked, resulting in different hash-based action fingerprints that cannot be detected by the method in Embodiment 1. However, from a semantic perspective, the model may still be stuck, without making substantial progress. This embodiment describes how to identify such problems by performing lightweight semantic analysis on the tool invocation results.

[0058] The system architecture of this embodiment is basically the same as that of Embodiment 1, and can be referred to accordingly. Figure 1 and Figure 4The main difference lies in the internal functions of the determination module 20. In addition to the fingerprint generation unit 21 and the rule matching unit 22, the determination module 20 also integrates a semantic similarity calculation unit.

[0059] Scenario: A weak language model is tasked with "writing a segmented summary of a 5000-word article." The model has a design flaw; its logic for segmenting text is incomplete, causing it to consistently process the beginning of the article.

[0060] The execution process of the method is as follows:

[0061] Step 1: The model initiates multiple summary calls consecutively. The model then begins executing the task.

[0062] First call: Extracts the first 300 characters of the article and calls the summarize_chunk tool. tools / call (text=document [0:300]) After the server executes the model context protocol, it returns a tool_result, which is a summary of the first paragraph, denoted as Summary_1.

[0063] Second call: Due to an error in its internal pagination logic, the model may have only slightly moved the window, capturing characters 10 to 310 of the article, and called the summarize_chunk tool again.

[0064] The tools / call (text=document [10:310]) model context protocol server returned another summary, denoted as Summary_2.

[0065] Because the input text content highly overlaps, Summary_2 and Summary_1 are semantically very similar.

[0066] The third call: The model again incorrectly truncated the first 305 words of the article. tools / call(text=document [0:305]) The model context protocol server returned a summary Summary_3, the content of which is still very close to the previous two summaries in semantics.

[0067] Step Two: The supervisor captures and analyzes the data. In the above process, the independent supervisor's capture module 10 (such as...) Figure 4 As shown, it intercepted every tools / call request and the corresponding tool_result response.

[0068] Decision module 20 (e.g.) Figure 4 (As shown) These messages are analyzed. First, the fingerprint generation unit 21 (as shown) Figure 4 As shown, three different action fingerprints (Action_ID_C, Action_ID_D, Action_ID_E) were generated for these three calls because the text content in the params of each call was not exactly the same. Therefore, the detection based on the duplicate call rule would fail.

[0069] However, the semantic similarity calculation unit in the determination module 20 comes into play at this point. This unit caches the content field from the most recent K (e.g., K=3) tool_result messages, namely Summary_1, Summary_2, and Summary_3.

[0070] Step 3: Semantic Similarity Calculation and Determination. The semantic similarity calculation unit employs a lightweight text vectorization technique. For example, this unit can use a preloaded lightweight sentence vector model, such as a distilled sentence vector model. This model can map any text into a high-dimensional vector space, and semantically similar texts will have similarly close vectors in this space.

[0071] This unit sequentially inputs Summary_1, Summary_2, and Summary_3 into the sentence vector model, resulting in three vector representations: .

[0072] Next, it calculates the cosine similarity between these vectors. Cosine similarity is an indicator of the consistency of direction between two vectors, with values ​​ranging from -1 to 1. The closer the value is to 1, the more semantically similar the two vectors (i.e., the corresponding text) are. Its calculation formula is:

[0073] Assume the calculation results are as follows:

[0074] The judgment module 20 has a preset "semantic stagnation rule", which defines a similarity threshold (e.g., 0.95). Its content can be: "If the semantic similarity between the tool call results of M consecutive times (M≥2) is higher than the threshold 0.95, the model behavior is judged to be unhealthy and the type is 'semantic stagnation'."

[0075] In this example, the similarity calculation results for two consecutive times (0.98 and 0.97) are both higher than 0.95. Therefore, the rule is triggered, and the decision module 20 determines that the model has entered an unhealthy state of semantic stagnation (corresponding to...). Figure 3 Step S303 is determined to be "yes".

[0076] Step 4: Intervention Injection. Injection module 30 (e.g.) Figure 4 (As shown) is activated. It retrieves the corresponding guidance information from the policy library based on the "semantic stagnation" behavior type. The policy library may contain entries like: {"type": "semantic_stagnation","hint": "You seem to be repeating similar content in the summary. Please check your pagination or chunking logic and try processing the next part of the document."}

[0077] After receiving this guidance instruction, the injection module 30 uses a method similar to that in Example 1 to inject it as a system prompt into the next communication sent to the model (corresponding to...). Figure 3 (Step S304). For example, it can be appended after the third call to tool_result.

[0078] Step 5: Model Behavior Correction. After receiving this prompt containing a clear problem diagnosis and correction suggestions, the model's internal planner understands that "page turning or chunking logic" is the key issue. This guides the model to re-examine and adjust its strategy for handling long texts, thereby fixing logical flaws and correctly processing the subsequent parts of the article.

[0079] By employing this lightweight semantic analysis-based approach, the proposed solution can identify logical stagnation issues that are more subtle than simple repetition, further broadening the scope of its monitoring and guidance capabilities and demonstrating a higher level of intelligence.

[0080] Example 3

[0081] This embodiment aims to illustrate how the proposed solution addresses more complex unhealthy behavior patterns, namely "alternating deadlock." In this pattern, the model does not repeat a single operation, but switches back and forth between two or more different failed operations, forming a loop that cannot be broken on its own.

[0082] The system structure of this embodiment is the same as that of embodiment 1. The key difference is that the rule matching unit 22 of the determination module 20 contains detection logic for a specific sequence pattern.

[0083] Scenario: An agent's task is to update a user's profile on a website. This task requires two steps: first, logging in, and then updating the profile. The model fails to log in successfully for some reason (e.g., the saved password has expired), but its behavioral logic is flawed, causing it to loop ineffectively between "attempting to update profile" and "attempting to log in."

[0084] The execution process of the method is as follows:

[0085] Step 1: The model gets stuck in an alternating call loop. First action: The model optimistically assumes it is logged in and directly attempts to call the update_profile() tool. Since it is not logged in, the call fails, and the model context protocol server returns an error message such as "Error: Not Authenticated". The supervisor captures this call, generates an action pointer Action_ID_A, and stores it in the history sequence.

[0086] The second action: The model recognizes the need for login from the error message, so it calls the `login()` tool and uses its stored (but expired) credentials. Login fails, and the model context protocol server returns an error message such as "Error: Invalid Credentials". The supervisor captures this call, generates an action fingerprint `Action_ID_A`, and stores it in the history sequence. At this point, the history sequence is [Action_ID_A, Action_ID_B].

[0087] Third action: The model's logic failed to handle the specific failure reason of "incorrect password," and instead simply reverted to the original goal of the task, attempting to call update_profile() again. This call failed again due to unauthentication. The supervisor generated the same action fingerprint Action_ID_A as the first time. The history sequence was updated to [Action_ID_A, Action_ID_B, Action_ID_A].

[0088] Fourth action: The model again recognizes the need to log in, calls login() again with incorrect credentials, and fails again. The supervisor generates the same action fingerprint Action_ID_B as the second one. The history sequence is updated to [Action_ID_A, Action_ID_B, Action_ID_A, Action_ID_B].

[0089] At this point, the model has fallen into an alternating deadlock loop of ABAB.

[0090] Step 2: The inspector detects an alternating deadlock pattern. After the fourth action is captured and a fingerprint is generated, the rule matching unit 22 of the judgment module 20 performs pattern analysis on the updated historical action fingerprint sequence [A, B, A, B] (corresponding to...). Figure 3 Step S303).

[0091] The rule matching unit 22 pre-defines an "alternating deadlock rule." Specifically, the logic of this rule can be used to detect whether there is a recurring subsequence in the historical sequence. One implementation is to check whether there is a subsequence P of length L (L≥2) at the end of the historical sequence, which also appears once immediately before it. For example, when the historical sequence is [..., X, Y, X, Y], let L=2, and the ending subsequence P is (X, Y). Check the preceding subsequence of the same length, which is also (X, Y). In this case, the pattern (X, Y) appears repeatedly, and the rule is hit.

[0092] In this example, when the sequence is [A, B, A, B], the rule matching unit 22 detects that the subsequence (A, B) of length 2 appears twice consecutively. Therefore, the "alternating deadlock rule" is triggered, and the determination module 20 determines that the model behavior is unhealthy and the type is "alternating deadlock".

[0093] Step 3: Targeted Injection Intervention. Injection module 30 is activated (corresponding to step S304 in Figure 3). Using "alternating deadlock" as the key, and possibly combined with the tool names (update_profile and login) corresponding to Action_ID_A and Action_ID_B as context, it queries the policy library for the most suitable bootstrap instruction. The policy library may contain the following rule: {"type": "alternating_deadlock","tools": ["update_profile", "login"],"hint": "A loop failure between updating information and logging in was detected. Please use the forgot_password tool to reset the password first, or check if the login credentials are correct."}

[0094] The injection module 30 obtains this highly relevant guidance information, packages it into a system prompt, and injects it into the communication sent to the model.

[0095] Step 4: The model breaks the deadlock. Upon receiving this prompt, the model not only recognizes that it is trapped in a loop, but more importantly, it obtains a concrete solution to the core problem (login failure): calling the `forgot_password` tool. This allows the model to escape the invalid ABAB loop and enter a new, constructive task path, thus enabling the entire task to continue.

[0096] This embodiment demonstrates that by performing pattern matching on action fingerprint sequences, the proposed solution can effectively identify and process cyclical behaviors that are more complex than simple repetitions, showcasing its depth and breadth in behavioral pattern analysis.

[0097] Example 4

[0098] This embodiment primarily demonstrates a variant implementation of the "silent injection" mechanism, aiming to further reduce interference with model behavior and make the guidance information appear as intelligent suggestions provided by the tool itself. In embodiments 1, 2, and 3, the guidance information appeared as explicit additional text tagged with "[System Hint]". In this embodiment, however, the guidance information will be directly integrated into the data structure of the tool's return result (tool_result).

[0099] The system structure of this embodiment is the same as that of the previous embodiment, the main difference being the specific implementation of the injection module 30.

[0100] Scenario: An agent is performing a market research task and needs to call a search engine tool, search_api, to query a very obscure keyword. This keyword has no matching results in the database.

[0101] The execution process of the method is as follows:

[0102] Step 1: The model initiates a query and receives an empty result. The model calls the search_api(query='a very obscure query term'). After the model context protocol server executes the query, no results are found, so it returns a tool_result indicating "empty". In JSON format, this is usually an empty array [].

[0103] tool_result: []

[0104] Step 2: The model repeats the query. The model may interpret this as an accidental network failure or a lack of proper strategy for handling empty results, so it calls the search_api again with the exact same parameters.

[0105] Step 3: The supervisor determines the behavior to be unhealthy. Similar to Example 1, the supervisor identifies this as a duplicate call through action fingerprinting, and the result of the previous call is empty (the supervisor can cache tool_result for analysis). The determination module 20 determines this behavior to be "unhealthy" and of type "duplicate invalid query".

[0106] Step 4: Injection using enhanced messages. Injection module 30 is activated. This time, it does not generate a [System Hint]. It matches a bootstrapping strategy from the strategy library for "empty result queries," such as: "Try broadening your keywords, or use the 'Advanced Search' tool and add filters."

[0107] The injection module 30 intercepted the empty result [] returned from the model context protocol server during the second query. It then structurally modified this JSON response. Instead of simply passing through [], it created a new, more structurally rich JSON object, using the original empty result as one field and the bootstrapping information as another. The modified tool_result (i.e., the enhanced message) is shown in Table 4.

[0108]

[0109] Table 4 shows the modified tool_result table.

[0110] This modified tool_result is sent to the weak language model.

[0111] Step 5: The model naturally adopts the suggestion. After receiving the `tool_result`, the model parses the JSON object. It sees that the `result` field is empty, which is expected; however, it also sees a field called `suggestion` containing very useful suggestions. To the model receiving this message, the `suggestion` field is like a friendly hint or advanced feature description provided by the `search_api` tool itself. Understandably, this injection method is more "silent" and natural; the model is less likely to perceive external guidance and instead treats the suggestion as an inherent intelligent function of the tool, thus adopting it more naturally and adjusting its query strategy in the next step, such as using broader keywords or calling the `advanced_search` tool.

[0112] This embodiment demonstrates the flexibility and versatility of the intervention injection mechanism. By deeply integrating the guidance information with the results returned by the original tool, seamless guidance of model behavior can be achieved. This approach has better compatibility for models with strict requirements on input formats, further enhancing the universality and effectiveness of the proposed solution.

[0113] In summary, this application, by deploying a non-intrusive, independent supervisor and utilizing technologies such as action fingerprinting, sequence pattern analysis, and lightweight semantic analysis, can monitor various unhealthy behaviors of a large language model when interacting with external tools in real time and efficiently. Furthermore, through a flexible, silent injection mechanism, it proactively guides the model to correct its behavior. Thus, without increasing system complexity or maintenance costs, it significantly improves the stability and task success rate of an intelligent agent system driven by a weak, large language model.

[0114] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A non-invasive model behavior monitoring and guidance method, characterized in that, Includes the following steps: An independent monitor deployed between the Model Context Protocol (MCP) client and server intercepts MCP messages transmitted between the client and server in real time. These MCP messages include at least tool call requests initiated by the model side. The independent supervisor analyzes the tool call request to determine whether the model's behavior is unhealthy. When the model's behavior is determined to be unhealthy, the independent supervisor injects structured guidance information into the model to guide it to correct its subsequent behavior.

2. The method according to claim 1, characterized in that, The step of analyzing the tool call request to determine whether the model's behavior is unhealthy includes: Based on the method name and parameters in the tool call request, an action fingerprint identifying the call is generated; Add the currently generated action fingerprint to a historical action fingerprint sequence; The historical action fingerprint sequence is compared with the preset unhealthy behavior pattern rules to determine whether the model's behavior is unhealthy, and if it is unhealthy, to determine its corresponding unhealthy behavior type.

3. The method according to claim 2, characterized in that, The method for generating the action fingerprint is to perform a hash operation on the method name and parameters.

4. The method according to claim 2, characterized in that, The unhealthy behavior pattern rules include: detecting the occurrence of the same action fingerprint multiple times consecutively.

5. The method according to claim 2, characterized in that, The unhealthy behavior pattern rules include: detecting two or more different action fingerprints that appear periodically and alternately.

6. The method according to claim 1, characterized in that, The step of analyzing the tool call request to determine whether the model's behavior is unhealthy further includes: Intercept the tool invocation result returned by the server that corresponds to the tool invocation request; Cache the results of multiple consecutive intercepted tool calls; The semantic similarity is determined by calculating the cosine similarity between the text vectors of multiple consecutive tool call results in the cache. When the semantic similarity is consistently higher than a preset threshold, the behavior of the model is determined to be unhealthy, and its unhealthy behavior type is determined.

7. The method according to claim 1, characterized in that, The step of injecting structured guidance information into the model specifically includes: The structured guidance information is appended as a system prompt to the next message sent to the model.

8. The method according to claim 1, characterized in that, The method further includes: Intercept the tool invocation result returned by the server that corresponds to the tool invocation request; The step of injecting structured guidance information into the model specifically includes: The structured guidance information is appended to the tool invocation result to form an enhanced message, and the enhanced message is sent to the model.

9. The method according to claim 1, characterized in that, Also includes: A policy library is pre-defined, which stores the correspondence between unhealthy behaviors and structured guidance information; In the step of injecting structured guidance information into the model, the corresponding structured guidance information is retrieved from the policy library and injected according to the type of unhealthy behavior determined.

10. A non-invasive model behavior monitoring and guidance device, characterized in that, include: The capture module is configured to intercept MCP messages transmitted between the Model Context Protocol (MCP) client and the server in real time. The MCP messages include at least a tool call request initiated by the model side. A determination module, connected to the capture module, is configured to obtain the MCP message from the capture module and analyze the tool call request therein in order to determine whether the model's behavior is unhealthy. An injection module, connected to the determination module, is configured to inject structured guidance information into the communication sent to the model side when the determination module determines that the model's behavior is unhealthy, so as to guide the correction of the model's behavior.

Images