Unified gateway-based mcp full-link tracking method and system

Abstract

The application provides a MCP full-link tracking method and system based on a unified gateway, and relates to the technical field of artificial intelligence.The application obtains key information such as a session identifier, a Token input quantity and tool calling information from a MCP protocol request and stores the key information in association after receiving the MCP protocol request initiated by a client through a gateway, then forwards the MCP protocol request to a MCP server, extracts response information after receiving a MCP protocol response returned by the MCP server, and finally generates a structured tracking log according to the response information and the key information, so that the MCP full-link tracking is completed at the gateway level, the MCP server is completely transparent, the MCP SDK source code does not need to be modified or each tool method needs to be individually implanted, the high coupling between business code and monitoring logic is avoided, and the maintenance cost is significantly reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an MCP end-to-end tracing method and system based on a unified gateway. Background Technology

[0002] With the widespread application of LLM (Large Language Model), AI (Artificial Intelligence) tool invocation systems based on MCP (Model Context Protocol) have gradually become the core technical architecture connecting large language models with external services. However, this architecture lacks a systematic solution for the end-to-end tracing problem of AI tool invocation: existing solutions usually require in-depth modification of the MCP SDK (Software Development Kit) source code or the insertion of monitoring points in each tool method, resulting in high coupling between business code and monitoring logic and high maintenance costs. Summary of the Invention

[0003] The purpose of this invention is to provide an MCP full-link tracing method and system based on a unified gateway, so as to alleviate the problem of high coupling between business code and monitoring logic and high maintenance costs in the existing technology.

[0004] In a first aspect, the present invention provides an MCP full-link tracing method based on a unified gateway, applied to a gateway deployed between a client and an MCP server. The MCP full-link tracing method based on a unified gateway includes: Receive MCP protocol requests initiated by the client. The MCP protocol requests carry a session identifier and are constructed by the client based on the tool invocation decision of the large language model. After obtaining key information from the MCP protocol request and storing it in association, the MCP protocol request is forwarded to the MCP server so that the MCP server can call the corresponding tool to execute business logic. The key information includes the session identifier and the amount of token input, as well as the tool call information when the MCP protocol request is a tool call type. Receive the MCP protocol response returned by the MCP server, extract the response information from the MCP protocol response, and forward the MCP protocol response to the client; Based on the response information and key information, a structured tracking log is generated.

[0005] In an optional implementation, the MCP protocol request is an HTTP request in JSON-RPC format, and the tool call information is obtained by parsing the method and params fields in the MCP protocol request; when the value of the method field is tools / call, the MCP protocol request is determined to be a tool call type, and the tool name and call parameters are extracted from the params field.

[0006] In an optional implementation, the response information includes the token output quantity, which is calculated based on the byte length of the MCP protocol response.

[0007] In an optional implementation, receiving the MCP protocol response returned by the MCP server, extracting response information from the MCP protocol response, and forwarding the MCP protocol response to the client includes: Based on the Content-Type and Transfer-Encoding fields in the response header, determine whether the MCP protocol response returned by the MCP server is a streaming response; If so, during the block-by-block reception process of the MCP protocol response, the token output amount of each received data block is accumulated and forwarded in real time until the transmission ends.

[0008] In an optional implementation, the gateway is a Kong gateway, and the acquisition of key information, extraction of response information, and generation of structured tracing logs correspond to the lifecycle hooks of the access phase, body_filter phase, and log phase of the Kong gateway, respectively.

[0009] In an optional implementation, the structured tracing log is in JSON format, including tracing identifier, session identifier, timestamp, request address, response status code, user agent, tool call list, token input, and token output; each item in the tool call list includes the tool name, call parameters, and call result.

[0010] In an alternative implementation, critical information is shared between different processing stages of the same MCP protocol request via the gateway's request context object.

[0011] Secondly, this invention provides an MCP end-to-end tracing system based on a unified gateway, applied to a gateway deployed between a client and an MCP server. The MCP end-to-end tracing method based on the unified gateway includes: The request receiving module is used to receive MCP protocol requests initiated by the client. The MCP protocol request carries a session identifier and is constructed by the client based on the tool invocation decision of the large language model. The request processing module is used to obtain key information from the MCP protocol request, store it in association, and then forward the MCP protocol request to the MCP server so that the MCP server can call the corresponding tool to execute business logic. The key information includes the session identifier and the amount of token input, as well as the tool call information when the MCP protocol request is a tool call type. The response processing module is used to receive the MCP protocol response returned by the MCP server, extract the response information from the MCP protocol response, and forward the MCP protocol response to the client. The log generation module is used to generate structured tracing logs based on response information and key information.

[0012] Thirdly, the present invention provides a gateway device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the MCP full-link tracing method based on a unified gateway according to any of the foregoing embodiments.

[0013] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, performs the MCP full-link tracing method based on a unified gateway according to any of the foregoing embodiments.

[0014] The present invention provides a unified gateway-based MCP end-to-end tracing method and system. After receiving an MCP protocol request initiated by a client, the gateway obtains key information such as session identifier, token input amount, and tool call information from the MCP protocol request and stores them in association. Then, it forwards the MCP protocol request to the MCP server. After receiving the MCP protocol response returned by the MCP server, it extracts the response information and finally generates a structured tracing log based on the response information and key information. This achieves MCP end-to-end tracing at the gateway level, which is completely transparent to the MCP server. It does not require modification of the MCP SDK source code or the insertion of points in each tool method, avoiding high coupling between business code and monitoring logic, and significantly reducing maintenance costs. Attached Figure Description

[0015] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0016] Figure 1 A flowchart illustrating an MCP end-to-end tracing method based on a unified gateway, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall system architecture corresponding to an MCP full-link tracing method based on a unified gateway provided in an embodiment of the present invention; Figure 3 Interaction timing diagram of phases one to five in an MCP full-link tracing method based on a unified gateway provided in an embodiment of the present invention; Figure 4 The interaction sequence diagram from stage six to the final stage in an MCP full-link tracing method based on a unified gateway provided in an embodiment of the present invention; Figure 5 This is a schematic flowchart of a response volume streaming process provided in an embodiment of the present invention; Figure 6 A schematic diagram of the structure of an MCP full-link tracing system based on a unified gateway provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a gateway device provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To facilitate understanding, the technologies related to the embodiments of the present invention will be introduced first.

[0019] I. Large Language Models and AI Agents (i.e., Intelligent Agents).

[0020] In recent years, large language models (referred to as large models) such as GPT-4, Claude, Gemini, DeepSeek, and Qwen have made groundbreaking progress. Their powerful natural language understanding and generation capabilities have enabled them to rapidly move from academic research to industrial production applications. Building on this foundation, AI Agent technology has emerged, endowing large models with the ability to invoke external tools, access real-time data, and perform complex tasks, thus evolving them from simple question-and-answer systems into intelligent executors capable of autonomously completing multi-step, cross-system tasks.

[0021] One of the core capabilities of AI agents is tool calling / function calling. This means that the large model automatically decides which external tool to invoke, what parameters to pass, and integrates the tool's return results into the final answer based on the user's natural language commands. This capability greatly expands the application boundaries of large models, enabling them to query weather in real time, access databases, execute code, and control IoT (Internet of Things) devices, forming a new technological paradigm of large model + tool ecosystem. As the scale of applications continues to expand, how to effectively track, monitor, and audit these tool calls has become one of the key challenges in the engineering implementation of AI systems.

[0022] II. Model Context Protocol (MCP).

[0023] MCP aims to solve the problem of fragmentation in the integration of large models with external tools and data sources. MCP adopts a client-server architecture and defines a standardized set of communication specifications, enabling any client that follows the MCP protocol (usually a large model inference framework or AI application) to discover and invoke any tools and resources exposed by the MCP server in a unified manner.

[0024] The MCP protocol's transport layer supports multiple modes, including stdio (Standard Input / Output), SSE (Server-Sent Events), and Streamable HTTP (Hypertext Transfer Protocol) mode. In Streamable HTTP mode, the MCP service runs as a standard HTTP server. Clients send messages in JSON-RPC (Remote Procedure Call) format via HTTP POST requests, and the server can return a single HTTP response or stream multiple messages via SSE.

[0025] MCP's core message types include: tool list discovery (tools / list), tool call execution (tools / call), resource access (resources / read), and prompt template retrieval (prompts / get). Taking a weather query as an example, a typical tools / call request body format is as follows: { "jsonrpc": "2.0", "id": 1, "method": "tools / call", "params": { "name": "GetWeather", "arguments": { "city": "Beijing"} } } The standardized nature of the MCP protocol has led to its rapid adoption in the industry, with numerous AI frameworks and enterprise service providers announcing their support for the MCP standard, resulting in an increasingly prosperous tool ecosystem.

[0026] III. Kong API Gateway (Kong Gateway for short) technology.

[0027] Kong is a high-performance, scalable, open-source API gateway built on Nginx and OpenResty (LuaJIT), developed and maintained by Kong Inc. Kong holds a significant position in the enterprise API management field and is widely deployed in microservice architectures, cloud-native environments, and digital transformation projects.

[0028] Kong's core architecture consists of a data plane and a control plane. The data plane is responsible for the actual request brokering and plugin execution, while the control plane is responsible for configuration management and distribution. Kong's plugin system is its most important extension mechanism. Plugins are written in Lua scripts and intervene in the request processing flow by inheriting the BasePlugin base class and implementing specific lifecycle hook functions.

[0029] The Kong plugin's lifecycle hooks, arranged in the order of request processing, include: certificate (TLS (Transport Layer Security) handshake phase), rewrite (URI (Uniform Resource Identifier) ​​rewriting phase), access (access control phase, before request forwarding), header_filter (response header filtering phase), body_filter (response body filtering phase), and log (logging phase, after request completion). This granular lifecycle management allows the Kong plugin to precisely intervene at each critical stage of request processing, forming the technological foundation for non-intrusive monitoring.

[0030] IV. Distributed Link Tracing Technology.

[0031] Distributed tracing is an indispensable observability technique in modern microservice architectures. Its core idea is to assign a unique trace ID to each request and record the corresponding span information at each service node through which the request flows. Finally, these scattered spans are linked together to form a complete call chain, which is used for performance analysis, fault diagnosis, and dependency management.

[0032] In AI system tracing, in addition to the metrics such as latency and error rate that traditional microservice tracing focuses on, special attention needs to be paid to AI-specific dimensions: tool call parameters and return values, token consumption (input / output are counted separately), model session context (Session ID), the decision-making process of tool selection logic, and the relationships between multi-turn dialogues. These AI-specific tracing dimensions are of great significance for cost control, security auditing, and business insights.

[0033] V. The Importance of Observability and AI Link Tracing.

[0034] Observability is a core capability for the operation and maintenance of modern distributed systems, typically composed of three pillars: logs, metrics, and traces. In AI tool usage scenarios, observability holds particular importance, as shown in Table 1 below: Table 1

[0035] As AI applications expand in scale, the complexity of tool call chains increases dramatically. Without effective tracking methods, problems such as difficulty in locating faults, difficulty in controlling costs, and difficulty in auditing security will arise.

[0036] The shortcomings of existing technical solutions are described below.

[0037] 1. Observability challenges of MCP tool calls.

[0038] While the MCP protocol provides a standardized communication interface for tool invocation, its observability architecture still suffers from significant deficiencies. When an AI system involves multiple MCP services and various tool types, developers and operations personnel face the following core challenges: Unable to trace tool call chain: It is impossible to know which tools were called, the order of calls, and the parameters of each call when the large model was processing a user request; Unable to assess tool execution quality: Lack of systematic data on tool execution success rate, response time distribution, and error type statistics; Unable to control token consumption: The lack of accurate measurement of the number of input / output tokens generated for each tool call makes cost accounting difficult; Unable to perform security audits: It is impossible to retrospectively trace the specific parameters passed to the tool by the large model in a particular session, which poses a security and compliance risk. Unable to associate multiple rounds of dialogue: It is difficult to establish the contextual association between multiple tool calls in an MCP session, which brings great difficulties to troubleshooting.

[0039] 2. Limitations of existing tracking solutions.

[0040] To address the above challenges, existing solutions mainly follow these paths, but all have significant limitations: (a) Modify the MCP SDK source code.

[0041] Embedding tracing code directly in the message handling logic of the MCP SDK allows for global interception by modifying the transport or message dispatch layer. While this method provides complete tracing, it faces the following problems: It is strongly dependent on a specific SDK version. Once the SDK is upgraded, the tracing code needs to be re-adapted, resulting in extremely high maintenance costs. This violates the best practice of not modifying the source code of third-party libraries, which can easily introduce stability risks. The MCP SDK cannot cover implementations in different languages ​​(such as Python, Java, C#, Go, TypeScript, etc.) and needs to be implemented separately in each SDK.

[0042] (ii) Manually embed points in the tool method.

[0043] Manually adding logging or tracing code to each specific tool implementation function is the most direct solution, but it also has serious problems: Huge workload: As the number of tools increases, each tool requires independent addition and maintenance of tracing code, resulting in a large amount of repetitive work; Business logic pollution: Business code and monitoring logic are highly mixed and coupled, which reduces the readability and maintainability of the code and violates the single responsibility principle; Incomplete coverage: Relying on manual operations by developers makes it easy for omissions to occur, resulting in incomplete tracking data; High cross-team coordination costs: If the MCP tool is developed by different teams, a lot of cross-team communication is needed to unify the tracking specifications and promote implementation.

[0044] (iii) Application layer AOP (Aspect-Oriented Programming) aspect scheme.

[0045] By using AOP (Aspect-Oriented Programming) technology, utility methods are dynamically proxied at the framework level, enabling unified injection of tracking logic. While more elegant than manual event tracking, it still has limitations: This still falls under the category of application-layer intrusion, requiring the introduction of an AOP framework and modification of the tool registration process; Due to the strong dependencies between languages ​​and frameworks, cross-language MCP services cannot be handled uniformly, and it is impossible to handle each language individually. It is impossible to track information about the request at the network transport layer (such as client IP, request headers, original message size, etc.).

[0046] 3. Analysis of the root causes of the problem.

[0047] The fundamental flaw in the existing solutions mentioned above lies in the fact that they all choose to address the tracing problem at the application layer, while the complete lifecycle of an MCP tool call actually spans multiple layers, including the client, network transport layer, gateway layer, and service layer. Tracing within the application layer inherently fails to obtain network-level information and cannot achieve true decoupling from specific business code.

[0048] In a production environment, if an AI tool call malfunctions or produces unexpected results, troubleshooting becomes extremely difficult due to the lack of a complete call log. Developers often need to compare logs from multiple services one by one: LLM call logs, MCP service application logs, and network layer access logs. These logs have different formats and are stored in different locations, making efficient correlation and analysis impossible. This results in long problem localization cycles and high manpower costs.

[0049] The correct solution is to push the tracing logic down to the network infrastructure layer, that is, to intercept and parse all MCP requests (i.e., MCP protocol requests) at the gateway nodes that all MCP requests must pass through, thereby achieving unified, transparent, and non-intrusive tracing of all MCP services. This is the core idea of ​​the solution proposed in this invention.

[0050] The comparative analysis of multiple options is shown in Table 2 below: Table 2

[0051] Based on this, this invention provides a method and system for end-to-end MCP tracing based on a unified gateway. At the gateway layer, a custom plugin intercepts and parses MCP request and response messages, extracting key tracing information such as tool name, parameters, session identifier, and token statistics. This information is then output to a log system in structured JSON format, achieving comprehensive observability of the entire AI tool call lifecycle. The core advantage of this solution lies in its complete transparency to the MCP server, requiring no modification to any business code, and possessing excellent scalability and engineering practical value.

[0052] Experimental results show that the solution of this invention can completely track the entire process from client query, large model tool decision-making, Kong gateway routing, MCP tool execution to result return without affecting the normal performance of the service, providing strong technical support for the operation and maintenance monitoring, security auditing and cost control of AI systems.

[0053] The purpose of this invention is to achieve complete observability of the entire AI MCP tool call chain through the Kong gateway plugin mechanism, without modifying any MCP server-side code, relying on a specific MCP SDK implementation, or adding any tracing code to business tool methods.

[0054] The specific technical objectives of this invention include: 1. Non-intrusive: The MCP server-side code requires zero modification, the tracing logic is completely transparent to the business logic, and the addition or removal of plugins does not affect the normal operation of business functions.

[0055] 2. Completeness: Capable of capturing complete information about tool calls, including key fields such as method name, input parameters, output parameters, session identifier, timestamp, status code, and token estimation.

[0056] 3. Accuracy: Based on the precise parsing of MCP JSON-RPC protocol messages, the semantic correctness of the tracking data is ensured, avoiding information omissions or misinterpretations.

[0057] 4. End-to-end tracing: Covers the entire lifecycle from client request to tool call result return, including key dimensions such as tool name, call parameters, response status, time consumption, and data volume.

[0058] 5. Session-level association: Based on Mcp-Session-Id, multiple tool calls within the same AI session are associated and tracked, supporting the reconstruction of the complete AI inference-tool call sequence.

[0059] 6. Low performance overhead: The core tracing logic is executed asynchronously in the log phase (after the response has been sent), which does not increase the user's perceived request latency and has a negligible impact on system throughput.

[0060] 7. Scalability: The architecture of the tracking plugin should be highly scalable, allowing for easy addition of new tracking dimensions (such as statistics by user, grouping by tool type, anomaly detection, etc.).

[0061] 8. Engineering practicality: The output tracking data should be in structured JSON format, which is easy to integrate with existing log collection systems (such as Filebeat or Fluentd) and data analysis platforms (such as Kafka or Elasticsearch).

[0062] 9. Multi-language coverage: Since the tracing logic is located at the gateway layer, it is independent of the implementation language of the MCP service and naturally supports MCP services written in various languages ​​such as Python, Java, C#, and Go.

[0063] The core innovation of this solution lies in choosing the Kong gateway as the implementation layer for MCP link tracing, a choice with profound engineering significance: 1. Separation of Concerns: Separate monitoring concerns from the business layer to the infrastructure layer, with each layer performing its own duties; 2. Centralized Management: Tracking data for all MCP services is collected uniformly at the gateway layer, avoiding data fragmentation; 3. Language Independence: The tracing scheme at the gateway layer does not depend on any specific programming language, thus having the greatest versatility; 4. Protocol Awareness: By parsing the MCP JSON-RPC protocol, it achieves deeper semantic-level tracing than general HTTP tracing.

[0064] To facilitate understanding of this embodiment, a detailed description of the MCP full-link tracing method based on a unified gateway disclosed in this embodiment of the invention will be provided first.

[0065] This invention provides an MCP end-to-end tracing method based on a unified gateway. This method is applied to a gateway deployed between the client and the MCP server, and can be executed by the gateway device. See also... Figure 1 The diagram shows a flowchart of an MCP end-to-end tracing method based on a unified gateway. The method mainly includes the following steps S110 to S140: Step S110: Receive the MCP protocol request initiated by the client.

[0066] The aforementioned client is an AI application assistant, which can be delivered via, but is not limited to, a web page, an app (i.e., a mobile application), or a CLI (Command Line Interface). The client receives natural language input from the user and interacts with a large language model. The aforementioned MCP server is a backend service that deploys specific business tools, such as weather query services, web search services, and database query services. The gateway is deployed between the client and the MCP server, serving as a unified entry point for the MCP service; the gateway embeds an MCP tracing and parsing plugin, and can be, but is not limited to, a Kong gateway.

[0067] The aforementioned MCP protocol request is constructed by the client based on the tool invocation decision of the large language model and carries a session identifier. Specifically, when a user enters a natural language question on the client, the client sends the user message along with a list of tool definitions to the large language model. After understanding the user's intent, the large language model decides whether to invoke a tool; if so, it returns a tool invocation instruction. Upon receiving this instruction, the client converts it into an HTTP request conforming to the MCP protocol specification and sends it to the MCP server through the gateway.

[0068] For example, a user inputs "Please tell me what the weather is like in Beijing right now?" into a client (such as a custom AI application on a PC or mobile device). After receiving the user's input, the client constructs a dialogue request payload, adds the user message to the dialogue history, and appends a list of tool definitions to the request. The tool definitions describe the names, functional descriptions, and parameter schemas of currently available tools; this information comes from a pre-registered list of MCP tools. Taking the GetWeather tool as an example, its tool definition structure can be represented as follows: { "name": "GetWeather", "description": "Gets the current weather for a city", "input_schema": { "type": "object", "properties": { "city": { "type": "string", "description": "The city name" } }, "required": ["city"] } } The client sends the complete dialogue request to the large language model API endpoint (e.g., https: / / api.gpt.com / v1 / messages) via HTTPS (Hypertext Transfer Protocol Secure); records the request sending timestamp and waits for the large language model's response. Upon receiving the request, the large language model performs semantic understanding, tool matching, parameter extraction, and tool invocation decision-making within the model. For example, the large language model parses the user's intent, identifying that the user needs weather information and the target city is "Beijing"; then, it retrieves the GetWeather tool's function description from the tool definition list, finding it highly consistent with the user's intent, and extracts the key parameter "Beijing" from the user input, mapping it to the GetWeather tool's city parameter; finally, the large language model generates a tool invocation response, instructing the client to invoke the GetWeather tool and pass in the parameter city=Beijing. The structure of the tool invocation response returned by the large language model can be represented as follows: { "id": "msg_01XFDUDYJgAACTu7DVvzYtgN", "type": "message", "role": "assistant", "content": [{ "type": "tool_use", "id": "toulu_01A09q90qw90lq917835lq9", "name": "GetWeather", "input": { "city": "Beijing"} }], "stop_reason": "tool_use" } After receiving the tool call instruction from the large language model, the client converts it into an HTTP request conforming to the MCP protocol specification and sends it to the gateway. Specific steps may include: protocol conversion, converting the tool_use block into a request body in MCP JSON-RPC2.0 format; session management, reading and including the Mcp-Session-Id request header; and sending the request to the MCP endpoint exposed by the gateway using the POST method.

[0069] For example, a complete HTTP request can be represented as: POST / mcp / weather HTTP / 1.1 Host: my.kong-gateway:80 Content-Type: application / json Mcp-Session-Id: sess_abc123xyz Authorization: Bearer<user token> { "jsonrpc": "2.0", "id": 1, "method": "tools / call", "params": { "name": "GetWeather", "arguments": { "city": "Beijing"} } } One explanation is that by delegating the construction logic of MCP protocol requests to the client, the gateway only needs to receive standard HTTP requests and does not need to concern itself with the interaction details between the client and the large language model, thus maintaining the gateway's versatility and simplicity.

[0070] Step S120: After obtaining key information from the MCP protocol request and storing it in association, the MCP protocol request is forwarded to the MCP server so that the MCP server can call the corresponding tools to execute business logic.

[0071] The core task of this step is to extract the key information required for subsequent tracing from the MCP protocol request that arrives at the gateway, associate and store this information with the current request, and then transparently forward the request to the backend MCP server.

[0072] The key information mentioned above includes the session identifier and token input, as well as tool invocation information when the MCP protocol request is a tool invocation type. The session identifier is used to associate multiple tool invocations within the same AI session; the token input is used to estimate the invocation cost; and the tool invocation information records which tool was specifically invoked and what parameters were passed in, which is the basis for generating semantic tracking logs later.

[0073] In some possible embodiments, the aforementioned gateway is a Kong gateway, and obtaining key information corresponds to the lifecycle hook of the access phase of the Kong gateway. Specifically, after the MCP protocol request arrives at the Kong gateway, the access function of the MCP tracing plugin (with an execution priority of PRIORITY=2000, a high priority) is triggered, executing the following refined tracing logic: request body reading, session identifier extraction, token input estimation, protocol parsing and method judgment, and tool call information extraction.

[0074] Specifically, the processing logic in the access phase may include the following steps: First, read the request body. Call `ngx.req.read_body()` and `get_body_data()` to obtain the raw request body byte stream. If the MCP protocol request is cached in a temporary file due to a large request body, use `get_body_file()` to read it back.

[0075] Secondly, extract the session identifier. Read the Mcp-Session-Id field from the request header and store it in ngx.ctx.mcp_session_id. Here, ngx.ctx is a context object shared across all stages of the same request.

[0076] Next, estimate the token input size. Subtract the fixed length of the common fields (corresponding to the outer structure) from the original byte length of the request body to obtain the length of the params parameter content. Use this as an approximation of the token input size and store the result in ngx.ctx.total_input_tokens. This method offers the highest performance and reduces the overhead of re-serialization.

[0077] Next, the protocol is parsed and the method type is determined. `cjson.decode()` is called to parse the JSON request body, and the `method` field is read to determine the MCP method type.

[0078] Finally, extract the tool call information. When the value of the method field is tools / call, extract the tool name from the params.name field, extract the call parameters from the params.arguments field, and assemble them into a structured Lua table (i.e., a Lua table) and store it in ngx.ctx.tool_calls.

[0079] In one possible implementation, the core logic of the access phase can be represented as follows: function MCP_Handler:access(conf) MCP_Handler.super.access(self) ngx.req.read_body() local body = ngx.req.get_body_data() -- Extract session identifier and estimate token ngx.ctx.mcp_session_id = ngx.req.get_headers()["Mcp-Session-Id"] or"" ngx.ctx.tool_calls = {} --To obtain the length, simply subtract the fixed length of the common fields (outer structure) from the body length. --The remaining params parameters offer the highest performance and reduce the need for re-serialization.

[0080] ngx.ctx.total_input_tokens = body and #body - 72 or 0 -- Parsing the MCP protocol if body then local ok, req_obj = pcall(cjson.decode, body) if ok and req_obj and req_obj.method == "tools / call" then ngx.ctx.tool_calls = { name = req_obj.params and req_obj.params.name, arguments = req_obj.params and req_obj.params.arguments } end end end Based on this, in some possible embodiments, the aforementioned MCP protocol request is an HTTP request in JSON-RPC format, and the tool invocation information is obtained by parsing the method and params fields in the MCP protocol request. When the value of the method field is tools / call, it is determined that the MCP protocol request is a tool invocation type, and the tool name and invocation parameters are extracted from the params field.

[0081] Specifically, the MCP protocol is based on the JSON-RPC 2.0 specification, and its request body contains four top-level fields: jsonrpc, id, method, and params. The method field identifies the type of MCP operation. Utility methods include tools / call (calling a tool) and tools / list (listing available tools), while resource methods include resources / read (reading resources), etc. When the value of the method field is tools / call, it indicates a tool invocation request. In this case, the params field contains two subfields: name and arguments, representing the tool name and its invocation parameters, respectively.

[0082] For example, in the MCP protocol request example given in step S110 above, the parsing result is: the value of the method field is tools / call, the value of params.name is GetWeather, and the value of params.arguments is {"city":"Beijing"}. Based on this, the plugin determines that this request is a tool call type and extracts the tool name as GetWeather and the call parameter as city=Beijing.

[0083] This can be explained by the fact that, unlike general HTTP tracing plugins that only record HTTP layer information such as URLs (Uniform Resource Locators), methods, and status codes, this solution implements semantic-level parsing of the MCP JSON-RPC protocol. By identifying the `method` field, the plugin can distinguish different types of MCP operations and extract deeper semantic information based on the tool call type. This protocol-aware tracing gives the monitoring data rich business semantics, making it far more valuable than simple HTTP logs.

[0084] After extracting and storing key information (i.e., after the access phase), the gateway forwards the MCP protocol request to the registered MCP upstream service according to the routing configuration. During the forwarding process, the gateway performs standard load balancing (such as Round Robin, Least Connections, etc.), health checks, and connection pool management. The entire forwarding process is transparent to the MCP tracing plugin, which does not need to intervene in this phase.

[0085] After receiving the request forwarded by the gateway, the MCP server (Streamable HTTP Transport Layer) parses the JSON-RPC request using the MCP SDK. Based on the `method` and `params.name` fields, it routes the request to the corresponding utility method. The SDK automatically deserializes the JSON-formatted `arguments` object and binds it to the utility method parameters. The utility method then executes the actual business logic.

[0086] For example, the implementation of the GetWeather method can be represented as follows: [McpServerTool] [Description("Gets the current weather for a city")] public static string GetWeather( [Description("The city name")] string city) { / / Actual production environment: Calling a real weather API / / var weather = await weatherApiClient.GetAsync(city); / / Example: Simulated weather data returned var random = new Random(); var temp = random.Next(0, 30); var conditions = new[] { "Sunny", "Cloudy", "Rainy", "Partly Cloudy"}; var condition = conditions[random.Next(conditions.Length)]; return $"Weather in {city}: {temp}℃, {condition}"; } The MCP SDK encapsulates the return value of utility methods into a JSON-RPC success response conforming to the MCP protocol specification. The structure can be represented as follows: { "jsonrpc": "2.0", "id": 1, "result": { "content": [{ "type": "text", "text": "Weather in Beijing: 72 ℃, Partly Cloudy" }], "isError": false } } It's important to note that this solution is completely transparent to the MCP server; the MCP server needs no awareness of the gateway's tracing behavior. The MCP server does not require installing any additional monitoring agents, modifying any business code, or importing any specific logging libraries. The gateway achieves truly zero-intrusion monitoring by intercepting requests and responses and executing tracing logic before and after forwarding.

[0087] In this embodiment of the invention, the gateway extracts key information from the MCP protocol request during the access phase, stores it in association, and then transparently forwards it to the MCP server, achieving completely non-intrusive monitoring of the MCP server. The MCP server can obtain full-link tracing capabilities without any modification, fundamentally solving the problem of high coupling between business code and monitoring logic in traditional solutions, and significantly reducing the system maintenance cost.

[0088] Step S130: Receive the MCP protocol response returned by the MCP server, extract the response information from the MCP protocol response, and forward the MCP protocol response to the client.

[0089] After the MCP server completes the tool call and returns a response, this step is responsible for extracting key response information from the response as it passes through the gateway, and then forwarding the response back to the client.

[0090] The response information mentioned above includes the token output amount, which, together with the token input amount recorded in step S120, constitutes a complete token consumption record to support cost estimation. In some possible embodiments, the token output amount in the response information is calculated based on the byte length of the MCP protocol response. Specifically, the length of the response body is reduced by the fixed length of the common fields (corresponding to the outer structure), leaving the length of the content parameter, which is used as an approximation of the token output amount, and ngx.ctx.total_output_tokens is updated. This method offers the highest performance and reduces the overhead of re-deserialization and serialization.

[0091] In some possible embodiments, the above-mentioned receiving the MCP protocol response returned by the MCP server, extracting response information from the MCP protocol response, and forwarding the MCP protocol response to the client includes: determining whether the MCP protocol response returned by the MCP server is a streaming response based on the Content-Type and Transfer-Encoding fields in the response header; if so, then during the block-by-block reception of the MCP protocol response, accumulating and counting the token output of each received data block and forwarding it in real time until the transmission ends.

[0092] Specifically, extracting response information corresponds to the body_filter phase of the Kong gateway. When the response returned by the MCP server passes through the Kong gateway, the body_filter hook function is triggered. Due to Nginx's streaming processing mechanism, the response body may be transmitted in multiple chunks (i.e., data blocks), and the body_filter hook function will be called multiple times. The tracing plugin captures the response body through the chunk mechanism: when Content-Type is text / event-stream, or Transfer-Encoding is chunked, it is determined that the MCP protocol response is a streaming response. ngx.ctx.total_output_tokens is used to directly accumulate each received response body (i.e., data block). No data caching occurs during this process; that is, the current chunk is immediately forwarded to the client by Nginx after the body_filter hook function returns (i.e., after the body_filter hook function accumulates and counts the token output), thus achieving high-performance real-time pass-through while receiving, processing, and sending.

[0093] In this embodiment of the invention, the entire process always adopts chunk pass-through rather than overall caching, thus avoiding problems such as streaming output blocking, additional memory occupation and increased response latency caused by response body caching, while meeting the requirements of low latency, high throughput and low memory overhead for scenarios such as high concurrency and large model streaming inference.

[0094] Step S140: Generate a structured tracking log based on the response information and key information.

[0095] After extracting key information from the request side and information from the response side, this step brings all the information together and generates a structured tracing log in a predefined format for subsequent persistent storage and analysis.

[0096] In some possible implementations, generating structured tracing logs can correspond to lifecycle hooks in the Kong gateway's log phase. Specifically, the log phase hook function is triggered after request processing is completely complete (the response has been sent to the client). This phase is the optimal time for logging because all complete request and response information can be obtained at this point without affecting client response latency. The tracing plugin can read all the information collected in the preceding phases from ngx.ctx, assemble it, and serialize it into a JSON-formatted tracing log.

[0097] The structured tracing logs mentioned above are in JSON format and include the trace identifier (trace_id), session identifier (mcp_session_id), timestamp (timestamp), request URL (uri), response status code (status), user agent (user_agent), tool call list (tools_called), total input tokens (total_input_tokens), and total output tokens (total_output_tokens). Each item in the tool call list includes the tool name (name), call parameters (arguments), and call result.

[0098] For example, the final output trace log JSON can be represented as: { "trace_id": "a1b2c3d4e5f6g7h8", "mcp_session_id": "sess_abc123xyz", "timestamp": 1735689600.123, "uri": " / mcp ", "status": "200", "tools_called": { "name": "GetWeather", "arguments": { "city": "Beijing"} }, "total_input_tokens": 128, "total_output_tokens": 87, "user_agent": "ai agent desktop / 1.0" } In some possible implementations, the aforementioned key information is shared across different processing stages of the same MCP protocol request via the gateway's request context object. Specifically, Kong / Nginx's ngx.ctx is a Luatable bound to the request, whose lifecycle is completely consistent with the current request and is shared across all processing stages of the same request. This solution cleverly utilizes this mechanism: request-side information collected in the access stage is stored in ngx.ctx, response-side information collected in the body_filter stage is also stored in ngx.ctx, and finally, all information is aggregated and output in the log stage.

[0099] In one possible implementation, the core logic of the log phase can be represented as: function MCP_Handler:log(conf) MCP_Handler.super.log(self) local ctx = ngx.ctx if ctx.tool_calls then local ok, json_str = pcall(cjson.encode, { trace_id = ctx.trace_id or "", mcp_session_id = ctx.mcp_session_id, timestamp = ngx.now(), uri = ngx.var.uri, status = ngx.var.status, tools_called = ctx.tool_calls, total_input_tokens = ctx.total_input_tokens or 0, total_output_tokens = ctx.total_output_tokens or 0, user_agent = ngx.var.http_user_agent }) If OK then ngx.log(ngx.INFO,"[MCP-TRACE] " .. json_str) -- Input to log files for easy collection by Filebeat / Fluentd, or input to Kafka, etc. end end end In some possible embodiments, the above method further includes: persistently storing the tracing logs to support visual queries and monitoring alerts. Specifically, the tracing logs can be output to standard output, collected by a log collection system, and then sent to a data analysis platform for persistent storage.

[0100] Specifically, the output tracing logs can be collected in real time by log collection agents such as Filebeat / Fluentd, pushed to the Kafka message queue, and then written to Elasticsearch by consumers (i.e., downstream data processing components) for persistent storage and indexing. Finally, the Kibana Dashboard enables visual queries and monitoring alerts to support various application scenarios such as real-time monitoring, offline analysis, and auditing alerts.

[0101] For example, Filebeat can be used as a log collection agent to monitor Kong's log files and send new log entries to the Kafka message queue in real time. Kafka acts as a buffer layer, ensuring reliable log transmission in high-concurrency scenarios and preventing log write blocking from impacting business performance. Downstream Logstash or Elasticsearch consumes log data from Kafka for parsing, indexing, and storage. Finally, operations personnel can use visualization tools such as Kibana to perform full-text searches, aggregation analysis, and chart displays on the tracking logs, and can also promptly notify relevant personnel when anomalies occur through preset alert rules.

[0102] It should be noted that the JSON format trace logs output by this solution are specifically designed for log collection pipelines, with clear field naming and standardized data types, and are fully compatible with Filebeat's automatic field discovery and Elasticsearch's dynamic mapping.

[0103] In this embodiment of the invention, the following key call chain information can be collected by parsing the MCP request / response body: Session ID (Mcp-Session-Id): Associated with multiple rounds of conversations with the same user; The tool name invoked: specifies which tool was triggered; Call parameters: Records complete tool input parameters to support subsequent problem reproduction; Request / response body size (approximate value of input / output tokens): Cost estimation supported; HTTP status codes and response results: Determine whether the call was successful; Request URI, timestamp, User-Agent: Supplemental context information.

[0104] In this embodiment of the invention, the method, after receiving the MCP protocol request initiated by the client, obtains key information such as session identifier, token input amount, and tool call information from the MCP protocol request through the gateway, and stores them in association. Then, it forwards the MCP protocol request to the MCP server. After receiving the MCP protocol response returned by the MCP server, it extracts the response information and finally generates a structured tracing log based on the response information and key information. This achieves full-link tracing of MCP at the gateway level, which is completely transparent to the MCP server. It does not require modification of the MCP SDK source code or the insertion of points in each tool method, avoiding the high coupling between business code and monitoring logic, and significantly reducing maintenance costs.

[0105] For ease of understanding, such as Figure 2 As shown, the system architecture of this solution consists of four core roles and two auxiliary systems, forming a complete tracing loop. The four core roles are: the client (client layer, i.e., the user interface layer, providing the user interface), the large language model (AI inference layer), the Kong gateway (gateway layer), and the MCP server (MCP service layer, i.e., the business tool layer). The two auxiliary systems are: a log collection system (Filebeat / Fluentd) and a data analysis platform (Kafka / Elasticsearch / Kibana).

[0106] The responsibilities of each role are defined in Table 3 below: Table 3

[0107] like Figure 2As shown, the user interface at the client layer (Web / App / Claude Desktop) triggers the large language model (Claude / GPT-4) at the AI ​​inference layer to make tool invocation decisions through natural language interaction. Subsequently, an MCP request (HTTPPOST) is constructed and passed to the gateway layer (Kong API Gateway). The gateway layer sends the request into the plugin execution pipeline through routing and load balancing, processing it sequentially through three stages: 1_access (parsing the MCP request and extracting tool information), 2_body_filter (capturing response body token statistics), and 3_log (outputting tracing logs in JSON format). The first two stages transparently forward the GetWeather (weather tool), GetCalendar (calendar tool), and SearchWeb (search tool) tools at the MCP service layer. After processing, the MCP service layer returns the results to the AI ​​inference layer, where the large language model integrates the results and generates the final response, which is then fed back to the user interface. Simultaneously, the structured tracing logs output at stage 3_log flow to the log analysis layer, are pushed to the Kafka message queue via the Filebeat log collection agent, and then written to Elasticsearch data storage by the consumer. Finally, queries and monitoring are achieved through the Kibana visualization dashboard. The entire process achieves a closed loop from user interaction, tool invocation, log collection to analysis and visualization.

[0108] Starting with a user entering "check Beijing's weather" into the client, the complete request processing lifecycle goes through the following eight key stages, such as... Figure 3 and Figure 4 As shown: Phase 1: User initiates a request (Client → LLM).

[0109] The user types a natural language question into the input box of the client (such as a custom AI application on PC / Mobile): "Please tell me what the weather is like in Beijing right now?" After receiving the user's input, the client performs the following operations: Construct a dialogue request payload to add the user message to the dialogue history.

[0110] Add a list of tool definitions to the request. Tool definitions describe the names, functional descriptions, and parameter schemas of currently available tools; this information is derived from a pre-registered list of MCP tools. For example... Figure 3 As shown, the client sends the constructed request POST{role:user,content:“Query Beijing weather”,tools:[GetWeather]defined} to the large language model.

[0111] Phase Two: Understanding the Large Model's Intent and Making Tool Decisions (LLM Internal Processing).

[0112] After receiving a user request, the large language model performs the following reasoning process internally: Semantic understanding: The model parses the user's intent and identifies that the user needs to obtain weather information (i.e., weather query intent), and the target city is "Beijing".

[0113] Tool matching: The model searches the tool definition list and finds that the function description of the GetWeather tool is highly matched with the user's intent (i.e., selecting the GetWeather tool).

[0114] Parameter extraction: The model extracts key parameters from user input and maps "Beijing" to the city parameter of the GetWeather tool, i.e., city = Beijing.

[0115] Tool Invocation Decision: The model generates a tool invocation response, instructing the client to invoke the GetWeather tool and pass in the parameter city=Beijing.

[0116] like Figure 3 As shown, the large language model returns the tool call response {name: GetWeather,input:{city=Beijing}} to the client.

[0117] Phase 3: The client constructs an MCP request (Client → Kong Gateway).

[0118] After receiving the tool invocation instruction from the model, the client is responsible for converting it into an HTTP request conforming to the MCP protocol specification and sending it to the Kong gateway. Specific steps include: protocol conversion (converting the tool_use block into a request body in MCP JSON-RPC 2.0 format), session management (reading and including the Mcp-Session-Id request header), and sending the request via POST to the MCP endpoint exposed by the Kong gateway. Figure 3 As shown, the client sends the constructed MCP request POST / mcp / weather Mcp-Session-Id:sess_abc123 Body:{method:tools / call,params:name:GetWeather,arguments: {city=Beijing}}} to the Kong gateway.

[0119] Phase 4: Kong Gateway receives requests – access phase (core tracing steps).

[0120] After the MCP request reaches the Kong gateway, the access function of the MCP tracing plugin (PRIORITY=2000, higher priority) is triggered, executing the following refined tracing logic: Request body reading: Call ngx.req.read_body() and get_body_data() to obtain the raw request body byte stream. If the request body is large and cached in a temporary file, use get_body_file() to read it back.

[0121] Session ID extraction: Read the Mcp-Session-Id field from the request header and store it in ngx.ctx.mcp_session_id (ngx.ctx is a context object shared across all stages of the same request).

[0122] Token input quantity estimation: Subtract the fixed public field length (outer structure) from the original byte length of the request body, leaving the params parameter content. This method has the highest performance and reduces the need for re-serialization. Then, it is stored in ngx.ctx.total_input_tokens.

[0123] Protocol parsing and method determination: Call cjson.decode() to parse the JSON request body and read the method field to determine the MCP method type.

[0124] Tool call information extraction: When the method is tools / call, the tool name is extracted from params.name, and the call parameters are extracted from params.arguments. These are then assembled into a structured Lua table and stored in ngx.ctx.tool_calls. For example... Figure 3 As shown, extract Name=GetWeather, Arguments={city: Beijing}.

[0125] Phase 5: Kong Gateway Transparently Forwards Requests (Kong→MCP Service).

[0126] After the access phase is completed, the Kong gateway forwards the MCP request to the registered MCP upstream service according to the routing configuration. For example... Figure 3 As shown, Kong performs standard load balancing (algorithms such as Round Robin and LeastConnections), health checks, and connection pool management during the forwarding process. The entire forwarding process is transparent to the MCP tracing plugin, which does not need to intervene in this stage.

[0127] Phase Six: MCP Service Execution Tool Logic (Internal to MCP Service).

[0128] like Figure 4 As shown, after the MCP service (Streamable HTTP) receives the request forwarded by Kong, the MCP SDK parses the JSON-RPC request and routes the request to the corresponding utility implementation method based on the method field (tools / call) and the params.name field (GetWeather). The SDK automatically deserializes the JSON-formatted arguments object and binds it to the utility method parameters. Then, the GetWeather method executes the actual business logic.

[0129] The MCP SDK encapsulates the return values ​​of utility methods into JSON-RPC success responses that conform to the MCP protocol specification.

[0130] Phase 7: Kong Gateway Response Interception – Body_Filter Phase.

[0131] When the response returned by the MCP service passes through the Kong gateway, the `body_filter` hook function is triggered. Due to Nginx's streaming mechanism, the response body may be transmitted in multiple chunks, resulting in the `body_filter` hook function being called multiple times. Figure 4 As shown, the tracking plugin captures the response body through a chunk mechanism: it determines whether the response is streaming (i.e., whether it belongs to a streaming response), obtains response chunks, counts the response body, and forwards the response to the client in real time without caching any data during the process. Similar to total_input_tokens, the response body's byte length is reduced by the fixed length of common fields (outer structure), leaving the content parameter content, which has the highest performance to reduce the need for re-deserialization and serialization. Finally, ngx.ctx.total_output_tokens is updated.

[0132] like Figure 5 As shown, the response volume flow processing flow is as follows: When the upstream service returns a response header, the plugin first enters the header_filter phase. It checks the Content-Type and Transfer-Encoding fields in the response header to determine if the current response is a streaming response. If the Content-Type is text / event-stream or the Transfer-Encoding is chunked, the current response is marked as a streaming response (i.e., ctx.is_stream=true); otherwise, it is marked as a normal response (i.e., ctx.is_stream=false). Subsequently, the output byte statistics variable (i.e., the number of tokens output) is initialized to total_output_tokens=0.

[0133] Subsequently, as the upstream service continues to return response data, Nginx triggers the body_filter hook function once for each chunk. The plugin first checks whether ctx.is_stream is true, that is, whether the current response is marked as a streaming response. If it is marked as a non-streaming response, it does not process and returns directly. If it is marked as a streaming response, the plugin obtains the ngx.arg[1] chunk (i.e. the currently received data chunk) and checks whether the chunk is empty. If the chunk is not empty, it accumulates the byte length of the chunk (i.e., counts the output size) and records the chunk log (such as debug log or monitoring information) as needed; at the same time, in order to maintain the streaming transmission characteristics, the plugin does not cache, splice or reassemble the data chunk, nor does it modify the content of ngx.arg[1] (i.e., keeps ngx.arg[1] unchanged), so that the current chunk is immediately forwarded to the client by Nginx after the body_filter returns, thereby realizing high-performance real-time transparent transmission while receiving, processing and sending; then it checks whether it is finished. If the chunk is empty, it directly checks whether it is finished. If the process is not complete, return to body_filter and wait for the next chunk; if the process is complete, output statistics (i.e., the final number of tokens output).

[0134] Throughout the response process, the above processing is repeated for each chunk until the last chunk is received. At this point, the plugin stops statistics, records the total number of output bytes in the entire response body and related log information, and then terminates the processing of this request. Because the entire process uses chunk pass-through rather than overall caching, this method can fully preserve the real-time output capabilities of SSE and HTTP Chunked Streaming, avoiding problems such as streaming output blocking, additional memory consumption, and increased response latency caused by response body caching. At the same time, it meets the requirements of low latency, high throughput, and low memory overhead in scenarios such as high concurrency and large model streaming inference.

[0135] In one possible implementation, the core logic of the body_filter phase can be represented as follows: --Is Streaming local function is_stream_response() local headers = ngx.header -- Content-Type local content_type = headers["Content-Type"] or "" if content_type then content_type = string.lower(content_type) -- SSE if string.find(content_type,"text / event-stream",1, true) then return true end end -- Transfer-Encoding local te = headers["Transfer-Encoding"] or "" if te then te = string.lower(te) if string.find(te, "chunked", 1, true) then return true end end return false end -- Header stage: Whether to Stream function MCP_Handler:header_filter(conf) HttpFilterHandler.super.header_filter(self) local ctx = ngx.ctx ctx.is_stream = is_stream_response() -- Output Statistics ctx.total_output_tokens = 0 end -- body_filter phase: Calculate the token output function MCP_Handler:body_filter(conf) HttpFilterHandler.super.body_filter(self) local ctx = ngx.ctx -- Exit directly if not streaming If not ctx.is_stream then return end local chunk = ngx.arg[1] -- Statistical output size If chunk and #chunk > 0, then ctx.total_output_tokens = ctx.total_output_tokens + #chunk end end Phase 8: Tracking Log Output – The log phase.

[0136] After request processing is completely complete (the response has been sent to the client), the hook function in the log phase is triggered. This phase is the optimal time for logging because all complete request and response information can be obtained at this time without affecting the client's response latency. Figure 4As shown, the tracing plugin reads all the information collected in the preceding stages from ngx.ctx, assembles and serializes it into a JSON format tracing log.

[0137] The output structured JSON trace logs are collected in real time by log collection agents such as Filebeat / Fluentd, pushed to the Kafka message queue, and then written to Elasticsearch by the consumer for persistent storage and indexing. Finally, the Kibana Dashboard enables visualization, querying, monitoring, and alerting to support various application scenarios such as real-time monitoring, offline analysis, and auditing alerts.

[0138] like Figure 4 As shown, the final client integrates the results and returns the tool results to the large language model (tool_result message); the large language model integrates the tool results, generates a natural language answer, and returns the final reply to the client: "Current weather in Beijing: 71F, cloudy turning sunny..."

[0139] In summary, the key points of the embodiments of the present invention include: 1. Implementation mechanism of non-intrusive design: The non-intrusive nature of this solution stems from its full utilization of the gateway lifecycle hook mechanism. Kong executes registered hook functions sequentially when processing each HTTP request. By parsing the original request body during the access phase and capturing response information during the body_filter and log phases, the plugin can fully acquire all the critical information of the MCP protocol, without the MCP service itself being aware of it. This interception design is similar to the principles of Aspect-Oriented Programming (AOP), but implemented at a lower-level network infrastructure layer.

[0140] 2. In-depth analysis of the MCP JSON-RPC protocol: Unlike general HTTP tracing plugins (which only record HTTP-level information such as URLs, methods, and status codes), this solution implements semantic-level parsing of the MCP JSON-RPC protocol. By identifying the method field (tools / call, tools / list, resources / read, etc.), the plugin can distinguish different types of MCP operations and extract deeper semantic information (tool name, call parameters) based on tool call types. This protocol-aware tracing gives the monitoring data rich business semantics, making it far more valuable than simple HTTP logs.

[0141] 3. ngx.ctx cross-stage data sharing mechanism: Kong / Nginx's ngx.ctx is a Lua table bound to the request, with its lifecycle perfectly aligned with the current request, and shared across all processing stages of the same request (access, header_filter, body_filter, log). This solution cleverly leverages this mechanism: request-side information collected in the access stage (tool call parameters, Session ID, input token estimation) is stored in ngx.ctx, response-side information collected in the body_filter stage (output token estimation) is also stored in ngx.ctx, and finally, all information is aggregated and output in the log stage. This cross-stage data transfer mechanism is the key technological foundation for achieving complete tracing.

[0142] 4. Chunk uncached statistics strategy for response streaming processing: The response body employs a streaming chunk pass-through strategy. During the `body_filter` phase, response chunks are not cached, concatenated, or reassembled; instead, each chunk is processed statistically in real-time and immediately passed through to the client. Simultaneously, statistical information such as the number of output bytes is collected during the data pass-through process. This strategy fully preserves the real-time transmission characteristics of SSE and HTTP Chunked Streaming, avoiding streaming output failures, increased response latency, and additional memory consumption in large response scenarios caused by caching the entire response body. This ensures low latency, high throughput, and low memory overhead in high-concurrency and large-model streaming output scenarios.

[0143] 5. Session-level association tracking based on Mcp-Session-Id: The MCP protocol identifies session state through the Mcp-Session-Id header. Plugins extract this identifier during the access phase and use it throughout the entire request lifecycle, ultimately writing it to the tracing log. Downstream analytics systems can use this field to aggregate and analyze multiple tool calls within the same AI session (potentially including calls to GetWeather, web search, database queries, and other different tools), reconstructing the complete AI inference-tool call chain and providing a data foundation for understanding AI behavior patterns.

[0144] 6. Downstream integration design of structured logs: The ultimate value of trace data lies in its efficient querying and analysis. The JSON-formatted trace logs output by this solution are specifically designed for log collection pipelines, featuring clear field naming (trace_id, mcp_session_id, tools_called, etc.), standardized data types (numeric timestamps, integer token statistics), and full compatibility with Filebeat's automatic field discovery and Elasticsearch's dynamic mapping. Using Kafka as a buffer layer ensures reliable log transmission in high-concurrency scenarios, preventing log write blocking from impacting business performance.

[0145] 7. Low operation and maintenance costs and high scalability: Plugins are configured and attached to Kong routes or services. When adding a new MCP service, simply bind the plugin to the Kong management route API; no code changes are required, resulting in extremely low maintenance costs. Furthermore, the plugin structure is clear, and tracing dimensions can be expanded as needed.

[0146] Corresponding to the above-described MCP end-to-end tracing method based on a unified gateway, this embodiment of the invention also provides an MCP end-to-end tracing system based on a unified gateway. See also... Figure 6 The diagram shown illustrates the architecture of an MCP end-to-end tracing system based on a unified gateway. The system includes: The request receiving module 601 is used to receive the MCP protocol request initiated by the client. The MCP protocol request carries a session identifier and is constructed by the client based on the tool invocation decision of the large language model. The request processing module 602 is used to obtain key information from the MCP protocol request, store it in association, and then forward the MCP protocol request to the MCP server so that the MCP server can call the corresponding tool to execute business logic. The key information includes the session identifier and the amount of token input, as well as the tool call information when the MCP protocol request is a tool call type. The response processing module 603 is used to receive the MCP protocol response returned by the MCP server, extract response information from the MCP protocol response, and forward the MCP protocol response to the client. The log generation module 604 is used to generate structured tracking logs based on response information and key information.

[0147] The MCP end-to-end tracing system based on a unified gateway provided in this invention obtains key information such as session identifier, token input amount, and tool call information from the MCP protocol request after receiving it from the client, and stores them in association. Then, it forwards the MCP protocol request to the MCP server. After receiving the MCP protocol response returned by the MCP server, it extracts the response information and finally generates structured tracing logs based on the response information and key information. This system achieves end-to-end MCP tracing at the gateway level, which is completely transparent to the MCP server. It does not require modification of the MCP SDK source code or the insertion of points in each tool method, avoiding high coupling between business code and monitoring logic and significantly reducing maintenance costs.

[0148] Furthermore, the aforementioned MCP protocol request is an HTTP request in JSON-RPC format. The tool call information is obtained by parsing the method and params fields in the MCP protocol request. When the value of the method field is tools / call, the MCP protocol request is determined to be a tool call type, and the tool name and call parameters are extracted from the params field.

[0149] Furthermore, the above response information includes the token output quantity, which is calculated based on the byte length of the MCP protocol response.

[0150] Furthermore, the response processing module 603 is specifically used to: determine whether the MCP protocol response returned by the MCP server is a streaming response based on the Content-Type and Transfer-Encoding fields in the response header; if so, then during the block-by-block reception of the MCP protocol response, the token output of each received data block is accumulated and forwarded in real time until the transmission ends.

[0151] Furthermore, the aforementioned gateway is a Kong gateway, and the acquisition of key information, extraction of response information, and generation of structured tracing logs correspond to the lifecycle hooks of the access phase, body_filter phase, and log phase of the Kong gateway, respectively.

[0152] Furthermore, the aforementioned structured tracing log is in JSON format, including a tracing identifier, the session identifier, a timestamp, a request address, a response status code, a user agent, a tool call list, token inputs, and token outputs; each item in the tool call list includes the tool name, call parameters, and call result.

[0153] Furthermore, the aforementioned key information is shared between different processing stages of the same MCP protocol request through the request context object of the gateway.

[0154] Furthermore, the aforementioned MCP end-to-end tracing system based on a unified gateway also includes: The log storage module is used to persistently store tracing logs to support visual queries and monitoring alerts.

[0155] The MCP full-link tracing system based on a unified gateway provided in this embodiment has the same implementation principle and technical effects as the aforementioned MCP full-link tracing method embodiment based on a unified gateway. For the sake of brevity, any parts not mentioned in the embodiment of the MCP full-link tracing system based on a unified gateway can be referred to the corresponding content in the aforementioned MCP full-link tracing method embodiment based on a unified gateway.

[0156] like Figure 7 As shown, an embodiment of the present invention provides a gateway device 700, including: a processor 701, a memory 702 and a bus. The memory 702 stores a computer program that can run on the processor 701. When the gateway device 700 is running, the processor 701 and the memory 702 communicate through the bus, and the processor 701 executes the computer program to implement the above-mentioned MCP full-link tracing method based on a unified gateway.

[0157] Specifically, the memory 702 and processor 701 mentioned above can be general-purpose memory and processor, without any specific limitations here.

[0158] This invention also provides a computer-readable storage medium storing a computer program. When a processor executes the computer program, it performs the MCP end-to-end tracing method based on a unified gateway as described in the preceding method embodiments. The computer-readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), RAM, magnetic disks, or optical disks.

[0159] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0160] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limitations; therefore, other examples of exemplary embodiments may have different values.

[0161] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface, and the indirect coupling or communication connection between modules may be electrical, mechanical, or other forms.

[0163] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0164] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for end-to-end MCP tracing based on a unified gateway, characterized in that, The MCP end-to-end tracing method based on a unified gateway, applied to a gateway deployed between the client and the MCP server, includes: Receive the MCP protocol request initiated by the client, the MCP protocol request carrying a session identifier, the MCP protocol request being constructed by the client based on the tool invocation decision of the large language model; After obtaining key information from the MCP protocol request and storing it in association, the MCP protocol request is forwarded to the MCP server so that the MCP server can call the corresponding tool to execute business logic; wherein, the key information includes the session identifier and the token input amount, as well as the tool call information when the MCP protocol request is a tool call type; Receive the MCP protocol response returned by the MCP server, extract response information from the MCP protocol response, and forward the MCP protocol response to the client; A structured tracking log is generated based on the response information and the key information.

2. The MCP end-to-end tracing method based on a unified gateway according to claim 1, characterized in that, The MCP protocol request is an HTTP request in JSON-RPC format. The tool call information is obtained by parsing the method and params fields in the MCP protocol request. When the value of the method field is tools / call, the MCP protocol request is determined to be a tool call type, and the tool name and call parameters are extracted from the params field.

3. The MCP end-to-end tracing method based on a unified gateway according to claim 1, characterized in that, The response information includes a token output quantity, which is calculated based on the byte length of the MCP protocol response.

4. The MCP end-to-end tracing method based on a unified gateway according to claim 3, characterized in that, The step of receiving the MCP protocol response returned by the MCP server, extracting response information from the MCP protocol response, and forwarding the MCP protocol response to the client includes: Based on the Content-Type and Transfer-Encoding fields in the response header, determine whether the MCP protocol response returned by the MCP server is a streaming response; If so, during the block-by-block reception process of the MCP protocol response, the token output amount of each received data block is accumulated and forwarded in real time until the transmission ends.

5. The MCP end-to-end tracing method based on a unified gateway according to claim 1, characterized in that, The gateway is the Kong gateway. Obtaining key information, extracting response information, and generating structured tracing logs correspond to the lifecycle hooks of the access phase, body_filter phase, and log phase of the Kong gateway, respectively.

6. The MCP end-to-end tracing method based on a unified gateway according to claim 1, characterized in that, The structured tracing log is in JSON format and includes a tracing identifier, the session identifier, a timestamp, a request address, a response status code, a user agent, a tool call list, token inputs, and token outputs. Each item in the tool call list includes the tool name, call parameters, and call result.

7. The MCP end-to-end tracing method based on a unified gateway according to claim 1, characterized in that, The critical information is shared between different processing stages of the same MCP protocol request through the request context object of the gateway.

8. An MCP end-to-end tracing system based on a unified gateway, characterized in that, The MCP end-to-end tracing method based on a unified gateway, applied to a gateway deployed between the client and the MCP server, includes: The request receiving module is used to receive the MCP protocol request initiated by the client. The MCP protocol request carries a session identifier and is constructed by the client based on the tool invocation decision of the large language model. The request processing module is used to obtain key information from the MCP protocol request, store it in association, and then forward the MCP protocol request to the MCP server so that the MCP server can call the corresponding tool to execute business logic; wherein, the key information includes the session identifier and the token input amount, as well as the tool call information when the MCP protocol request is a tool call type; The response processing module is used to receive the MCP protocol response returned by the MCP server, extract response information from the MCP protocol response, and forward the MCP protocol response to the client; The log generation module is used to generate structured tracking logs based on the response information and the key information.

9. A gateway device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the MCP full-link tracing method based on a unified gateway as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the MCP full-link tracing method based on a unified gateway as described in any one of claims 1-7.

Images