System for secure MCP-mediated tool use by AI agents and generative AI / LLM services in cloud-native distributed applications

Abstract

A system (100) for the secure MCP-mediated use of tools by AI agents and generative AI / LLM services in cloud-native distributed applications, wherein the system (100) comprises: a KL agent interface (1) configured to receive natural language commands and application events from a variety of client applications and to generate appropriate tool call commands for one or more generative KL or Large Language Model (LLM) services; an MCP mediator service (2) that is configured to: (a) to convert the tool request requests into messages compatible with a model context protocol (MCP); and (b) to maintain the conversation context, including at least one of the following: user identity, tenant identity and application identity; a tool register (3) that stores a plurality of tool descriptions, each tool description defining at least a tool identifier, an input and output scheme, an endpoint location and allowed functions, wherein the tool register (3) is accessible to the MCP mediator service (2); a policy and security manager (4) configured to evaluate each MCP tool call against one or more security and access policies based on the conversation context and the corresponding tool description, and to issue a decision to allow, modify or block the tool call; a tool connector layer (5) comprising a plurality of tool adapters, each tool adapter being configured to communicate securely with a corresponding external tool, service or data source using credentials and permissions restricted according to the decision of the policy and security manager (4); an observation and audit manager (6) configured to record, for each tool call, at least a timestamp, the calling KL agent, the tool identifier, the policy decision, and a summary of the tool response, and to provide audit logs and metrics for monitoring and compliance purposes; and a cloud-native deployment controller (7) configured to provide the MCP mediator service (2), policy and security manager (4), tool connector layer (5) and observation and audit manager (6) as distributed microservices with network isolation between tenants in a cloud-native environment.

Description

INVENTION AREA

[0001] The present invention relates to the field of artificial intelligence-enabled software systems and cloud-native distributed applications. In particular, the invention relates to a system for the secure, policy-driven, and observable use of tools by AI agents and generative AI / LLM (Large Language Model) services via a model context protocol (MCP) in multi-tenant cloud environments. BACKGROUND OF THE INVENTION

[0002] The subject matter discussed in the "Background" section should not be considered prior art solely because it is mentioned therein. Likewise, a problem mentioned in or related to the subject matter of the "Background" section should not be considered prior art. The subject matter in the "Background" section merely presents various approaches, which could themselves be inventions.

[0003] Generative AI and large language model (LLM)-based agents are increasingly being integrated into enterprise and cloud-native applications. These AI agents often require access to external tools, services, and data sources such as databases, software-as-a-service (SaaS) applications, cloud applications, and internal microservices to perform tasks on behalf of users.

[0004] Existing integrations between AI agents and tools are often implemented as ad hoc, tightly coupled connectors or plugins. Such approaches have several limitations: • Missing standard protocol: There is no unified protocol for representing tool descriptors, tool call requests, responses, and conversational context. Each application or agent framework provides tools in a proprietary way, making it difficult to reuse connectors and enforce consistent controls across different agents and tenants. • Insufficient security and enforcement of policies: Direct access by AI agents to tools or external systems often bypasses fine-grained access control. Policies are typically hard-coded, applied only at the infrastructure level, or not contextualized with user identity, tenant identity, or application context. This increases the risk of data exfiltration, unauthorized operations, and privilege escalation. • Limited observability and verifiability: In many systems, tool calls triggered by AI agents are not adequately logged with contextual information such as the calling agent, the user, the tool parameters, and the policy decisions. As a result, it is difficult to understand why a particular AI-generated response was produced or which tools and data sources were involved, complicating compliance and troubleshooting. • Challenges in multi-tenant cloud-native environments: In cloud-native deployments, multiple tenants can share a common AI infrastructure while simultaneously requiring strict isolation of tools, data, and policies. Traditional designs do not offer strong network and credential isolation between tenants and cannot dynamically adapt to different tool workloads and AI request traffic. • Inconsistent handling of synchronous and asynchronous tools: Some tools respond instantly, while others may require lengthy processing, background tasks, or the streaming of intermediate results. Existing systems struggle to provide a unified, secure, and observable interface for synchronous and asynchronous tool interactions.

[0005] Accordingly, there is a need for a system that mediates tool usage by AI agents and LLM services via a standard model context protocol, enforces detailed security and access policies, provides strong observability and audit trails, and supports scalable, cloud-native deployments with tenant-level isolation.

[0006] The use of any examples or illustrative phrases (e.g., "as") relating to specific embodiments serves only to better illustrate the invention and does not constitute a limitation of the otherwise claimed scope of the invention. No wording in the description shall be construed as referring to an unclaimed element that is essential for carrying out the invention.

[0007] The information disclosed above in this "Background" section is provided solely for a better understanding of the background of the invention and may therefore contain information that is not part of the prior art already known to a person skilled in the art in this country. SUMMARY

[0008] Before describing the systems and methods presented here, it should be noted that this application is not limited to the specific systems and methods described, as there may be several possible embodiments not expressly presented in this disclosure. It should also be noted that the terminology used in the description serves only to describe the specific versions or embodiments and is not intended to limit the scope of this application.

[0009] In one aspect, the invention provides a system (100) for the secure MCP-mediated use of tools by AI agents and generative AI / LLM services in cloud-native distributed applications.

[0010] The system (100) includes a KL agent interface (1) configured to receive natural language commands and application events from a variety of client applications and to generate appropriate tool call commands for one or more generative KL or Large Language Model (LLM) services.

[0011] The system includes an MCP mediator service (2) configured to convert tool call requests into messages compatible with a Model Context Protocol (MCP) and to maintain the conversational context, including at least one of the following: user identity, tenant identity, and application identity. The MCP mediator service (2) can also add a signed context token to each tool call, encoding at least the user identity, the tenant identity, and the requested tool identifier.

[0012] A tool registry (3) stores a variety of tool descriptions, each defining at least a tool identifier, an input and output schema, an endpoint location, and allowed functions. The tool registry (3) is accessible to the MCP mediator service (2) and can also store editorial rules for sensitive fields.

[0013] A policy and security manager (4) evaluates each MCP tool call against one or more security and access policies based on the conversation context and the corresponding tool description, and issues a decision to allow, modify, or block the tool call. The policy and security manager (4) may include a policy store (4a) that stores declarative policies and a policy engine (4b) that evaluates these policies at runtime.

[0014] A tool connector layer (5) comprises a variety of tool adapters, each configured to communicate securely with a corresponding external tool, service, or data source using credentials and permissions restricted according to the policy and security manager's decision (4). The tool connector layer (5) can support both synchronous calls and asynchronous, long-running tasks and can stream partial results back to the KL agent interface (1) via the MCP mediator service (2).

[0015] An observability and audit manager (6) records at least a timestamp, the calling AI agent, the tool identifier, the policy decision, and a summary of the tool response for each tool call. It provides audit logs and metrics for monitoring and compliance purposes and can generate a tool usage lineage view that links a generated AI response with the sequence of tools used, policy decisions, and data sources.

[0016] A cloud-native deployment controller (7) provides the MCP mediator service (2), the policy and security manager (4), the tool connector layer (5), and the observability and audit manager (6) as distributed microservices with network isolation between tenants in a cloud-native environment. It can orchestrate microservices using a container orchestration platform, enforce network policies per tenant, and automatically scale components based on load metrics retrieved from the observability and audit manager (6).

[0017] In some embodiments, the system (100) further includes a secret and key management module (8) configured to store, rotate, and provide short-lived access tokens, API keys, and encryption keys to the tool connector layer (5) based on decisions made by the policy and security manager (4). The AI ​​agent interface (1) can also receive security feedback and error signals from the policy and security manager (4) and the tool connector layer (5) and send line messages back to AI agents and LLM services to control future tool calls in accordance with the policies. The MCP mediator service (2) can be configured to redact or mask sensitive fields in the tool's inputs and outputs according to rules defined in the tool registry (3) and the policy and security manager (4). BRIEF DESCRIPTION OF THE DRAWING

[0018] To clarify various aspects of some embodiments of the present invention, a more detailed description of the invention is given with reference to specific embodiments shown in the accompanying drawings. It is understood that these drawings represent only illustrative embodiments of the invention and are therefore not to be considered as limiting its scope. The invention is described and explained with additional specificity and detail using the accompanying drawings.

[0019] To make the advantages of the present invention easily understandable, the invention is described in detail below with reference to the accompanying drawing. However, it should be noted that the scope of the invention is not limited to the accompanying drawing, which: Fig. A block diagram representation of the system (100) for secure MCP-mediated tool use by AI agents and generative AI / LLM services in cloud-native distributed applications is shown. DETAILED DESCRIPTION

[0020] The present invention relates to the system (100) for the secure MCP-mediated use of tools by AI agents and generative AI / LLM services in cloud-native distributed applications.

[0021] Fig. shows a detailed block diagram representation of the system (100) for secure MCP-mediated tool use by AI agents and generative AI / LLM services in cloud-native distributed applications.

[0022] Although the present disclosure has been described with the purpose of a system for the secure MCP-mediated use of tools by AI agents and generative AI / LLM services in cloud-native distributed applications, it should be noted that this is merely to illustrate the invention by way of example and to highlight other purposes or functions for which the described structures or configurations could be used and which fall within the scope of the present disclosure.

[0023] The following detailed description is intended to enable those skilled in the art to manufacture and use the invention and is not intended to limit the scope of the invention as defined in the claims. Where appropriate, the same reference numerals are used to denote similar components. Overall system architecture

[0024] With reference to Fig. 1. The system (100) is deployed in a cloud-native environment and can be implemented using one or more physical or virtual servers, containerized application workloads, and managed cloud services. The system (100) is accessible via one or more networks, such as the internet or a private network, to a variety of client applications that embed or communicate with AI agents and generative AI / LLM services.

[0025] The system (100) is inherently multi-tenant capable. Each tenant can correspond to an organization, a department, or an application area. Tenant-specific configurations such as tool definitions, policies, login credentials, and network coupling rules are retained. KL agent interface (1)

[0026] The AI ​​agent interface (1) is configured to receive natural language commands and application events from client applications. The AI ​​agent interface (1) can provide one or more APIs, messaging endpoints, or streaming interfaces to which client applications can connect.

[0027] In one embodiment, the AI ​​agent interface (1) performs the following: • Receives a request that includes natural language input, a user ID, a tenant ID, and optional application metadata. • forwards the natural language input to a generative AI or LLM service, optionally with a list of candidate tools retrieved from the tool registry (3). • Receives tool call intents from the AI ​​agent or LLM service, which may be expressed as function calls, tool call objects, or structured messages. • converts these intentions into tool call requests enriched with the conversation context, including user identity, tenant identity and application identity, and sends them to the MCP mediator service (2).

[0028] The AI ​​agent interface (1) is further configured to receive security feedback, policy error messages, and explanations from the policy and security manager (4) and the tool connector layer (5). Based on this feedback, the AI ​​agent interface (1) can generate guidance for the AI ​​agents and LLM services, informing them about blocked tools, restricted parameters, or suggested alternative tools, thereby controlling future tool calls in accordance with the policies. MCP Mediator Service (2)

[0029] The MCP mediator service (2) acts as a central coordination layer that implements a Model Context Protocol (MCP). The MCP provides a standardized way to represent tool descriptors, call messages, responses, and conversation contexts.

[0030] In one embodiment, the MCP mediator service (2) performs the following tasks: • Receives tool call requests from the KL agent interface (1). • searches for the corresponding tool description in the tool registry using a tool identifier (3) • Creates MCP-compliant messages that contain the following: • the tool identifier and version; • the input payload structured according to the input scheme; • Conversational context including user identity, tenant identity, and application identity; and • Additional metadata, such as correlation identifiers and trace identifiers.

[0031] The MCP mediator service (2) maintains a conversation context store that maps conversation identifiers to stored context fields, including but not limited to user identity, tenant identity, application identity, previous tool calls and previous AI messages.

[0032] In some embodiments, the MCP mediator service (2) is further configured such that it • Add a signed context token to each tool call. The signed context token can be structured as a token that encodes at least the user identity, the tenants, the requested tool identifier, and the timestamp, and is digitally signed with a key obtained from the secret and key management module (8). • Redacting or masking sensitive data in the input or output data of the tools according to the redaction rules stored in the tool registry (3) and evaluated in conjunction with the policy and security manager (4). Sensitive fields can be replaced with masked values, hash values, or pseudonyms before being sent to external tools or returned to AI agents. Tool registration (3)

[0033] The tool registry (3) is a logical repository in which a variety of tool descriptions are stored. Each tool description defines at least: • A tool identifier, a name, and optionally a version; • An input scheme that specifies permissible parameters, types, ranges, and default values; • An output scheme that describes the expected result structure; • an endpoint location, e.g., an HTTP URL, a message topic, an RPC endpoint, or a database connection descriptor; and • Permitted functions, including allowed operations, data ranges, and rate limits.

[0034] In addition, the tool registry (3) can store the following: • Editorial rules for input and output fields that specify which fields contain sensitive data (e.g., personal data, secret keys, financial data) and define masking strategies; • Client-specific configurations, e.g., which clients are allowed to use a particular tool, and client-specific parameter restrictions; • Metadata used by the Observability and Audit Manager (6), such as tool categories, business impact levels, or criticality assessments.

[0035] The tool registration (3) can be implemented using one or more databases or configuration services and is accessible to the MCP mediator service (2) and the policy and security manager (4) via secure APIs. Policy and Security Manager (4)

[0036] The policy and security manager (4) enforces security and access policies at the granularity of each MCP tool call. As described in Fig. As shown, the policy and security manager includes (4): • A policy store (4a) configured to store declarative policies that define allowed tools, parameters, and data ranges for different tenants, roles, user groups, and applications. Policies can be expressed in a domain-specific language or a standardized policy language and may include conditions relating to context attributes, tool identifiers, parameter values, and time-based restrictions. • A policy engine (4b) configured to evaluate the policies at runtime for each tool call. The policy engine (4b) receives as input the conversation context, the tool description from the tool registry (3), and the suggested call parameters from the MCP mediator service (2).

[0037] For each tool call, the policy and security manager (4) issues a decision, which may include, for example, the following: • Allow: The tool call is allowed as requested. • Modification: The tool call is permitted with modifications such as parameter editing, parameter limitation, reduced data size, rate limiting, or redirection to a more secure tool. • Block: The tool call is rejected.

[0038] In the event of a change or blocking decision, the policy and security manager (4) can also generate explanations or security feedback, which are returned to the Kl agent or LLM via the MCP mediator service (2) and the Kl agent interface (1).

[0039] The policy and security manager (4) can consult the secret and key management module (8) to ensure that only suitable credentials and data areas are provided to the tool connector layer (5) based on the evaluated policies and the tenant configuration. Tool connector layer (5)

[0040] The tool connector layer (5) comprises a variety of tool adapters, each configured to communicate securely with a corresponding external tool, service, or data source. Examples of external tools include internal microservices, cloud APIs, SaaS applications, databases, message queues, search engines, and workflow systems.

[0041] Each tool adapter within the tool connector layer (5): • Receives a normalized tool call request and the associated policy decision from the MCP mediator service (2). • receives the minimum required login credentials, access tokens, API keys or connection parameters from the secret and key management module (8), as authorized by the policy and security manager (4). • Communicates with the target tool endpoint using a secure protocol (e.g., HTTPS, TLS-secured database connections, secure message queues). • Enforces all restrictions specified by the policy decision, such as limiting accessible resources, filtering query parameters, or restricting operations to read-only mode.

[0042] The tool connector level (5) supports: • Synchronous tool calls where the adapter waits for the tool's response and returns a structured output data load to the MCP mediator service (2). • Asynchronous, long-lasting tool tasks where the adapter submits a job to the external system, receives a job ID, and later returns the result or intermediate updates upon completion.

[0043] For asynchronous tasks, the tool connector layer (5) can stream partial results and progress updates back to the AI ​​agent interface (1) via the MCP mediator service (2), enabling AI agents to handle long-running operations smoothly. Observability and Audit Managers (6)

[0044] The Observability and Audit Manager (6) provides centralized logging, metrics, and provenance tracking for the system (100). On each tool invocation, the Observability and Audit Manager (6) records at least the following: • a timestamp of the call; • An identifier of the calling KL agent or LLM; • the user identity and the client identity; • The tool identifier and, optionally, the version; • The policy decision (allow, change, block) and all changes made; • A summary of the tool's response, e.g., status, error codes, and general content descriptions.

[0045] The Observability and Audit Manager (6) can store such data in an attachment-only log, a time-series database, or another storage system optimized for audits and analysis.

[0046] In one embodiment, the monitoring and audit manager (6) is further configured such that it • Creation of metric dashboards that display tool usage volume, error rates, policy violations, and latency distributions. • Creating a view of the tool usage origin (see Fig.), which links a final, AI-generated answer to the sequence of tool calls, policy decisions, and data sources that influenced the answer. This origin view is useful for troubleshooting, regulatory compliance, and explaining AI behavior to human reviewers.

[0047] The Observability and Audit Manager (6) also provides metrics and events to the cloud-native deployment controller (7) to support decisions for automatic scaling and health management. Cloud-native deployment controller (7)

[0048] The cloud-native deployment controller (7) is responsible for the provisioning and management of the system components (100) as distributed microservices in a cloud-native environment.

[0049] In one embodiment, the cloud-native deployment controller (7) performs the following tasks: • the MCP Mediator Service (2), the Policy and Security Manager (4), the Tool Connector Layer (5) and the Observability and Audit Manager (6) as separate containerized microservices. • uses a container orchestration platform to schedule, restart, and update these microservices. • Configures network policies per tenant so that tool adapters for one tenant are isolated from tool adapters of another tenant, for example by assigning separate namespaces, virtual networks or service meshes. • Monitors the load metrics and health indicators provided by the Observability and Audit Manager (6), such as request throughput, average latency, error rates and queue depths. • Automatically scales the MCP mediator service (2) and the tool connector layer (5) horizontally by adding or removing instances according to observed load metrics, thereby maintaining performance and cost efficiency.

[0050] The cloud-native deployment controller (7) can also coordinate rolling upgrades, canary deployments and failover strategies to ensure high system availability (100). Module for managing secrets and keys (8)

[0051] The secrets and keys management module (8) manages sensitive login credentials and cryptographic keys used throughout the system (100). In one embodiment, the secrets and keys management module (8) has the following functions: • Stores API keys, access tokens, database passwords, private keys and encryption keys in a secure vault. • It issues short-lived access tokens to tool adapters in the tool connector layer (5), the scope and lifetime of which are limited according to the decisions of the policy and security manager (4). • It rotates keys and tokens regularly or on request and passes updated login credentials to the relevant microservices. • provides signature keys for the MCP mediator service (2) to generate signed context tokens that are attached to tool calls.

[0052] All access to the Secrets and Keys Management Module (8) is controlled and audited, with access policies adapted to tenant boundaries and the principle of least privilege.

[0053] The figure and the preceding description provide examples of embodiments. Those skilled in the art will recognize that one or more of the described elements can be combined to form a single functional element. Alternatively, certain elements can be divided into several functional elements. Elements from one embodiment can be added to another embodiment. For example, the sequence of the processes described here can be changed and is not limited to the manner described here. Furthermore, the actions of a block diagram need not be implemented in the order shown, nor does it necessarily have to be executed all actions. In addition, those actions that are not dependent on other actions can be executed in parallel with the other actions. The scope of embodiments is by no means limited by these specific examples.

[0054] Although the embodiments of the invention have been described in language relating to structural features and / or methods, it should be noted that the appended claims are not necessarily limited to the specific features or methods described. Rather, the specific features and methods are disclosed as examples of embodiments of the invention.

Claims

[1] A system (100) for secure MCP-mediated tool use by AI agents and generative AI / LLM services in cloud-native distributed applications, wherein the system (100) comprises: a KL agent interface (1) configured to receive natural language commands and application events from a variety of client applications and to generate appropriate tool call commands for one or more generative KL or Large Language Model (LLM) services; an MCP mediator service (2) that is configured to: (a) to convert the tool request requests into messages compatible with a model context protocol (MCP); and (b) to maintain the conversation context, including at least one of the following: user identity, tenant identity and application identity; a tool register (3) that stores a plurality of tool descriptions, each tool description defining at least a tool identifier, an input and output scheme, an endpoint location and allowed functions, wherein the tool register (3) is accessible to the MCP mediator service (2); a policy and security manager (4) configured to evaluate each MCP tool call against one or more security and access policies based on the conversation context and the corresponding tool description, and to issue a decision to allow, modify or block the tool call; a tool connector layer (5) comprising a plurality of tool adapters, each tool adapter being configured to communicate securely with a corresponding external tool, service or data source using credentials and permissions restricted according to the decision of the policy and security manager (4); an observation and audit manager (6) configured to record, for each tool call, at least a timestamp, the calling KL agent, the tool identifier, the policy decision, and a summary of the tool response, and to provide audit logs and metrics for monitoring and compliance purposes; and a cloud-native deployment controller (7) configured to provide the MCP mediator service (2), policy and security manager (4), tool connector layer (5) and observation and audit manager (6) as distributed microservices with network isolation between tenants in a cloud-native environment. [2] System (100) according to claim 1, wherein the MCP mediator service (2) is further configured to add a signed context token to each tool call, wherein the signed context token encodes at least the user identity, the tenant identity and the requested tool identifier. [3] System (100) according to claim 1, wherein the policy and security manager (4) comprises: a policy store (4a) configured to store declarative policies that define allowed tools, parameters, and data ranges for different tenants and roles; and a policy engine (4b) configured to evaluate the policies at runtime for each tool call. [4] System (100) according to one of the preceding claims, wherein the tool connector layer (5) is configured to support both synchronous tool calls and asynchronous, long-running tool tasks and to return partial results to the AI ​​agent interface (1) via the MCP mediator service (2). [5] System (100) according to any of the preceding claims, wherein the observability and audit manager (6) is further configured to generate a tool usage origin view that links a generated AI response with the sequence of tools, policy decisions and data sources used to create the response. [6] System (100) according to any one of the preceding claims, wherein the cloud-native deployment controller (7) is configured such that it: to orchestrate the microservices using a container orchestration platform; and Enforcement of network policies per tenant, so that tool adapters for one tenant are isolated from tool adapters of another tenant. [7] System (100) according to any of the preceding claims, further comprising a secret and key management module (8) configured to store, rotate and make available to the tool connector layer (5) short-lived access tokens, API keys and encryption keys based on the decisions of the policy and security manager (4). [8] System (100) according to one of the preceding claims, wherein the AI ​​agent interface (1) is further configured such that it: To receive security feedback and error signals from the policy and security manager (4) and the tool connector layer (5); and Providing appropriate line messages to the KL agents and LLM services to control future tool calls in accordance with the policies. [9] System (100) according to any of the preceding claims, wherein the MCP mediator service (2) is further configured to redact or mask sensitive fields in tool inputs and outputs according to the redaction rules defined in the tool registry (3) and the policy and security manager (4). [10] System (100) according to any of the preceding claims, wherein the cloud-native deployment controller (7) is configured to automatically scale the MCP mediator service (2) and the tool connector layer (5) based on the load metrics observed by the observability and audit manager (6).

Images