A multi-tenant-oriented sandbox-isolated agent execution method and system

Abstract

The application relates to a multi-tenant-oriented sandbox isolation agent execution method and system, which realizes physical-level safe isolation by creating file system isolated sandbox instances for different tenants and limiting operation boundaries; guarantees prompt cache hits to greatly reduce reasoning costs by preloading high-frequency tools based on user historical behaviors in the first round of sessions and locking the tool array; realizes the unity of context window capacity and information integrity by constructing a double-layer context management mechanism composed of trace compression driven by idle intervals and full compression driven by token thresholds, and cooperating with sandbox persistence of super-large tool results; realizes on-demand discovery of tools through a matching scoring mechanism supporting tool search requests, and guarantees the calling safety in a multi-tenant environment through in-sandbox execution. The application can be applied to an enterprise-level cloud intelligent agent service platform, and effectively improves the safety, economy and long-time operation stability of intelligent agent execution in a multi-tenant scenario.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and cloud computing, and in particular to a sandbox-isolated intelligent agent execution method and system for multi-tenant environments. Background Technology

[0002] In recent years, the rapid development of large language model technology has driven the widespread application of agent-based automated task execution systems in enterprise scenarios. In multi-tenant cloud service models, different enterprises or departments share underlying computing resources, and agents must achieve strict security isolation and resource control when performing operations such as file reading and writing, command execution, and API calls. To address these needs, academia and industry have conducted extensive research on agent collaborative scheduling, sandbox isolation, context compression, memory storage, and tool invocation.

[0003] In the field of agent collaborative scheduling, existing solutions mainly focus on task decomposition and multi-agent collaboration. For example, relevant patent literature proposes an agent collaborative question-answering method based on context awareness and access control, which decomposes tasks through a central scheduler and performs full-process verification and desensitization by an access control engine; there are also agent platforms based on the service-oriented architecture paradigm that call services through message matching mechanisms. The core focus of these solutions is on logical-level access control and task routing. However, the actual execution environment of agents lacks physical-level isolation for operating system file systems and network commands. Tool selection is usually statically completed by the scheduler rather than actively discovered by the agents, and they lack context compression and streaming interruption billing capabilities.

[0004] In terms of isolation technology, existing solutions cover various implementation paths, including sandbox isolation based on plug-in communication, class loader isolation based on process engines, and lightweight trusted container sandboxes, providing different levels of isolation capabilities from the perspectives of process communication, class loading mechanisms, and container infrastructure. However, these sandbox solutions mostly exist as independent operation and maintenance components at the infrastructure layer, targeting the communication middleware or container infrastructure domain. They have not yet been architecturally integrated with the state transitions and tool call loops of intelligent agents, nor have they formed a synergy with multi-tenant identity management.

[0005] To address the context window overflow problem caused by long dialogues, existing technologies have proposed various compression strategies, including using compression models to extract features and preserve details from the context, and dynamically selecting compression strategies based on the task scenario. In terms of agent memory management, some solutions also employ attention mechanisms for historical memory summarization, memory distillation through importance scoring and knowledge graph construction, or design multi-layered memory modules to avoid redundant storage. However, most of these solutions are single-layer compression structures, generating summaries of the entire dialogue or truncating and discarding information based on thresholds. They cannot simultaneously meet the dual requirements of ensuring the complete availability of recent context and preventing the loss of historical information, and they also lack a mechanism for processing extremely large results in conjunction with persistent sandbox storage.

[0006] Of particular note is that most mainstream large language model APIs currently support hint caching mechanisms, which cache the invariant prefix portion of a request using key-value (KV) state caching. A cache hit can reduce input token costs by up to 90% and first token latency by up to 85%. The core premise of a cache hit is that the tool array remains byte-level consistent across consecutive requests. However, existing tool scheduling schemes face a dilemma under this constraint: if all tool descriptions are injected into the context at once to maintain cache stability, the complete descriptions of dozens of tools will occupy tens of thousands of tokens, severely crowding out the limited context window; if incremental on-demand loading is used to save context space, each new tool will change the tool array structure, causing the entire cache prefix to become invalid, and the entire message history needs to be re-encoded. In long-conversation scenarios, a single cache miss requires re-encoding the entire historical message, costing several dollars, while the incremental cost of preloading several tools is only a fraction of that. Even more serious is that the cached information is usually only retained on the server for about five minutes. Once the idle interval of the conversation exceeds this time limit, the cache prefix will automatically expire. In this case, the existing solution still retains a large amount of old tool output in the context. This content can neither bring caching benefits, nor can it continuously occupy the coding space of thousands or even tens of thousands of tokens, resulting in a double waste.

[0007] Furthermore, existing agent memory solutions mostly rely on independent vector databases or memory modules. The memory data is independent of the sandbox file system, and the isolation between multiple tenants is maintained only by application-layer logic, lacking underlying physical isolation guarantees. Moreover, the memory retrieval process often synchronously blocks the main process response. In terms of billing, existing solutions only cover the execution path completed normally by the agent. When the client actively interrupts the streaming response, the consumed tokens cannot be accurately measured, and there is a lack of a unified billing strategy for different interruption times.

[0008] In summary, existing technologies exhibit a fragmented, single-point optimization pattern across five dimensions: sandbox isolation, tool scheduling, context compression, interruption billing, and memory storage. These mechanisms lack a unified anchor point—tool output cannot be automatically persisted to isolated storage, compression decisions are not linked to the sandbox file system, billing and metering are independent of sandbox resource usage, and memory storage is not bound to the tenant sandbox. The direct consequences of this isolated design are: agent execution lacks physical-level security isolation; there is a fundamental conflict between tool scheduling and caching mechanisms; context management in long-dialogue scenarios cannot balance integrity and window limitations; interruption billing has systemic vulnerabilities; and multi-tenant memory isolation relies on a weak logic control layer. These combined deficiencies make it difficult for existing systems to meet the comprehensive requirements of enterprise-level cloud-based agent services for security, economy, and efficiency.

[0009] In other words, existing technologies operate independently in various dimensions, lacking a collaborative architecture with a sandbox as a unified anchor point. This results in a lack of physical-level security isolation in agent execution, a fundamental conflict between tool scheduling and prompt caching, an inability of context management to balance integrity and window limitations, systemic vulnerabilities in interruption billing, and a disconnect between memory storage and multi-tenant isolation. Consequently, these technologies fail to meet the comprehensive requirements of enterprise-level cloud-based agent services for security, economy, and efficiency. Summary of the Invention

[0010] Therefore, this application provides a sandbox-isolated intelligent agent execution method and system for multi-tenant environments, aiming to overcome the problems existing in the prior art.

[0011] To achieve the above objectives, this application adopts the following technical solution:

[0012] Firstly, this application provides a sandbox-isolated intelligent agent execution method for multi-tenant environments, including: Receive requests sent by clients and extract tenant codes and user identifiers from the requests; A unique identifier is constructed based on the tenant code and user identifier. The corresponding sandbox instance is determined based on the unique identifier. The runtime context of the agent is constructed. The runtime context includes at least the sandbox instance and tool configuration information. The tool configuration information includes the core toolset that is always injected into the context of the large language model, and the deferred toolset that is not injected into the context by default. Based on the runtime context, in the first round of the session, the system retrieves a preset number of delayed tools with the highest call frequency within a preset review time window from the tool preference event library according to the user identifier, and injects the tool description of the user's preferred tool into the large language model context, locking the preferred tool description to remain unchanged during the session; the following large language model call process is executed in a loop until the response message returned by the large language model no longer contains tool call requests and tool search requests: Compress the current message history; The compressed message history and the current tool list are sent as the first request to the large language model; the current tool list includes the core toolset, user preference tools, and deferred tools that have been discovered and injected through the tool search request. Receive the response to the first request returned by the large language model; When the response to the first request contains a tool call request, the tool call request is executed within the sandbox instance. After the tool call request is executed, the result of the tool call request is added to the current message history. If the number of characters in the result of the tool call request exceeds a preset result threshold, the result of the tool call request is written to the file system of the sandbox instance, and the tool output in the message history is replaced with a placeholder containing a file reference identifier and a partial preview. When the response includes a tool search request, the tool search request carries the search query terms. The delayed toolset is matched and scored based on the search query terms. Several candidate delayed tools with the highest scores are selected and their complete tool descriptions are injected into the context of the large language model. At the same time, the current tool list is updated. If the response to the first request does not contain a tool invocation request or a tool search request, exit the loop and return the natural language content of the response to the first request as the final execution result to the client.

[0013] On the other hand, this application also provides a sandbox-isolated intelligent agent execution system for multi-tenancy, including: The tenant identification module is used to receive requests sent by clients and extract tenant codes and user identifiers from the requests. The sandbox management module is used to construct a unique identifier based on the tenant code and the user identifier, determine the corresponding sandbox instance based on the unique identifier, and construct the runtime context of the agent. The runtime context includes at least the sandbox instance and tool configuration information. The tool configuration information includes a core toolset that is always injected into the large language model context, and a delayed toolset that is not injected into the context by default. The context building module is used to, based on the running context, in the first round of the session, retrieve the preset number of delayed tools with the highest call frequency within the preset review time window from the tool preference event library according to the user identifier as preferred tools, inject the tool description of the user preferred tool into the large language model context, and lock the preferred tool description to remain unchanged during the session; The large language model invocation module is used to repeatedly execute the following large language model invocation process until the response message returned by the large language model no longer contains tool invocation requests and tool search requests: The tool prefers to preload modules for compressing the current message history; The compressed message history and the current tool list are sent as the first request to the large language model; the current tool list includes the core toolset, the user preference tools, and the delayed tools that have been discovered and injected through the tool search request. Receive the response to the first request returned by the large language model; When the response to the first request contains a tool invocation request, the tool invocation request is executed within the sandbox instance. After the tool invocation request is executed, the result of executing the tool invocation request is added to the current message history. If the number of characters in the result of the tool invocation request exceeds a preset result threshold, the result of the tool invocation request is written to the file system of the sandbox instance, and the tool output in the message history is replaced with a placeholder containing a file reference identifier and part of the preview content. When the response includes a tool search request, the tool search request carries a search query term. Based on the search query term, the delayed toolset is matched and scored. Several candidate delayed tools with the highest scores are selected, and their complete tool descriptions are injected into the context of the large language model. At the same time, the current tool list is updated. When the response to the first request does not contain a tool invocation request or a tool search request, the loop exits, and the natural language content of the response to the first request is returned to the client as the final execution result.

[0014] The application employs the above technical solution and has at least the following beneficial effects: This invention constructs a unique sandbox identifier based on tenant coding and user identification, creating sandbox instances that are file system isolated from each other for different tenants, and restricting all file operations and command execution to the root directory subtree of the sandbox instance. Tenant A's agent cannot access Tenant B's sandbox file system, and all file operations are subject to path boundary checks, achieving physical isolation between multiple tenants at the operating system level. The shortcomings of existing technologies, such as the separation of the sandbox and the agent execution engine and the reliance on application-layer logic, are fundamentally solved in this invention, thus meeting the security and compliance requirements of enterprise-level cloud services.

[0015] Meanwhile, this invention, in the first round of the session, retrieves a preset number of delayed tools with the highest call frequency within a preset review time window from the tool preference event library based on the user identifier, and uses them as preferred tools. The complete tool description is injected into the large language model context and locked, ensuring that the preferred tools remain visible throughout the session and that the tool array remains unchanged. The hint caching mechanism of large language models requires the tool array to remain consistent at the byte level between consecutive requests to hit the cache. However, in existing progressive tool discovery schemes, each on-demand loading of a new tool causes a change in the tool array, invalidating the entire cache prefix and requiring recoding of all historical messages. In long-conversation scenarios, the recoding cost for a single cache miss is extremely high. This invention locks the tool array in the first round of the session, ensuring that subsequent requests continuously hit the hint cache, significantly improving the cache hit rate and significantly reducing inference costs and response latency in long-conversation scenarios.

[0016] In terms of context management, this invention performs dual context compression before each large language model call. The first compression, for scenarios where the idle interval exceeds a preset threshold, removes historical tool outputs exceeding the preset retention limit from the context view and records them in a compressed ledger. These compressed placeholders are then replaced during request construction, freeing up context space without modifying the original message sequence, allowing the front-end presentation layer to still display the complete history. The second compression is triggered when the number of tokens used reaches a preset full compression threshold. It calls the large language model to generate a summary to replace the original message history and configures a circuit breaker to prevent compression deadlock. Compared to existing single-layer compression schemes, this invention, through a dual-layer collaborative mechanism, simultaneously satisfies the dual requirements of "complete and usable recent context and no loss of historical information," effectively solving the context window overflow problem in long dialogue scenarios.

[0017] Regarding tool output processing, this invention detects the number of characters output by the tool after execution. When the number exceeds a preset threshold, the complete tool output is written to the file system of the sandbox instance. Only placeholders containing file reference identifiers and partial preview content are retained in the message history. When the complete content is needed, it can be read back from the sandbox file system through the file reference identifier. Large tool outputs are no longer truncated and lose key information, nor do they occupy the context window, achieving a unified approach of "no information loss, no window overflow." Simultaneously, sandbox isolation ensures that persistent content is only accessible within the tenant's sandbox.

[0018] In terms of tool discovery and execution, this invention supports large language models to proactively initiate tool search requests. The system matches and scores the delayed toolset based on search query terms (name exact match has the highest weight, followed by semantic match of function description, and then weighted cumulative score based on multiple keywords). After selecting several candidate delayed tools with the highest scores, their complete tool descriptions are injected into the context, allowing the agent to proactively discover and load tools according to task requirements, rather than passively receiving the entire tool list or relying on static allocation by the scheduler. Combined with skill scanning within the sandbox, different tenant sandboxes contain different skill files, naturally achieving tenant-level differentiated toolsets. All tool calls are executed within the sandbox instance, providing execution isolation guarantees at the operating system file system level. Even if a single tool call has security vulnerabilities such as path traversal, the attack behavior is blocked by the sandbox boundary and cannot affect other tenant data or the host system.

[0019] This invention employs a closed-loop process: cyclical execution of context compression processing → sending requests to a large language model → receiving responses → tool execution or tool search. This process continues until the response returned by the large language model no longer contains tool call or tool search requests. This enables the agent to autonomously complete complex tasks requiring multiple rounds of tool calls, while continuously performing context compression in each loop to ensure system stability in long-term operating scenarios. A three-tiered tool supply system—constantly injecting core tools to maintain basic execution capabilities, locking preferred tool sessions in the first round to ensure cache hit rates and reduce the cost of high-frequency tool calls, and loading delayed tools on demand through tool search to avoid context waste—works collaboratively to ensure the system simultaneously achieves guarantees in terms of cost, performance, and functional integrity. The shortcomings of existing technologies, such as isolated points in each dimension and a lack of architecture-level collaboration, are fundamentally solved in this invention.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating a sandbox-isolated intelligent agent execution method for multi-tenant environments, according to an exemplary embodiment. Figure 2 This is a schematic diagram illustrating the composition of a sandbox-isolated intelligent agent execution system for multi-tenancy, according to an exemplary embodiment. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be described in detail below. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other implementation methods obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Example 1 Please see Figure 1 This embodiment provides a sandbox-isolated intelligent agent execution method for multi-tenant environments, including the following steps.

[0025] Step S1: Authentication and Tenant Identification. The system receives the request sent by the client and extracts the tenant code and user identifier from the request header or authentication token. To clearly describe the technical solution of this invention, a symbol system is first defined. Referring to Table 1, let the tenant code be C and the user identifier be U, both of which are string types. The sandbox identifier of user U under tenant C is denoted as S. C,U This identifier is a string and is globally unique. The system verifies tenant validity and user permissions through a unified authentication layer. After successful authentication, C and U are injected into the request context to ensure that all subsequent operations are performed within the tenant isolation boundary, preventing cross-tenant data leakage.

[0026] Table 1

[0027] Step S2: Sandbox Acquisition or Creation. The system constructs a unique sandbox identifier S based on the tenant code C and the user identifier U. C,U This identifier is used to obtain or create a sandbox instance. Sandbox S C,U The root directory path is denoted as R s The file path to be operated on is denoted as P. Different tenant codes correspond to different sandbox instances, and the file systems of each sandbox instance are isolated from each other. A sandbox instance is an execution environment isolated by the operating system's file system; file operations and command execution are confined to the root directory subtree of the sandbox instance. All file operations enforce sandbox path escape verification, the core formula of which is:

[0028] If path The normalization result is not the sandbox root directory. subpaths (i.e.) If the system detects an escape path, it will throw a sandbox exception and block the operation. This mechanism mathematically ensures strong isolation of the multi-tenant file system. After the sandbox is created, a bootstrapping operation is performed, creating a standard subdirectory structure under the sandbox root directory, including a memory directory, a tool results directory, and a skills directory. It also checks the sandbox environment information, including the working directory, shell type, operating system version, and whether it is a Git repository, injecting this environment information into the runtime context.

[0029] Step S3: Construct the runtime context. The system constructs the agent's runtime context, which includes at least a sandbox instance and tool configuration information. Tool configuration information includes the core toolset, which is always injected into the large language model context, and a deferred toolset, which is not injected into the context by default. The core toolset includes runtime-essential tools such as file reading / writing, command execution, and content search; its tool definitions always appear in the tools array during each large language model call. The deferred toolset includes all other external tools and business system interface tools; its tool definitions are not injected into the context by default and are discovered on demand by the agent through tool search. The runtime context also includes model configuration (model identifier, temperature parameters, maximum number of output tokens, whether to enable think mode), compression configuration, and memory configuration.

[0030] Step S4: Preloading and Locking Preference Tools. Based on the agent's runtime context, in the first round of the session, the system retrieves the preset number of delayed tools with the highest call frequency within the preset review time window from the tool preference event library according to the user identifier, and injects the complete tool description of the preference tool into the large language model context, and locks the complete tool description of the preference tool to remain unchanged during the session.

[0031] Step S5: Execute the large language model invocation process repeatedly. The system executes the following large language model invocation process repeatedly until the response returned by the large language model no longer contains tool invocation requests and tool search requests.

[0032] Step S5.1: Context Compression Processing. Referring to Table 2, perform context compression processing on the current message history to obtain the compressed message history. To accurately describe the compression logic, the following symbols are defined: the idle interval threshold is denoted as T. gap The unit is seconds, and the value ranges from 600 to 7200 seconds; the actual time difference between the two most recent messages is denoted as T. idle The unit is seconds; the number of recently called tools retained is denoted as K, a dimensionless positive integer ranging from 3 to 10; the number of tokens occupied by the current context is denoted as W. ctx The full compression trigger threshold is denoted as W. threshold The unit is tokens, and the value is typically 80% to 95% of the model context window; the maximum number of consecutive failures of the circuit breaker is denoted as f.max , is a dimensionless positive integer, with a value range from 2 to 5.

[0033] Table 2

[0034] Context compression processing includes two layers of compression mechanisms.

[0035] The first layer is micro-compression. It detects the time difference between the two most recent messages in the message history. When the micro-compression trigger condition is met, it maintains a compression ledger, removes historical tool outputs exceeding the preset retention limit k from the context view, and replaces the removed tool outputs with compression placeholders during the build request. The formula for the micro-compression trigger condition is: .

[0036] The second layer is full compression. It detects the token holding count Wctx in the current context. When the full compression trigger condition is met, it calls a preset large language model to generate a digest to replace the original message history. Let f be the current consecutive compression failure count. fail The formula for triggering full compression is:

[0037] in This represents the current consecutive compression failure count.

[0038] Step S5.2: Send a request. The compressed message history and the current tool list are sent as a request to the large language model. The current tool list includes the core toolset, preferred tools, and deferred tools previously discovered and injected through the tool search request. The system prompt includes sandbox environment information, a list of available skills, and injected memory entries, enabling the model to understand the current sandbox environment and available capabilities.

[0039] Step S5.3: Receive and process the response. Receive the response returned by the large language model.

[0040] When a response includes a tool invocation request, the tool invocation is executed within the sandbox instance, and the tool output is added to the message history. Tool execution is performed through the sandbox file system interface, and all operations are subject to sandbox path escape checks. If the number of characters in the tool output exceeds a preset result threshold, the complete tool output is written to the sandbox instance's file system, and the tool output in the message history is replaced with a placeholder containing a file reference identifier and a partial preview. The result threshold ranges from 30,000 to 100,000 characters, with an example value of 50,000 characters. The preview character count ranges from 1,000 to 5,000 characters, with an example value of 2,000 characters. The placeholder contains a file reference identifier (pointing to the complete file within the sandbox), a preview of the first few characters, and original size information. When the agent needs the complete content, it can read back the sandbox file using a file reading tool, achieving "no information loss + no window overflow."

[0041] When a response includes a tool search request, the delayed toolset is matched and scored based on the search query terms. The top-scoring candidate delayed tools are selected, and their complete tool descriptions are injected into the context of the large language model. At the same time, the current tool list is updated.

[0042] Step S6: Dialogue End Processing. The dialogue ends when the large language model stops returning tool call requests and tool search requests. The system summarizes all model call usage during the agent's execution, deducts fees from each tenant's sub-account, and records usage details. Simultaneously, a background memory retrieval task is initiated to analyze the current dialogue content, identify new memories worth persisting, write these new memories to the sandbox memory directory, and persist them to the database.

[0043] This embodiment organically integrates security isolation, tool discovery and preference preloading, context double-layer compression, persistence of ultra-large results, interruption billing, and automatic memory through a six-layer collaborative architecture anchored by a sandbox. Tool results are persisted in the sandbox, the summary placeholder for interruption billing points to the sandbox, the memory file is stored in the sandbox, skill scanning is performed within the sandbox, and tool preference data is persisted across sessions at the user level. This forms an architectural innovation that cannot be naturally derived from the single-tenant intelligent agent paradigm.

[0044] Example 2 This embodiment, based on embodiment one, further adds a background memory retrieval and zero-wait consumption injection mechanism.

[0045] During multi-turn dialogues between an agent and a user, the user may mention important information such as personal preferences and work context in previous conversations. If this information can be persisted across dialogues and retrieved and injected as needed in new dialogues, the agent's response quality and personalization can be significantly improved. However, the memory retrieval process has a certain delay; if the system waits for the retrieval to complete synchronously, it will directly affect the system response time.

[0046] The method in this embodiment includes: initiating a background memory retrieval task at the start of a session. This task retrieves relevant memory entries from memory files within the sandbox instance based on the current conversation content. Memory entries are stored in a structured file format in a designated directory within the sandbox. The index file lists the paths and summaries of all memory entries. Each memory entry is an independent file containing metadata declarations, including name, summary, and category. Memory reads and writes are performed through a unified asynchronous interface within the sandbox, inherently isolated by tenant. Memory files from different tenants are stored in different sandboxes and cannot access each other.

[0047] During the loop of the large language model invocation process, the system checks whether the background memory retrieval task is complete. Specifically, in each round of tool invocation, the system checks the status of the background memory retrieval task. If the background memory retrieval task is complete, the retrieved memory entries are extracted, injected into the message history using a prompt message, and a deduplication flag is set. The deduplication flag can be the memory entry path, ensuring that the same memory entry is injected only once within the same round of dialogue, avoiding redundant context caused by repeated injection. If the background memory retrieval task is not complete, the injection is skipped, and the task status is checked again in the next round of tool loop. This zero-wait consumption mechanism allows memory retrieval to be executed in parallel with the main process, and the retrieval delay does not affect the response time of the main process. After the dialogue ends, the system starts the background memory retrieval task, analyzes the current dialogue content, identifies new memories worth persisting, writes the new memories to the sandbox memory directory, and persists them to the database. This dual-write ensures both multi-tenant isolation and cross-session retrieval.

[0048] This embodiment enables memory retrieval and the main process to be executed in parallel through background asynchronous retrieval and zero-wait consumption mechanism, so that retrieval delay does not affect the response time of the main process; at the same time, it uses sandbox file system to store memory, naturally achieving multi-tenant physical isolation, and memory retrieval and the main process of the intelligent agent form non-blocking collaboration.

[0049] Example 3 This embodiment adds a three-scenario interruption billing mechanism based on any of the above embodiments.

[0050] In scenarios where the client actively interrupts the streaming response, the resources consumed vary depending on the timing of the interruption. Existing solutions only cover the normal completion path; when the client actively interrupts the streaming response, the generated tokens cannot be accurately measured, leading to billing vulnerabilities and audit gaps. Furthermore, there is a lack of a unified metering strategy for different interruption times, including during streaming, between models, and during quiet periods.

[0051] The method in this embodiment includes determining the timing of an interruption upon receiving an interrupt signal from the client. Specifically, it is divided into the following three scenarios.

[0052] Scenario 1: Model Streaming Interruption. If the interruption occurs during the streaming output of the large language model, the system estimates token usage based on the already output text and constructs a composite billing detail for deduction. Specifically, the system detects currently incomplete model calls, obtains the already output text, which includes inference content and main text content. It estimates the prompt usage based on the current input, estimates the completion usage based on the already output text plus the interruption flag, separately measures the inference content and main text content, constructs a composite billing detail, and writes it to the database.

[0053] Scenario 2: Inter-model Interruption. If an interruption occurs between two large language model calls (i.e., after the tool finishes execution and before the next model call), the actual billing of the completed model call is retained, and a new billing detail is added to account for the token usage at the end of the interruption. The new detail uses an independent idempotent key. The purpose of the independent idempotent key is to prevent duplicate billing and ensure that the same interruption event is billed only once. In this scenario, the context generated by the executed tool has been recorded. The new detail carries the token usage at the end of the interruption, and the actual billing of the completed call is retained to avoid duplicate metering.

[0054] Scenario 3: Silent Interruption. If an interruption occurs during a silent waiting period, i.e., when nodes are silently waiting for each other, such as waiting for user input or for background tasks to complete, an interruption flag will be added to the last billing detail, and no new charges will be incurred. In this case, the system does not consume large language model computing resources; it only marks the interruption status to support audit tracing.

[0055] This embodiment is the first to systematically solve the billing and metering problems in three scenarios: model streaming interruption, inter-model interruption, and silent interruption. Separating inference and text measurement ensures audit integrity, and the billing accuracy is close to 100% under any interruption time, eliminating billing loopholes.

[0056] Example 4 This embodiment further defines the process of obtaining or creating the above-mentioned sandbox instance.

[0057] In multi-tenant, high-concurrency scenarios, the same user within the same tenant may initiate multiple concurrent requests. If each request simultaneously attempts to create a sandbox instance, it will lead to duplicate creation and wasted resources. It is essential to ensure the idempotency of sandbox creation—regardless of the number of creation requests received, only one corresponding sandbox instance will ultimately exist.

[0058] In this embodiment, the process of retrieving or creating a sandbox instance based on a unique identifier includes three execution branches: attempting to retrieve an existing sandbox instance based on the unique identifier; performing a creation operation if the sandbox instance does not exist; and performing a retrieval operation to retrieve the same sandbox instance created by another request if a concurrency conflict is detected during the creation operation.

[0059] All file operations within the sandbox instance, including reading, writing, editing, and execution, are performed through a unified sandbox backend protocol layer, with mandatory path escape verification. Specifically, let the sandbox root directory be R. s The path to be operated on is P. Before executing the operation, the system verifies the path according to Formula 1. If the verification fails, a sandbox exception is thrown and the operation is blocked. This path escape verification mechanism, combined with the aforementioned concurrency conflict degradation strategy, jointly ensures the security and idempotency of sandbox operations in multi-tenant scenarios.

[0060] Concurrency conflict detection can be achieved through database unique constraints or distributed lock mechanisms. When a creation request fails due to a violation of a unique constraint, the system catches the exception and converts it into a fetch operation, retrieving the sandbox instance successfully created by the concurrent request. The sandbox backend protocol layer provides a unified asynchronous file operation interface, including reading, writing, editing, executing, searching, uploading, downloading, and environment information retrieval. It supports transparent switching between local path-isolated backends and remote cloud sandbox backends, allowing for one-click switching via configuration options, so that upper-layer modules are unaware of the underlying implementation differences when accessing the sandbox. The local path-isolated backend uses a specified directory as the sandbox root on the host machine, and all file operations are restricted to subtrees within this directory. The remote cloud sandbox backend interfaces with any cloud sandbox platform, managing the sandbox lifecycle through the platform SDK, including creation, fetching, starting, stopping, and archiving operations.

[0061] This embodiment ensures high concurrency idempotency in sandbox creation through three-way branching, while enabling pluggable switching between local and remote backends through the backend protocol layer, so that the business logic is unaware of the underlying differences.

[0062] Example 5 This embodiment further defines the process of obtaining the preference tool described above.

[0063] In progressive tool discovery mechanisms, agents search for tools and load deferred tools on demand. However, each on-demand loading of a new tool causes changes to the tool array, disrupting the hint cache prefix of large language model APIs. Mainstream large language model APIs include the tool definition array in their prefix cache, with the core condition for a cache hit being byte-level consistency of the tool definition array across consecutive API requests. Once the tool array changes, subsequent requests require recoding the entire message history. In long-conversation scenarios with over 100,000 tokens in the context, the recoding cost for a single cache miss can reach several dollars, while the incremental context cost of preloading several tools is only a fraction of that.

[0064] To accurately describe the selection logic of preference tools and the guarantee of cache stability, the following special symbols are defined, as shown in Table 3: Table 3

[0065] In this embodiment, the method of retrieving a preset number of delay tools with the highest call frequency within a preset review time window from the tool preference event library based on the user identifier includes: counting the call frequency of each delay tool by the user identifier within the preset review time window, sorting them in descending order of frequency, and taking the intersection with the set of delay tools available in the current session, and then taking the top preset number of these as the preferred tools; each event record in the tool preference event library contains at least the user identifier, the tool name, and the call timestamp.

[0066] Specifically, for users Statistics Review Window Frequency of use of each delay tool:

[0067] in This is the name of the delay tool. This is the current time.

[0068] The preference tools list is the top of the intersection of the preference tools list and the available toolset, sorted in descending order of frequency. indivual:

[0069] The preference list is locked after the first round of the session and will not be changed in subsequent rounds. This means that frequently used tools do not need to be loaded on demand by ToolSearch throughout the session, significantly reducing the frequency of tool array changes and improving the overall hit rate of the suggestion cache.

[0070] Note: Cache stability is guaranteed. Let the session number be... The tool array of the wheel is T (r) Preloading is preferred to guarantee:

[0071] This means the utility array remains constant throughout the session, ensuring byte-level consistency of the hint cache prefix for the large language model API across consecutive requests, maximizing cache hit rate. In long-conversation scenarios, cache hits can reduce input token costs by up to 90%, first token latency by up to 85%, and overall inference costs per session by approximately 40% to 60%.

[0072] The tool preference event library is persistently stored in a separate event log table at the user level, recording complete context dimensions, including fields such as user identifier, company code, company type, department, tool name, tool type, and invocation time. After tool execution, the event log is asynchronously written using a fire-and-forget method, without blocking the main process. The complete context dimension design of the event log supports frequency aggregation queries for the current version while reserving extensible feature input interfaces for subsequent integration with classification or sequence models.

[0073] This embodiment achieves an optimal balance between progressive tool discovery and the stability of prompt caching by preloading preferences based on user historical behavior patterns.

[0074] Example 6 This embodiment further defines the matching and scoring process for tool search described above. When the agent searches for delayed tools on demand, it sends a search query term along with the tool search request. The system needs to match the search query term with the description information of each delayed tool to determine the most relevant candidate tool to return. The accuracy of the matching score directly affects whether the agent can efficiently find the required tool. If the matching result is inaccurate, it will cause the agent to repeat the search or fail to find the correct tool, increasing unnecessary model call rounds.

[0075] As shown in Table 4, the following special symbols are defined to accurately describe the scoring logic of the tool discovery.

[0076] Table 4

[0077] Table 5

[0078] Specifically, in this embodiment, the delay toolset is matched and scored based on the search query term q, and several candidate delay tools with the highest scores are selected, including: for each tool d in the delay toolset... i The name matching score, function description semantic matching score, and search suggestion matching score are calculated separately, and then weighted and summed according to preset weights to obtain the tool's cumulative score; where the preset weight W for name matching is... name The preset weight W for semantic matching greater than the functional description desc Furthermore, the preset weight for semantic matching of the function description is greater than the preset weight W for matching search suggestions. hint Select the Kresult candidate delay tool with the highest cumulative score.

[0079] The scoring process can be formally represented by the following core formula:

[0080] in These are weighting coefficients. Specifically, exact name matching uses a string matching algorithm, where the highest single score is obtained when the search query exactly matches the tool name. Semantic matching of function descriptions uses a text matching algorithm based on word vector cosine similarity. The search query and the function description text of each tool are converted into vectors, and the cosine similarity is calculated as the semantic matching score. Search suggestion matching matches the tool's search suggestion keyword tags. In multi-keyword searches, the matching scores of each keyword can be weighted and accumulated to comprehensively reflect the degree of matching between the tool and the query intent.

[0081] This embodiment employs a hierarchical weighted scoring mechanism, prioritizing name matching, followed by semantic matching, and then suggestion matching. This aligns with the actual retrieval habits of intelligent agents, thereby improving the accuracy and efficiency of tool discovery.

[0082] Example 7 This embodiment further defines the compressed ledger maintenance process of the aforementioned micro-compression. Micro-compression is the first-layer context compression mechanism of this invention, designed to remove old tool outputs that have not been used for a long time and free up context window space. The core design principle of micro-compression is: not to modify the original message sequence, but only to maintain the compressed ledger records of the removed tool outputs, and to construct the compressed context view by real-time projection of the ledger during model invocation. This ledger-based projection mechanism allows the front-end presentation layer and the model invocation layer to use different context views—the front-end can display the complete history, and old outputs are compressed during model invocation, with the compression being transparent to the user.

[0083] In this embodiment, maintaining a compressed ledger involves removing historical tool outputs that exceed a preset retention quantity K from the context view and replacing the removed tool outputs with compressed placeholders during the build request, including the following steps.

[0084] Step 1: Idle Interval Detection and Filtering. Iterate through the tool call messages in the message history, filtering out messages whose creation time is earlier than the current time minus the idle interval threshold t. gap The tool call message. Let the current time be t. now The filtering criteria are: the creation time t of the tool call message. create Satisfy t create <t now -t gap .

[0085] Step 2: Sorting and Exclusion. Sort the filtered tool call messages in descending order of creation time. Exclude the K tool call messages with the latest creation time. Record the tool call identifiers corresponding to the remaining tool call messages in the compression ledger. The compression ledger is a dictionary structure, where the key is the tool call identifier and the value is a compression entry containing the compression reason. An example of a compression reason is "idle timeout".

[0086] Step 3: Projection Replacement. When constructing the message sequence to be sent to the large language model, the compressed ledger is queried. Tool outputs in the message sequence that identify messages existing in the compressed ledger are replaced with compressed placeholders, while preserving the original message history. An example of a compressed placeholder is that the tool output has been compressed during idle periods.

[0087] For a formal representation of the micro-compression trigger condition, see Formula 2. idle The value is set much larger than the TTL of the large language model API hint cache (usually about 5 minutes), with an example value of 3600 seconds. The reason for this setting is that if the idle interval is only a few minutes, the cache prefix is ​​likely still valid, and retaining the old tool output can assist the model's decision-making without adding extra re-encoding costs. However, once the idle interval significantly exceeds the cache TTL, the cache prefix automatically becomes invalid, and the old tool output has very little reference value for the current task. Removing them can free up context space and reduce the number of tokens that need to be re-encoded in the next request. Micro-compression primarily targets tools with large output volumes and high time sensitivity, typically including Bash command execution logs, file reading tool Read output, and WebSearch / WebFetch search results. Removing these typically frees up about 30% to 50% of the context space.

[0088] This embodiment achieves the separation of the context view between the presentation layer and the model calling layer through a ledger-style projection mechanism, and the compression is transparent to the user, allowing the user to view the complete dialogue history at any time.

[0089] Example 8 This embodiment further defines the execution process of the full compression described above. Full compression is the second-layer context compression mechanism of this invention, triggered when the number of context tokens reaches a preset threshold. It calls a large language model to generate a summary of the entire historical dialogue to replace the original message sequence. Full compression must ensure that the summary contains key information, including user intent, tool call results, and important decisions. Simultaneously, a circuit breaker mechanism prevents compression failures from causing an infinite loop.

[0090] In this embodiment, referring to Table 5, when the number of tokens occupied reaches the preset full compression threshold W... threshold When this happens, a large language model is invoked to generate a summary to replace the original message history, including the following steps.

[0091] Step 1: Trigger detection. The system detects that the number of tokens held in the current context has reached the preset full compression threshold W. threshold At this time, initiate the full compression process. Let f be the current number of consecutive compression failures. fail The upper limit of the fuse is f max (Values ​​range from 2 to 5; example, 3 values ​​are used). Triggering condition is... .

[0092] Step 2: Tool Invocation Pairing Processing. Each tool invocation message in the message history is associated with its corresponding tool output message as a processing unit. Specifically, tool invocation request messages from the same round in the message history are paired with their corresponding tool invocation result messages, and compressed suggestions are submitted in pairs. This process aims to prevent the loss of key contextual information in the digest due to the separation of tool invocation and result. Simultaneously, existing compression boundary messages, including previous digest messages, are removed to avoid digest nesting affecting digest quality.

[0093] Step 3: Invoke the model to generate the summary. A compressed prompt is constructed using processing units, and the large language model is invoked to generate the summary. The compressed prompt serves as a template of instructions to guide the model in generating the summary, containing explicit requirements for retaining key information such as user intent, tool call results, and important decisions.

[0094] Step 4: Digest Verification and Replacement. If the digest is generated successfully, the original message history is replaced with the digest, the compression boundary marker is updated, and the consecutive failure count is reset to 0. Digest verification includes checking whether the digest length is empty and whether it is below the preset minimum length threshold.

[0095] Step 5: Fuse Handling. If summary generation fails, the consecutive failure count is incremented. When the consecutive failure count reaches the preset fuse limit f, the limit is reached. max When compression fails, full compression is stopped and the original message history remains unchanged. The criteria for summary generation failure include: the model returning an error, the summary being empty, or the summary length being lower than the preset minimum length. Compression attempts are stopped once the circuit breaker limit is reached, ensuring that the agent can continue execution without interruption due to compression failure.

[0096] This embodiment ensures summary quality through pairing processing and prevents dead loops through a circuit breaker mechanism, thus ensuring the reliability and security of full compression. Full compression can reduce context by approximately 60% to 80%.

[0097] Example 9 This embodiment further adds a sandbox skill scanning mechanism based on any of the above embodiments.

[0098] In multi-tenant scenarios, different tenants may customize different professional skills, such as code review, documentation generation, and test assistance. These skills are deployed in the form of declaration files within each tenant's sandbox and need to be dynamically scanned and discovered during the build runtime context, enabling the agent to perceive the available capabilities within the current sandbox. Since the skill declaration files are stored within the sandbox, different tenants' sandboxes contain different skill files, naturally achieving tenant-level differentiated skill sets.

[0099] The method in this embodiment includes: when constructing the agent runtime context, performing a skill scan within the sandbox instance, enumerating skill declaration files in a specified directory through the sandbox file system interface, parsing the structured metadata in the skill declaration files to generate a tenant-specific list of available skills, and injecting the list of available skills into the agent runtime context.

[0100] The specific process of skill scanning is as follows: A directory enumeration operation is performed on the specified skill directory through the sandbox file system interface to obtain the names of all skill subdirectories; for each subdirectory, the skill declaration file (e.g., SKILL.md) is read, and the YAML frontmatter is parsed to extract structured information such as skill name, function description, applicable scenarios, and whether it is user-invokeable. Successfully parsed skills are constructed into a list of available skills and injected into the skill fields of the runtime context. Because the scanning is performed within the sandbox, different tenants' sandboxes contain different skill files, naturally achieving tenant-level differentiated skill sets. After the scan results are injected into the context, the agent can perceive the currently available skills when calling the large language model, without requiring additional discovery steps.

[0101] During the tool invocation process, a three-tiered security strategy can be combined to achieve defense in depth: the first tier requires manual approval, and high-risk operations require user confirmation before execution; the second tier requires mandatory interaction, which pauses and waits for user interaction even in automatic execution mode; the third tier is sandbox path protection, which adds inner path protection on the basis of the aforementioned path escape verification to form a defense in depth system.

[0102] This embodiment achieves tenant-level differentiated toolsets through sandbox skill scanning. Skill declaration files are stored in isolation along with the sandbox instance, ensuring secure isolation and dynamic discovery of skill sets in multi-tenant scenarios.

[0103] Example 10 Please see Figure 2 This embodiment provides a sandbox-isolated intelligent agent execution system for multi-tenants, applicable to any of the above embodiments, and includes the following modules.

[0104] Tenant identification module 10 is used to receive client requests and extract the tenant code and user identifier from the requests. This module extracts the tenant code and user identifier from the request header or authentication token, verifies the tenant's validity and user permissions, and injects the tenant code and user identifier into the request context after successful authentication.

[0105] The sandbox management module 20 is used to construct a unique sandbox identifier based on the tenant code and user identifier, and to obtain or create a sandbox instance corresponding to the tenant code based on this identifier. Different tenant codes correspond to different sandbox instances, and the file systems of each sandbox instance are isolated from each other. A sandbox instance is an execution environment isolated by an operating system-level file system; file operations and command execution are confined to the root directory subtree of the sandbox instance. The sandbox management module provides an asynchronous file operation interface through a unified sandbox backend protocol layer, supporting transparent switching between local path-isolated backends and remote cloud sandbox backends. All file operations undergo mandatory path escape verification before execution: if the normalized result of the path to be operated on is not a subpath of the sandbox root directory, the verification fails, and an exception is thrown to block the operation.

[0106] The context building module 30 is used to construct the agent's runtime context. The runtime context includes at least a sandbox instance and tool configuration information. The tool configuration information includes the core toolset that is always injected into the large language model context, as well as a deferred toolset that is not injected into the context by default. The context building module is also responsible for performing skill scanning within the sandbox, enumerating skill declaration files through the sandbox file system interface, parsing structured metadata to generate a tenant-specific list of available skills, and injecting the list of available skills into the runtime context.

[0107] The tool preference preloading module 40, in the first round of the session, retrieves a preset number of delayed tools with the highest call frequency within a preset review time window from the tool preference event library based on the user identifier. It then injects the complete tool descriptions of these preferred tools into the large language model context and locks these descriptions to remain unchanged throughout the session. Based on the user's historical tool call event logs, the module calculates the call frequency of each delayed tool within the review time window, sorts them in descending order of frequency, and takes the intersection of this sorted list with the available toolset, selecting the top preset number of tools as the preferred tool list. This preferred tool list is persisted across rounds using a checkpointing mechanism, ensuring the tool array remains unchanged throughout the session and maximizing the cache hit rate.

[0108] The large language model invocation module 50 is used to repeatedly execute the following large language model invocation process until the response returned by the large language model no longer contains tool invocation requests and tool search requests: Context compression processing is performed on the current message history to obtain a compressed message history; the context compression processing includes: detecting the time difference between the two most recent messages in the message history; when the time difference is greater than a preset idle interval threshold, maintaining a compression ledger, removing historical tool outputs exceeding the preset retention limit from the context view, and replacing the removed tool outputs with compression placeholders when constructing the request; and detecting the token occupancy count of the current context; when the token occupancy count reaches a preset full compression threshold and the number of consecutive compression failures has not reached the circuit breaker limit, calling the preset large language model to generate a digest to replace the original message history; and sending the compressed message history and the current tool list as a request to the large language model; the current tool list includes core tools. The system includes toolsets, preference tools, and deferred tools previously discovered and injected through tool search requests; it receives responses from the large language model; when a response contains a tool call request, it executes the tool call within the sandbox instance and adds the tool output to the message history; if the number of characters in the tool output exceeds a preset result threshold, it writes the complete content of the tool output to the file system of the sandbox instance and replaces the tool output in the message history with a placeholder containing a file reference identifier and partial preview content; when a response contains a tool search request, it performs a matching score on the deferred toolset based on the search query terms, and performs a weighted summation based on preset weights, with name exact matching having the highest weight, followed by semantic matching of function descriptions, and then matching of search suggestions, to obtain the cumulative score of each tool; it selects several candidate deferred tools with the highest scores, injects their complete tool descriptions into the context of the large language model, and updates the current tool list.

[0109] All modules access the sandbox instance uniformly through the sandbox backend protocol layer, with the sandbox management module serving as the fundamental anchor point for all other modules. The tool preference preloading module executes tool calls and records preference events within the sandbox; the context building module performs skill scanning within the sandbox; the large language model invocation module writes massive results to the sandbox and points context compression placeholders to sandbox files; the billing layer's interrupted billing summary placeholders point to the sandbox; and the memory layer's memory files are stored in the sandbox. All modules form a closed-loop collaborative architecture anchored by the sandbox, organically integrating six mechanisms: security isolation, tool discovery and preference preloading, double-layer context compression, persistence of massive results, interrupted billing, and automatic memory.

[0110] The device in this embodiment uses a sandbox as an anchor point to achieve six-layer collaboration. Tool results are persistently written to the sandbox, interrupt billing placeholders point to the sandbox, memory files are stored in the sandbox, skill scanning is performed within the sandbox, and tool preference data is persisted across sessions at the user level, forming a sandbox-isolated intelligent agent execution engine for multi-tenants.

[0111] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0112] It should be noted that in the description of this application, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "multiple" or "more" means at least two.

[0113] It should be understood that when an element is referred to as "fixed to" or "set on" another element, it may be directly on the other element or may have an intervening element present at the same time; when an element is referred to as "connected to" another element, it may be directly connected to the other element or may have an intervening element present at the same time. In addition, the term "connected" as used herein may include wireless connections; the word "and / or" as used includes any unit and all combinations of one or more of the associated listed items.

[0114] Any process or method description in the flowchart or otherwise herein can be understood as: representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the function involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0115] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0116] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium. When executed, the program includes one or a combination of the steps of the method embodiments.

[0117] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0118] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0119] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0120] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A sandbox-isolated intelligent agent execution method for multi-tenant environments, characterized in that, The method includes: Receive a request sent by the client and extract the tenant code and user identifier from the request; A unique identifier is constructed based on the tenant code and the user identifier. The corresponding sandbox instance is determined based on the unique identifier. The runtime context of the agent is constructed. The runtime context includes at least the sandbox instance and tool configuration information. The tool configuration information includes a core toolset that is always injected into the large language model context, and a deferred toolset that is not injected into the context by default. Based on the running context, in the first round of the session, the user identifier is used to retrieve the preset number of delayed tools that have the highest call frequency within the preset review time window from the tool preference event library as preferred tools. The tool description of the user's preferred tool is injected into the large language model context, and the preferred tool description is locked to remain unchanged during the session. Repeat the following large language model invocation process until the response message returned by the large language model no longer contains tool invocation requests and tool search requests: Compress the current message history; The compressed message history and the current tool list are sent as the first request to the large language model; the current tool list includes the core toolset, the user preference tools, and the delayed tools that have been discovered and injected through the tool search request. Receive the response to the first request returned by the large language model; When the response to the first request contains a tool invocation request, the tool invocation request is executed within the sandbox instance. After the tool invocation request is executed, the result of executing the tool invocation request is added to the current message history. If the number of characters in the result of the tool invocation request exceeds a preset result threshold, the result of the tool invocation request is written to the file system of the sandbox instance, and the tool output in the message history is replaced with a placeholder containing a file reference identifier and part of the preview content. When the response includes a tool search request, the tool search request carries a search query term. Based on the search query term, the delayed toolset is matched and scored. Several candidate delayed tools with the highest scores are selected, and their complete tool descriptions are injected into the context of the large language model. At the same time, the current tool list is updated. When the response to the first request does not contain a tool invocation request or a tool search request, the loop exits, and the natural language content of the response to the first request is returned to the client as the final execution result.

2. The method according to claim 1, characterized in that, Also includes: A background memory retrieval task is initiated at the start of the session; the background memory retrieval task is used to retrieve relevant memory entries from the memory files within the sandbox instance based on the current conversation content. During the loop of the large language model invocation process, the retrieved relevant memory entries are injected into the message history.

3. The method according to claim 1, characterized in that, Also includes: If an interrupt command sent by the client is received during the streaming output of the large language model, the token usage is estimated based on the already output text and the billing details are determined. If an interruption command is received from the client between two large language model calls, the actual billing of the completed model call is retained, and a new billing detail is added to cover the word usage of the interrupted tail segment. If an interrupt command is received from the client during the silent waiting period, an interruption flag will be added to the last billing detail, and no new charges will be incurred.

4. The method according to claim 1, characterized in that, The step of determining the corresponding sandbox instance based on the tenant code and the user identifier includes: Based on the unique identifier, perform a retrieval operation from the set of created sandbox instances; If the result of the search operation is that a target sandbox instance corresponding to the unique identifier exists, then the target sandbox instance is obtained as the currently used sandbox instance; If the result of the search operation is that there is no target sandbox instance corresponding to the unique identifier, then the sandbox instance creation operation is performed. If the sandbox instance creation operation fails because the sandbox instance corresponding to the unique identifier has already been created by other concurrent requests, the current creation operation is terminated, and an acquisition operation is performed to obtain the sandbox instance already created by the other concurrent requests as the currently used sandbox instance.

5. The method according to claim 1, characterized in that, The user preference tool is determined in the following way: The frequency of each delay tool called by the user identifier within the preset time window is counted, sorted in descending order of frequency, and the intersection of this intersection with the set of delay tools available in the current session is taken as the user's preferred tools.

6. The method according to claim 1, characterized in that, The process involves matching and scoring the delay toolset based on search query terms, and selecting several candidate delay tools with the highest scores, including: For each tool in the delayed toolset, a name matching score, a function description semantic matching score, and a search suggestion matching score are calculated respectively, and then weighted and summed according to preset weights to obtain the cumulative score of the tool; wherein, the preset weight of the name matching is greater than the preset weight of the function description semantic matching, and the preset weight of the function description semantic matching is greater than the preset weight of the search suggestion matching. Select the candidate delay tools with the highest cumulative scores.

7. The method according to claim 1, characterized in that, The compression process for the current message history includes: Iterate through the tool call messages in the message history and filter out tool call messages whose creation time is earlier than the current time minus the idle interval threshold; The selected tool call messages are sorted in descending order of creation time. A preset number of tool call messages with the latest creation time are excluded, and the tool call identifiers corresponding to the remaining tool call messages are recorded in the compressed ledger. When constructing the message sequence to be sent to the large language model, the compressed ledger is queried, and the tool output of the message in the message sequence that identifies the tool call in the compressed ledger is replaced with a compressed placeholder.

8. The method according to claim 1, characterized in that, The compression process for the current message history includes: When the number of tokens occupied in the current context is detected to reach the preset full compression threshold, each tool call message in the message history and its corresponding tool output message are associated as a processing unit. The processing unit constructs a compressed prompt and calls a large language model to generate a summary; If the digest is successfully generated, the original message history will be replaced with the digest. If the summary generation fails, the consecutive failure count is accumulated. When the consecutive failure count reaches the preset circuit breaker limit, full compression is stopped and the original message history is kept unchanged.

9. The method according to claim 1, characterized in that, Also includes: When constructing the runtime context, a tenant-specific list of available skills is generated based on the skill declaration file within the sandbox instance, and the list of available skills is injected into the runtime context.

10. A sandbox-isolated intelligent agent execution system for multi-tenancy, characterized in that, include: The tenant identification module is used to receive requests sent by clients and extract tenant codes and user identifiers from the requests. The sandbox management module is used to construct a unique identifier based on the tenant code and the user identifier, determine the corresponding sandbox instance based on the unique identifier, and construct the runtime context of the agent. The runtime context includes at least the sandbox instance and tool configuration information. The tool configuration information includes a core toolset that is always injected into the large language model context, and a delayed toolset that is not injected into the context by default. The context building module is used to, based on the running context, in the first round of the session, retrieve the preset number of delayed tools with the highest call frequency within the preset review time window from the tool preference event library according to the user identifier as preferred tools, inject the tool description of the user preferred tool into the large language model context, and lock the preferred tool description to remain unchanged during the session; The large language model invocation module is used to repeatedly execute the following large language model invocation process until the response message returned by the large language model no longer contains tool invocation requests and tool search requests: The tool prefers to preload modules for compressing the current message history; The compressed message history and the current tool list are sent as the first request to the large language model; the current tool list includes the core toolset, the user preference tools, and the delayed tools that have been discovered and injected through the tool search request. Receive the response to the first request returned by the large language model; When the response to the first request contains a tool invocation request, the tool invocation request is executed within the sandbox instance. After the tool invocation request is executed, the result of executing the tool invocation request is added to the current message history. If the number of characters in the result of the tool invocation request exceeds a preset result threshold, the result of the tool invocation request is written to the file system of the sandbox instance, and the tool output in the message history is replaced with a placeholder containing a file reference identifier and part of the preview content. When the response includes a tool search request, the tool search request carries a search query term. Based on the search query term, the delayed toolset is matched and scored. Several candidate delayed tools with the highest scores are selected, and their complete tool descriptions are injected into the context of the large language model. At the same time, the current tool list is updated. When the response to the first request does not contain a tool invocation request or a tool search request, the loop exits, and the natural language content of the response to the first request is returned to the client as the final execution result.

Images