A large language model tool capability expansion method based on a runner client long connection

Abstract

The application relates to the technical field of artificial intelligence, in particular to a large language model tool capacity expansion method and system based on a Runner Client long connection. After a bidirectional communication long connection between a Backend and an RC process is established, the Backend acquires a calling request sent by the RC process, tool positioning and RC selection are carried out based on the calling request, then code fingerprints corresponding to target tool versions are screened from a tool warehouse based on the tool positioning, a calling delivery package is constructed through the Backend and is sent to the RC process, then a return result of the RC process is received, execution results returned by one or more RC processes are aggregated, the execution results are injected into a LLM reasoning context as a tool calling response, and the LLM continues subsequent reasoning. Through a single LLM session, multiple distributed RC nodes can be simultaneously scheduled to cooperatively complete a task, the LLM tool capacity range can be expanded, network and data security can be guaranteed, and an LLM tool calling architecture supporting multi-tenant isolation, multi-device cooperation and online dynamic capacity expansion is constructed.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method for extending the capabilities of a large language model tool based on a long connection of Runner Client. Background Technology

[0002] As LLM capabilities continue to improve, LLM is no longer limited to plain text generation. It can now access file systems, execute code, invoke network services, and manipulate databases by calling external tools (ToolCalling / FunctionCalling) to accomplish complex tasks. Currently, common LLM tool deployment solutions in the industry mainly fall into four categories: local integrated agent solutions, pure cloud tool solutions, local MCP tool static configuration solutions, local pre-installed executable package solutions, and remote MCP (ModelContextProtocol) service solutions.

[0003] However, in existing technical solutions, the single-machine solution is limited to a single machine, while the cloud solution cannot access private resources. This requires the tool provider to expose ports or for data to leave the domain. Furthermore, a single LLM session is difficult to schedule multiple distributed tool nodes to complete cross-device tasks simultaneously. There is a lack of a mechanism to provide independent tool capability domains for different users and projects on the same LLM platform. At the same time, there is a lack of high availability guarantee mechanisms such as long connection, reconnection after disconnection, and failover. Summary of the Invention

[0004] The purpose of this invention is to provide a method for extending the capabilities of large language model tools based on Runner Client long connections, so as to solve the problems mentioned in the background art.

[0005] To address the aforementioned technical problems, one objective of this invention is to provide a method for extending the capabilities of large language model tools based on Runner Client long connections, comprising: S1: Establish a two-way long-lived communication connection; S2: Get the call request; S3: Tool location and RC selection based on call requests; S4: Based on tool location, filter the code fingerprint corresponding to the target tool version from the tool repository; S5: Construct and send the call delivery packet; S6: Retrieve the returned result; S7: Generate tool call response based on the aggregation of multiple return results.

[0006] Preferably, step S3 includes the following steps: S3.1: Retrieve the text content of the call request; S3.2: Definition of tools authorized to be accessed based on text content retrieval; S3.3: Match the target tool version based on the tool name.

[0007] Preferably, the executable code defined by the tool includes scripting language source code, bytecode, intermediate representation (IR), tool combinatorial call graph, and configurable process nodes, and the executable code is stored in the tool repository in plaintext or encrypted form and addressed by code hash.

[0008] Preferably, the delivery packet includes a request identifier, a tool identifier, a code hash of the target tool version, a key reference, calling parameters, and context information.

[0009] Preferably, during the regular tool invocation process, only a lightweight invocation package is issued, which includes a tool identifier, code hash, decryption key, and invocation parameters; The key required for decryption is provided in the call packet each time or issued through session-level negotiation, and can be bound to the validity period, number of calls, and session scope; When the code in the tool repository changes, the code hash changes synchronously. Stop issuing decryption keys to the tool when the tool is withdrawn or permissions are changed.

[0010] On the other hand, this application also provides a large language model tool capability extension device based on Runner Client long connection, applied to the method described in any one of the foregoing, comprising: RC process; deployed on the user terminal; Backend; the RC process communicates with the Backend, which maintains the RC connection registry and tool repository. The tool repository includes all tool definitions and their corresponding executable code. (Whether or not to add a Backend usually consists of one or more background services.) The Backend includes a scheduling module, which performs tool location and RC selection.

[0011] Preferably, the RC process holds the authentication credentials of the user and the device fingerprint of the device. The RC process initiates an outbound network connection to the Backend to establish a bidirectional long-lived communication connection. The long-lived connection carries message frames on top of the transport layer based on a streaming protocol.

[0012] Preferably, the RC process registers its context metadata with the Backend during the long connection handshake phase. The metadata includes the user identifier, the project or permission domain identifier, the running platform information, the local resource domains that are allowed to be accessed, and the current load status.

[0013] Preferably, after receiving the packet, the RC process needs to verify the integrity of the digital signature of the packet and search for the code hash in the "encrypted code cache or binary executable" in the RC process's local cache. If a match is found, the encrypted code or binary executable that already exists in the cache is retrieved directly; if no match is found, a code retrieval request is sent to the Backend through the same long connection.

[0014] Preferably, after the RC process passes the verification, it executes the code and passes the tool call parameters as input; after execution, the RC process destroys the sandbox and clears the code copy in memory, and the execution result is flowed back to the Backend through the same long connection according to the request identifier.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Compared to local integrated agent solutions (such as ClaudeCode, Cursor, and OpenClaw), the schedulable tool capabilities of LLM are no longer limited to a single local machine, but are the sum of the capabilities of N RC devices, thereby improving the scheduling smoothness of LLM; 2. Long-lived connections, combined with heartbeat, disconnection reconnection, and session recovery mechanisms, ensure the stability of long-term Agent tasks. When an RC instance fails, it can automatically switch to a backup RC instance in the same permission domain to complete the task. Furthermore, a single LLM session can issue multiple tool calls to different RC instances in parallel during a single inference, enabling cross-device parallel execution and result aggregation, which greatly improves the execution efficiency of complex tasks. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for extending the capabilities of a large language model tool based on a Runner Client long connection. Figure 2 This is a schematic diagram of a framework for a large language model tool capability extension system based on Runner Client long-connection; Figure label: 100, RC process; 200, Backend; 201, scheduling module. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] The technical concept of this invention is as follows: For existing technical solutions, such as local integrated agent solutions, the tool capabilities are limited by the hardware, software, and network environment of a single local machine, and cross-device collaboration is not possible. In contrast, for pure cloud-based tool solutions, the tool provider must expose public network endpoints, and user private resources (local files, intranet databases, and enterprise internal systems) cannot be included in the scope of LLM tool capabilities.

[0019] Therefore, this application aims to solve the following technical problems: 1. How to enable LLM to call tools deployed on user terminals without requiring user terminals to expose public network ports; 2. How to enable a single LLM session to simultaneously schedule multiple distributed RC nodes to collaboratively complete a task; 3. How to achieve fine-grained permission isolation for tool resources in multi-user, multi-project scenarios; 4. How to ensure security and controllability in deployments across network regions (such as public network LLM services calling internal network RC); 5. How to ensure the reliability and recoverability of tool invocation channels during long-running agent tasks; 6. How to enable the tool to support online production, dynamic deployment by permission domain, and hot reloading, and to expand capabilities without RC restart; 7. How to carry tool capabilities in the form of "real-time code delivery and execution", and protect the tool's code assets through mechanisms such as encrypted transmission, encrypted caching of hash indexes, key control by call, runtime decryption, and signature verification, while achieving zero-deployment version switching, transient plaintext execution, and immediate retraction of tool code.

[0020] Example 1 like Figure 1 As shown, one of the objectives of this invention is to provide a method for extending the capabilities of large language model tools based on Runner Client long connections, including: S1: Establish a two-way long-lived communication connection; S2: Get the call request; S3: Tool location and RC selection based on call requests; S4: Based on tool location, filter the code fingerprint corresponding to the target tool version from the tool repository; S5: Construct and send the call delivery packet; S6: Retrieve the returned result; S7: Generate tool call response based on the aggregation of multiple return results.

[0021] It should be noted that after establishing a bidirectional long-lived communication connection between the Backend and the RC process, the Backend obtains the call request issued by the RC process, performs tool location and RC selection based on the call request, then filters the code fingerprint corresponding to the target tool version from the tool repository based on the tool location, constructs a call delivery packet through the Backend and sends it to the RC process, then receives the return result from the RC process, aggregates the execution results returned by one or more RC processes, and injects them back into the LLM inference context as a tool call response, so that the LLM can continue subsequent inference. This application can simultaneously schedule multiple distributed RC nodes to complete tasks collaboratively through a single LLM session, which can not only expand the capabilities of LLM tools, but also ensure network and data security, and build an LLM tool call architecture that supports multi-tenant isolation, multi-device collaboration and online dynamic capacity expansion.

[0022] In one embodiment of this application, step S3 includes the following steps: S3.1: Retrieve the text content of the call request; S3.2: Definition of tools authorized to be accessed based on text content retrieval; S3.3: Match the target tool version based on the tool name.

[0023] It should be noted that after the RC process completes the bidirectional long-lived communication connection with Backend, when Backend receives a tool invocation request generated during the LLM inference process of the RC process, it will first retrieve the tool definition that the session is authorized to invoke from the Backend tool repository according to the current session context (user_id, project_id, etc.), and match the target tool version according to the tool name; then, it will filter out a set of candidate RC instances that are compatible with the target tool's permission domain and platform / resource constraints from the RC connection registry, and select one or more target RC instances according to preset strategies (affinity, load, proximity, user specification, etc.).

[0024] The scheduling process in step S3 does not depend on the capability declaration of the RC process. The callability of capabilities is determined solely by the Backend tool repository. The RC process is merely a scheduled execution unit.

[0025] In one embodiment of this application, the tool definition executable code includes script language source code, bytecode, intermediate representation (IR), tool combinatorial call graph, and configurable process nodes. The executable code is stored in a tool repository in plaintext or encrypted form and is addressed by code hash. It should be noted that the RC process is configured with a local "encrypted code cache". The cache stores the distributed tool code with code hash as index and encrypted ciphertext as content. During the regular tool call process, the Backend only sends a lightweight call package to the RC process, including the tool identifier, code hash, decryption key and call parameters. The RC process first looks up the local encrypted cache by hash and uses it. If it does not find the target code, it then initiates the on-demand retrieval of the encrypted code body. The cache is stored only in encrypted form, and the plaintext code exists only in the transient memory during the execution of this call and is destroyed when the call ends; the key required for decryption is provided by the Backend in the call packet each time, or is issued through session-level negotiation, and can be bound to the validity period, number of calls and session scope; When the code in the Backend tool repository changes, the code hash changes synchronously. The next time Backend is called, the new hash is issued, triggering the RC cache to miss and fetch again. No active notification or deployment action is required, and the tool version switches in seconds and atomically. When a tool is withdrawn or its permissions are changed, the Backend stops issuing decryption keys to the tool. Even if the RC holds the tool's encrypted cache, it cannot be restored to executable code. The withdrawal effect takes effect immediately. This achieves an integrated mechanism of "single point holding of code assets in the backend, key control by call, cacheable and reusable ciphertext, no plaintext left on the ground, zero deployment for version switching, and immediate effect of withdrawal".

[0026] In one embodiment of this application, the delivery packet includes a request identifier, a tool identifier, a code hash of the target tool version, a key reference, calling parameters, and context information.

[0027] It should be noted that the delivery packet is sent back to the target RC instance through the established long connection. The delivery packet does not carry the tool code body itself by default, so that the regular call message remains lightweight.

[0028] After receiving the distribution packet, the target RC instance executes the following steps in sequence: Verify the integrity of the digital signature of the sent packets; Search for the code hash in the local "encryption code cache" on the RC: If a match is found, the encrypted code body that already exists in the cache is retrieved directly; If the match is unsuccessful, a code fetch request is sent to the Backend via the same long connection; the code fetch request carries the code hash. Backend retrieves the corresponding encrypted code body from the tool repository based on the hash and pushes it to the RC process via a long connection. After the RC process verifies that the hash of the pushed code body matches the request, it writes the encrypted code body into the local encrypted code cache and stores it in ciphertext format, indexed by hash. The encrypted code is decrypted into plaintext using the decryption key carried in the packet sent during this call (or the key obtained from the trusted channel via key reference); the plaintext exists only in the protected memory of the RC process and is not written to any persistent storage. Verify whether the local resource access path involved in this call is within the allowed directory whitelist configured in the RC, and whether the operation type conforms to the read and write permissions allowed by the whitelist entry.

[0029] In one embodiment of this application, during a regular tool invocation process, only a lightweight invocation package is issued, which includes a tool identifier, a code hash, a decryption key, and invocation parameters; The key required for decryption is provided in the call packet each time or issued through session-level negotiation, and can be bound to the validity period, number of calls, and session scope; When the code in the tool repository changes, the code hash changes synchronously. Stop issuing decryption keys to the tool when the tool is withdrawn or permissions are changed.

[0030] It should be noted that the RC process is configured with a local "encrypted code cache". The cache stores the distributed tool code using code hash as index and encrypted ciphertext as content. The RC first searches for and uses the code in the local encrypted cache by hash. If the code is not found, it then initiates an on-demand retrieval of the encrypted code body. The cache is only stored in ciphertext form. The plaintext code only exists in the transient memory during the execution of this call and is destroyed when the call ends. Because the Backend stops issuing decryption keys for the tool when it is withdrawn or permissions are changed, even if the RC holds the encrypted cache of the tool, it cannot be restored to executable code, and the withdrawal effect takes effect immediately. This achieves an integrated mechanism of "single point holding of code assets by the Backend, key control by call, ciphertext can be cached and reused, plaintext is not stored on the ground, zero deployment for version switching, and withdrawal takes effect immediately".

[0031] like Figure 2As shown, in one embodiment of this application, a large language model tool capability extension device based on Runner Client long connection is also provided, applied to the method described in any one of the foregoing, comprising: RC process; deployed on the user terminal; Backend; The RC process communicates with the Backend, and maintains the RC connection registry and tool repository through the Backend. The tool repository includes all tool definitions and corresponding executable code. The Backend includes a scheduling module, which performs tool location and RC selection. Optionally, Backend further provides an online tool orchestration platform. This platform supports the online creation, editing, versioning, and management of tool definitions on the Backend side. The tool definition includes at least the tool name, parameter schema, executable code, permission domain, visible user / project scope, and version number. Once the tool definition is saved, it is written to the Backend tool repository. There is no need to pre-deploy or pre-load it to any RC. The next LLM call that meets the visibility scope will hit the target and complete the deployment and execution as described above. Similarly, the update, withdrawal, and permission scope change of the tool only require modification of the Backend tool repository. There is no need to notify any RC because the RC does not hold any code or status of the tool. After withdrawal, Backend will no longer deploy it, which means it "disappears immediately" from the user's perspective.

[0032] It should be noted that the RC connection registry records the connection handles and context metadata of all currently online RC instances, and establishes an index according to dimensions such as user identifier, project identifier, permission domain, platform, and load, so that the Backend can maintain long connections of multiple RC instances at the same time; The tool repository centrally holds all tool definitions and corresponding executable code. The tool's capability index, version management, permission domain ownership, and visibility scope are all maintained by the Backend tool repository, without relying on or depending on any RC's capability reporting.

[0033] In one embodiment of this application, the RC process holds the authentication credentials of the user and the device fingerprint of the device. The RC process initiates an outbound network connection to the Backend to establish a bidirectional long-lived communication connection. The long-lived connection carries message frames on top of the transport layer based on a streaming protocol.

[0034] It should be noted that the RC process does not pre-install, hold, or report any tool capability list, nor does it maintain any tool code or tool instances locally; it exists only as a controlled stateless code execution container.

[0035] In one embodiment of this application, the RC process registers its context metadata with the Backend during the long connection handshake phase. The metadata includes the user identifier, the project or permission domain identifier, the running platform information, the local resource domains that are allowed to be accessed, and the current load status.

[0036] It should be noted that the long connection establishment and registration process includes the following steps: RC process starts → Loads APIKey and device fingerprint → Actively initiates a TLS-encrypted outbound connection to Backend → Sends Hello frame (carrying authentication credentials) → Backend verifies credentials and queries device binding relationship → Returns Hello-Ack frame → RC process reports context metadata (user / project / permission domain, platform, allowed directory whitelist, network access restriction information, current load) → Backend writes to the connection registry and creates an index → ​​Enters heartbeat keep-alive loop.

[0037] In one embodiment of this application, after receiving the packet, the RC process needs to verify the integrity of the digital signature of the packet and search for the code hash in the "encrypted code cache or binary executable" in the RC process's local cache. If a match is found, the encrypted code or binary executable that already exists in the cache is directly retrieved; if no match is found, a code retrieval request is sent to the Backend through the same long connection.

[0038] It should be noted that the process is as follows: A user submits a request in the LLM session → LLM generates a tool call → The Backend scheduling module locates the target tool version from the tool repository based on the session context → Retrieves the code hash and decryption key for that version → Filters candidate RC processes with compatible permission domains / platforms from the connection registry → Selects the target RC process according to a strategy → Sends a call packet (containing tool ID + code hash + decryption key + parameters) via a long connection → The RC process searches in the encrypted cache by hash: if a match is found, the encrypted text is retrieved; otherwise, a CodeFetch is initiated to fetch and write to the encrypted cache → The RC decrypts the text into memory + executes it in a sandbox + destroys the plaintext → Response frames are fed back via the same long connection → The Backend aggregates the results and injects them into the LLM context.

[0039] In one embodiment of this application, after the RC process passes the verification, it executes the code and passes the tool call parameters as input; After execution, the RC process destroys the sandbox and clears the code copy in memory, and then uses the RC process to send the execution result back to the Backend via the same long connection according to the request identifier.

[0040] It should be noted that the ciphertext form in the local encrypted code cache can be retained for subsequent tool calls that hit the same code hash to directly reuse, thereby avoiding repeated transmission; however, the cached ciphertext cannot be restored to plaintext without the decryption key subsequently issued by the Backend, nor can it be executed independently; the execution result is returned to the Backend through the same long connection according to the request identifier.

[0041] The RC process only initiates outbound connections to the Backend, without exposing any listening ports on the user terminal. It naturally penetrates NAT and firewalls, with an attack surface close to zero. Compared to solutions such as remote MCP that require port exposure, it has an inherent security advantage.

[0042] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for extending the capabilities of large language model tools based on Runner Client long connections, characterized in that: include: S1: Establish a two-way long-lived communication connection; S2: Get the call request; S3: Tool location and RC selection based on call requests; S4: Based on tool location, filter the code fingerprint corresponding to the target tool version from the tool repository; S5: Construct and send the call delivery packet; S6: Retrieve the returned result; S7: Generate tool call response based on the aggregation of multiple return results.

2. The method for extending the capabilities of a large language model tool based on a Runner Client long connection as described in claim 1, characterized in that: Step S3 includes the following steps: S3.1: Retrieve the text content of the call request; S3.2: Definition of tools authorized to be accessed based on text content retrieval; S3.3: Match the target tool version based on the tool name.

3. The method for extending the capabilities of a large language model tool based on a long-lived Runner Client connection as described in claim 2, characterized in that: The executable code defined by the tool includes scripting language source code, bytecode, intermediate representation (IR), tool combinatorial call graph, and configurable process nodes. The executable code is stored in the tool repository in plaintext or encrypted form and is addressed by code hash.

4. The method for extending the capabilities of a large language model tool based on a Runner Client long connection as described in claim 3, characterized in that: The delivery package includes a request identifier, a tool identifier, a code hash of the target tool version, a key reference, calling parameters, and context information.

5. The method for extending the capabilities of a large language model tool based on a Runner Client long connection as described in claim 4, characterized in that: During the regular tool invocation process, only a lightweight invocation package is issued, which includes the tool identifier, code hash, decryption key, and invocation parameters; The key required for decryption is provided in the call packet each time or issued through session-level negotiation, and can be bound to the validity period, number of calls, and session scope; When the code in the tool repository changes, the code hash changes synchronously. Stop issuing decryption keys to the tool when the tool is withdrawn or permissions are changed.

6. A system for extending the capabilities of large language model tools based on Runner Client long connections, applied to the method of any one of claims 1 to 5, characterized in that, include: RC process; Deployed on user terminals; Backend; The RC process communicates with the Backend, and maintains the RC connection registry and tool repository through the Backend. The tool repository includes all tool definitions and corresponding executable code. The Backend includes a scheduling module, which performs tool location and RC selection.

7. The method for extending the capabilities of a large language model tool based on a long-lived Runner Client connection as described in claim 6, characterized in that, The RC process holds the authentication credentials of the user and the device fingerprint of the device. It initiates an outbound network connection to the Backend through the RC process to establish a bidirectional long-lived communication connection. The long-lived connection carries message frames on top of the transport layer based on a streaming protocol.

8. The method for extending the capabilities of a large language model tool based on a long-lived Runner Client connection as described in claim 7, characterized in that, During the long connection handshake phase, the RC process registers its context metadata with the Backend. The metadata includes the user identifier, the project or permission domain identifier, the running platform information, the local resource domains that are allowed to be accessed, and the current load status.

9. The method for extending the capabilities of a large language model tool based on a long-lived Runner Client connection as described in claim 8, characterized in that, After receiving the packet, the RC process needs to verify the integrity of the digital signature of the packet and search for the code hash in the "encrypted code cache or binary executable" in the RC process's local cache. If a match is found, the encrypted code or binary executable that already exists in the cache is directly retrieved; if a match is not found, a code retrieval request is sent to the Backend through the same long connection.

10. The system for extending the capabilities of a large language model tool based on a Runner Client long connection as described in claim 9, characterized in that, After the verification is passed, the RC process executes the code, passing the tool call parameters as input; after execution, the RC process destroys the sandbox and clears the code copy in memory, and the execution result is flowed back to the Backend through the same long connection according to the request identifier.

Images