Method for calling client tool and gateway

Abstract

A method for calling a client tool and a gateway, the method is executed by the gateway, and the method comprises the following steps: receiving a query statement of a to-be-input server large model and a connection channel identifier of a first connection from a client through the first connection; sending the query statement and the connection channel identifier to the server through a second connection, wherein the second connection transmits data based on a stateless protocol; receiving a tool calling request and the connection channel identifier from a server through the second connection, wherein the tool calling request comprises a request identifier and a tool name of a tool provided by the client; acquiring the first connection based on the connection channel identifier; the tool calling request is sent to the client through the first connection; receiving a tool calling result from the client through the first connection, wherein the tool calling result comprises the request identifier; and sending the tool calling result to the server through the second connection.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of network connection technology, and in particular relate to a method for invoking client tools and a gateway. Background Technology

[0002] In scenarios where users interact with an AI agent on their client devices, such as... Figure 1 As shown, for data security reasons, the Agent's server is located on the Agent service provider's intranet (i.e., the company's private internal network), while the client is located on the external network (public internet). The client cannot directly access the server. The boundary between the intranet and the external network is called the DMZ (Dedicated Local Area Network), which contains a gateway. This gateway includes public network connection addresses (IP addresses). Clients can connect to the gateway based on their IP addresses. Simultaneously, the gateway and the Agent server can connect to each other via the intranet.

[0003] With the development of Large Model (LLM) technology, agents may need to call external tools to complete complex tasks. For example, an agent that is an intelligent R&D assistant (CodeAgent) can read / write the client's code files by calling the client's local file system tools based on the Model Context Protocol (MCP) to modify the code files.

[0004] MCP provides a unified, machine-readable description of all external services (such as tools and databases), essentially providing a specification for the interaction between agents in LLM applications and external services. The MCP architecture includes the MCP host, MCP client, and MCP server. The MCP host, typically referring to the AI ​​application, is the initiator of the interaction. The MCP client, located within the MCP host, is used to discover services provided by the MCP server and to transmit information between the LLM and the MCP server. The MCP protocol includes authentication and access authorization capabilities based on Open Authorization (OAuth). OAuth is an open security protocol that provides an authorization standard for applications to access user resources. Its characteristic is that it grants resource access permissions through a token mechanism, avoiding the use of sensitive information such as account passwords.

[0005] In order to enable the agent to invoke local tools on the client, one related technology, such as Figure 2As shown, an HTTP connection is established between the client and the gateway, and an HTTP connection is established between the gateway and the Agent server. The HTTP connection between the client and the server can be based on Hypertext Transfer Protocol 1.0 (HTTP 1.0) or HTTP 1.1, for example. In HTTP communication, the client typically sends a request to the server, and the server responds based on the client's request. The client's request can include GET and POST methods; the GET method is used to request a specified resource, and the POST method is used to submit data to the server. Under this network connection, when the client and Agent are in a session, the client sends a query to the gateway, and the gateway sends the query to the Agent server. If the Agent determines that a tool call is needed, the Agent server sends the tool call information back to the client via the gateway. The client can then call the corresponding tool based on the tool call information, sending a new request to the Agent server via the gateway. This new request includes the previous query and the tool call result. Thus, the Agent can output the final result based on the query and the tool call result, and return the final result to the client.

[0006] Under the aforementioned HTTP-based network connection, each time the Agent needs to obtain more information from the client, it relies on the client actively including the full context (such as the query from the previous request and the tool call result) in the query, leading to data transmission redundancy. Due to the client-initiated nature of the HTTP protocol, the Agent cannot proactively initiate tool calls. Furthermore, under this network connection, to promptly obtain the Agent's tool call requests, the client needs to periodically poll the server for tool call instructions; however, this polling mechanism typically has a delay and cannot meet the requirements of real-time interaction.

[0007] In another related technology, such as Figure 3 As shown, a WebSocket connection is established between the client and the gateway, and a WebSocket connection is established between the gateway and the server. WebSocket supports bidirectional data streaming between the server and the client, meaning the server can proactively send messages to the client. However, while this network connection method solves the problem of the server not being able to proactively initiate tool calls in the HTTP connection method described above, some other issues still exist.

[0008] First, because the Agent server establishes a WebSocket connection with the gateway, the WebSocket connection explicitly manages its state. This means that both the Agent server and the gateway maintain protocol-layer and application-layer states. Protocol-layer states can include connection status (connected, open, closing, closed), frame parsing status, heartbeat keep-alive, security context, flow control, etc., while application-layer states can include user identifiers, business context, session data, statistical status, etc. This requires the Agent server to maintain a significant amount of state data, consuming considerable resources.

[0009] Secondly, WebSocket connections are established through an upgrade handshake based on the HTTP protocol. For security reasons, the WebSocket API strictly prohibits setting custom HTTP headers during the handshake phase on the client side, making it difficult to integrate the OAuth authentication method from the MCP protocol. Furthermore, during the handshake phase, current HTTP only supports upgrades using the HTTP GET method and not the POST method, thus hindering authentication operations such as uploading the client's user authorization token as required by the MCP protocol. Summary of the Invention

[0010] The purpose of this invention is to provide an effective method for reverse-calling client tools from a large model server.

[0011] This specification provides a method for invoking a client tool, executed by a gateway, wherein the gateway and the client establish a first connection, the first connection including a bidirectional long connection, the method comprising: receiving from the client a query statement to be input into a server-side large model and a connection channel identifier of the first connection via the first connection; sending the query statement and the connection channel identifier to the server via a second connection, the second connection transmitting data based on a stateless protocol; receiving from the server a tool invocation request and the connection channel identifier via the second connection, the tool invocation request including a request identifier and a tool name provided by the client; obtaining the first connection based on the connection channel identifier; sending the tool invocation request to the client via the first connection; receiving a tool invocation result from the client via the first connection, the tool invocation result including the request identifier; and sending the tool invocation result to the server via the second connection.

[0012] In one implementation, the first connection includes a WebSocket connection that transmits data based on the WebSocket protocol, and the second connection includes an HTTP connection that transmits data based on the HTTP protocol.

[0013] In one implementation, the gateway includes an external gateway and an internal gateway. The client establishes a WebSocket connection with the external gateway. Receiving a tool invocation request and the connection channel identifier from the server through the second connection includes: the internal gateway receiving the tool invocation request and the connection channel identifier from the server through the HTTP connection. The method further includes: the internal gateway sending the tool invocation request and the connection channel identifier to the external gateway via a Remote Procedure Call (RPC). Obtaining the first connection based on the connection channel identifier and sending the tool invocation request to the client through the first connection includes: the external gateway obtaining the WebSocket connection based on the connection channel identifier and sending the tool invocation request to the client through the WebSocket connection.

[0014] In one implementation, the gateway includes an external gateway and multiple internal gateways. The client establishes a WebSocket connection with the external gateway. Receiving a tool invocation request and the connection channel identifier from the server via the second connection includes receiving the tool invocation request and the connection channel identifier from the server via the second connection by a first internal gateway among the multiple internal gateways. The method further includes: the first internal gateway sending the tool invocation request, the connection channel identifier, and connection information of the first internal gateway to the external gateway; receiving a tool invocation result from the client via the first connection includes the external gateway receiving the tool invocation result from the client via the WebSocket connection and sending the tool invocation result to the first internal gateway based on the connection information.

[0015] In one implementation, the gateway includes an external gateway and multiple internal gateways. The client establishes a WebSocket connection with the external gateway. Receiving a tool invocation request and the connection channel identifier from the server through the second connection includes receiving the tool invocation request and the connection channel identifier from the server by a first internal gateway among the multiple internal gateways. The request identifier in the tool invocation request is generated based on the connection information of the first internal gateway. The method further includes: the first internal gateway sending the tool invocation request and the connection channel identifier to the external gateway. Receiving a tool invocation result from the client through the first connection includes: the external gateway receiving the tool invocation result from the client through the WebSocket connection, obtaining the connection information from the request identifier in the tool invocation result, and sending the tool invocation result to the first internal gateway based on the connection information.

[0016] In one implementation, the gateway includes multiple external gateways and multiple internal gateways. The client establishes a WebSocket connection with a first external gateway among the multiple external gateways. Receiving a tool invocation request and the connection channel identifier from the server through the second connection includes the first internal gateway among the multiple internal gateways receiving the tool invocation request and the connection channel identifier from the server. The request identifier in the tool invocation request is generated based on the connection information of the first internal gateway. The method further includes: the first internal gateway determining the first external gateway based on the connection channel identifier and sending the tool invocation request and the connection channel identifier to the first external gateway. Receiving a tool invocation result from the client through the first connection includes the first external gateway receiving the tool invocation result from the client through the WebSocket connection, obtaining the connection information from the request identifier in the tool invocation result, and sending the tool invocation result to the first internal gateway based on the connection information.

[0017] In one embodiment, the method further includes: receiving authentication information from the client via an HTTP connection; if authentication is successful based on the authentication information, sending a connection token corresponding to the client; establishing the WebSocket connection with the client based on the connection token; and after establishing the WebSocket connection, sending the connection channel identifier to the client via the WebSocket connection.

[0018] In one implementation, sending the tool invocation request to the client via the first connection based on the connection channel identifier includes: finding the WebSocket connection handle corresponding to the connection channel identifier based on the connection channel identifier, and sending the tool invocation request to the client based on the WebSocket connection handle.

[0019] In one embodiment, the method further includes: after receiving a tool invocation request from the server, storing the request identifier in association with the context of the tool invocation request; the step of sending the tool invocation result to the server via the second connection includes: obtaining the context based on the request identifier in the tool invocation result; and sending the tool invocation result to the server via the second connection according to the context.

[0020] In one implementation, the tool includes a file reading tool, and the result of the tool invocation includes a file read from the device where the client is located by executing the tool.

[0021] A second aspect of this specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0022] A third aspect of this specification provides a gateway, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method described in the first aspect.

[0023] A fourth aspect of this specification provides a computer program product including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0024] The method for invoking client tools provided in the embodiments of this specification establishes a bidirectional long connection between the client and the gateway, and a stateless protocol connection between the gateway and the Agent server. By associating the bidirectional long connection and the stateless protocol connection based on the ChannelID, the Agent service does not need to maintain the connection state and can proactively initiate multiple rounds of tool calls, saving Agent service resources. In addition, the client only needs to access the gateway based on the bidirectional long connection standard and does not need to worry about the complexity of the MCP protocol. The Agent server only needs to focus on the MCP protocol and does not need to be aware of the details of the long connection, thus realizing an MCP data channel from the intranet to the extranet, thereby achieving low-cost connection. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the system network structure in one embodiment; Figure 2 This is a schematic diagram of network connections between devices in a system according to one embodiment; Figure 3 This is a schematic diagram of network connections between devices in a system according to another embodiment; Figure 4 This is a schematic diagram of the network connection between devices in a system according to one embodiment of this specification; Figure 5 This is a schematic diagram illustrating the process of establishing a WebSocket connection between the client and the gateway in the embodiments of this specification; Figure 6This is a flowchart illustrating the method by which the Agent server calls the client tool during the inference process in the embodiments of this specification. Figure 7 This is a schematic diagram of the client structure in the embodiments of this specification; Figure 8 This is a schematic diagram of the network connection between devices in another embodiment of the system described in this specification; Figure 9 This is a flowchart illustrating the method by which the Agent server calls the client tool during the inference process in the embodiments of this specification. Figure 10 This is a system architecture diagram of yet another embodiment of this specification. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0028] To facilitate understanding of the embodiments in this specification, the relevant terms used in the embodiments will be introduced first.

[0029] I. Network Connection Related Terminology

[0030] TCP / IP protocol: An open communication protocol suite standardized by the Internet Engineering Task Force (IETF) through a series of RFC documents. It comprises four layers from bottom to top: physical layer, network layer, transport layer, and application layer. The physical layer enables the interface between hosts and physical networks; the network layer provides logical addressing and routing capabilities across networks based on IP addresses, enabling end-to-end forwarding of data packets; the transport layer establishes inter-process communication channels based on IP addresses and process port numbers, ensuring data transmission reliability and efficiency; and the application layer defines business interaction semantics and data formats to support user applications.

[0031] Transmission Control Protocol (TCP): TCP is a transport layer protocol. Based on TCP, a TCP connection is established through a three-way handshake and terminated through a four-way handshake. When transmitting messages over a TCP connection, reliable transmission is achieved through the message sender marking the sequence numbers of the messages it sends, and the message receiver sending ACK acknowledgments.

[0032] Hypertext Transfer Protocol (HTTP): A stateless application layer protocol based on a request-response model. HTTP operates on a client-server architecture, where the client sends a request to the HTTP server, and the server sends a response to the client based on the received request. Currently, multiple versions of HTTP coexist, including HTTP 1.0, HTTP 1.1, and HTTP 2.0. In HTTP 1.0, a new TCP connection is established for each request, and it supports GET and POST methods. In HTTP 1.1, multiple HTTP requests can be made after a single TCP connection, thus reducing handshake overhead; HTTP 1.1 connections are also known as persistent connections.

[0033] WebSocket Protocol: The WebSocket protocol is a full-duplex, persistent application-layer communication protocol. Through a single HTTP handshake upgrade mechanism, it establishes a bidirectional communication channel between the client and server, independent of HTTP, specifically designed for real-time interactive scenarios (such as instant messaging, collaborative editing, and IoT command issuance). The WebSocket protocol requires the maintenance of connection state, which includes: Protocol layer state: connection phase (CONNECTING, OPEN, CLOSING, CLOSED), heartbeat state, frame parsing state (whether the current frame is complete), etc.; Application layer state: user session information (such as user ID), subscribed topics, business context, etc. (maintained by the application code).

[0034] Server-Sent Events (SSE) connections are unidirectional persistent event push channels built on the standard HTTP protocol. Essentially, the client establishes a persistent connection (HTTP / 1.1 Keep-Alive or HTTP / 2) via an HTTP GET request, and the server continuously pushes a formatted event stream to the client. The client parses the events using the EventSource API, and automatically reconnects according to a policy if the connection is broken. This mechanism fully complies with HTTP semantics and requires no protocol upgrade.

[0035] Remote Procedure Call (RPC) is a distributed computing paradigm that allows client programs to invoke service methods on remote servers as if they were local functions. In RPC, connections can be established between the client and server using various transport layer protocols, such as HTTP, making it suitable for scenarios involving high-frequency internal service communication.

[0036] II. Introduction to the Large Model

[0037] Large language models, also known simply as large models, are natural language processing models based on deep learning techniques. Their parameter count typically ranges from billions to hundreds of billions or even higher, possessing powerful language understanding and generation capabilities. Large language models can employ the Transformer architecture or its variants (such as GPT and BERT), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large language models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, enabling them to generalize. The core capabilities of large language models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Their usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, users design prompts to guide the large language model in generating specific outputs. Prompts can be textual descriptions of the task or instructions used to stimulate the large language model's semantic understanding and generation capabilities. In fine-tuning mode, large language models are further trained on small-scale datasets within a specific domain to optimize their performance on specific tasks. The powerful generalization capabilities and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0038] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0039] An agent can have a Large Language Model (LLM) as its brain. The LLM is mainly used to understand the user's instructions and intentions, break down complex tasks into a step-by-step executable plan, determine which tools or knowledge to use in what situations, and integrate the results of tool execution and knowledge base information to generate the final answer.

[0040] III. Open Licensing

[0041] Open Authorization (OAuth) is an open security protocol that provides an authorization standard for third-party applications to access user resources. Its key feature is the granting of resource access permissions through a token mechanism, preventing third parties from obtaining sensitive information such as user account passwords. The main components of the OAuth framework include: resource owner, resource server, client, and authorization server. The resource owner's resources (such as external services) are stored on the resource server. The client is an application, website, application programming interface (API), or device requesting access to the resources. The client can request authorization from the authorization server. After confirming the resource owner's consent, the authorization server sends an access token to the client, allowing the client to access the resource owner's resources on the resource server based on this token. The access token may specify, for example, the scope of resources the client can access and the duration of access, and includes the authorization server's signature. The resource server verifies the access token by verifying the signature.

[0042] The OAuth protocol can be used in client login scenarios. On the client login screen, users can choose to log in based on an authorization server, which redirects the client to the authorization server's login page. The user completes authentication on the authorization server, confirms the scope of authorization, and the authorization server returns an authentication code to the client. The client can then exchange the authentication code and application signature for an access token from the authorization server, after which the client can establish a connection with the server based on the access token.

[0043] To address the problems existing in related technologies, this specification provides an embodiment as follows: Figure 4 The network connection is shown. (As shown in the image.) Figure 4 As shown, the client and the intranet gateway communicate via a bidirectional long connection. Figure 4 The image shows a WebSocket connection as an example for data transmission. The gateway transmits data through a connection based on a stateless protocol. Figure 4 (The example shown is an HTTP connection) sends a request to the Agent server. With this network connection architecture, the Agent server does not need to store connection state and can proactively initiate multiple tool calls through the gateway. This bidirectional long-lived connection includes, for example, WebSocket connections and gRPC connections. Connections based on stateless protocols include, for example, HTTP connections and SSE connections. The following description uses WebSocket and HTTP connections as examples.

[0044] Figure 5 This is a schematic diagram illustrating the process of establishing a WebSocket connection between the client and the gateway in the embodiments of this specification.

[0045] Steps S501 to S511 describe the process by which the client obtains user authorization information based on OAuth authentication. In this scenario, the user authorization information is an access token provided by the authorization server. This can be understood as follows: in a scenario where the user logs in by entering their account and password, the user authorization information is the account and password entered by the user through the client.

[0046] refer to Figure 5 In step S501, the client receives the user's request to log in through the authorization server.

[0047] The client interface provides controls for logging in through one or more authorization servers. Users can click on these controls to select an authorization server for login.

[0048] In step S503, the user is redirected to the authorization page.

[0049] After receiving the user's click action, if the user has not logged into the authorization server, the client will redirect to the authorization server's login page. On this login page, the user will enter their account and password registered with the authorization server so that the authorization server can authenticate the user's identity, that is, verify that the current user is indeed a user registered with the authorization server.

[0050] In step S505, the authorization server receives the user's confirmation of authorization.

[0051] After a user successfully logs in, the authorization server will prompt the user on the redirected screen to confirm whether to authorize the client's login. The user can confirm the authorization by clicking on this screen.

[0052] In step S507, the authorization server sends an authentication code to the client after the user confirms the authorization.

[0053] This authentication code is used to indicate the user's confirmation of authorized login and includes the authorization server's signature of the confirmation.

[0054] In step S509, the client sends the authentication code and client credentials to the authorization server.

[0055] After receiving the authentication code, the client verifies the signature in the authentication code using the pre-obtained public key of the authorization server. If the verification is successful, the client sends the authentication code along with its own client credentials to the authorization server. Client credentials may include, for example, a client ID and the corresponding key (symmetric key).

[0056] In step S511, after the authorization server verifies the client's credentials, it sends an access token (access_token) to the client.

[0057] The authorization server can verify the received client credentials based on a pre-stored client ID and its credentials. If the verification is successful, an access_token is generated and sent to the client. In one implementation, the access_token includes a string of random numbers and the authorization server's signature. The authorization server stores the following metadata corresponding to the string of random numbers: client identifier, user identifier, authorization scope, expiration time, etc. This metadata can be used to confirm whether the user has authorized login to the client. In another implementation, the access_token may include the aforementioned metadata and the authorization server's signature of that metadata.

[0058] In step S513, the client sends the user authorization information to the gateway via the HTTP POST method.

[0059] To establish a WebSocket connection, the client and the gateway first establish a TCP connection through a three-way handshake.

[0060] Because WebSocket connections are established through an upgrade using the HTTP GET method, where the GET method header is a highly standardized set automatically generated by the browser and cannot be customized, the client cannot send the access_token to the server in the HTTP request header during the upgrade process. If the access_token is sent to the server as an input parameter (the URL parameter) in the HTTP request, the access_token will be exposed due to several factors, including the URL being recorded in the access log, being recorded by all intermediate nodes during HTTP request transmission, and potentially being cached. This results in low security.

[0061] Therefore, after the client establishes a TCP connection with the gateway, the client first sends the user authorization information to the gateway via the HTTP POST method to verify the user's authorization.

[0062] As mentioned above, user authorization information includes, for example, the metadata corresponding to the access_token and the signature of the authorization server. This metadata includes, for example, the client identifier, user identifier, authorization scope, and expiration time.

[0063] In step S515, after the gateway verifies the user's authorization information, it generates a connection token.

[0064] After the gateway successfully verifies the signature of the authorization server, it can obtain metadata based on the access_token. Based on the content of the metadata, it confirms that the user has been authorized to log in to the client. Therefore, the gateway generates a connection token based on the access_token. This connection token may include information such as the client application identifier, authorization scope (connection), short-term validity information, issuer identifier, and gateway signature.

[0065] In step S517, the gateway sends the connection token to the client.

[0066] In step S519, the client upgrades the protocol using the HTTP GET method and establishes a WebSocket connection.

[0067] When establishing a WebSocket connection, the client sends an HTTP GET request. This request includes specific header fields to request an upgrade of the protocol from HTTP to WebSocket; this process is known as the "WebSocket handshake." The GET request used for the "WebSocket handshake" can take the following form:

[0068] GET wss: / / example.com:8181,"token123" / HTTP / 1.1

[0069] Host: example.com

[0070] Upgrade:websocket

[0071] Connection:Upgrade

[0072] Sec-WebSocket-Key:zy6D…

[0073] Sec-WebSocket-Version:13

[0074] The header fields of this GET request include Host, Upgrade, Connection, Sec-WebSocket-Key, and Sec-WebSocket-Version. The Host field specifies the server address, the Upgrade field declares the protocol to be upgraded to, the Connection field declares the connection type to be upgraded to, the Sec-WebSocket-Key field is a random key used for security authentication, and the Sec-WebSocket-Version declares the supported WebSocket versions.

[0075] By including the connection token "token123" in the parameters of this GET request, the gateway can upgrade the HTTP connection after verifying the connection token. Since the connection token itself is a short-lived token specifically used for upgrading protocols, placing the connection token in the URL parameters will not have a significant impact on security.

[0076] In step S521, the gateway returns an HTTP response indicating the switching protocol and the ChannelID for the WebSocket connection.

[0077] After receiving the GET request, the gateway verifies the gateway signature in the connection token and uses the information in the connection token to verify whether the client has connection permission. Once the gateway determines that the client has connection permission, it returns an HTTP 101 status code to the client.

[0078] HTTP / 1.1 101 Switching Protocols

[0079] Connection:Upgrade

[0080] Sec-WebSocket-Accept: DEJa…

[0081] Upgrade:WebSocket

[0082] Sec-WebSocket-Accept is a value obtained by encrypting and encoding the Sec-WebSocket-Key in the GET request, which is used to indicate its legitimacy.

[0083] After receiving the HTTP 101 status code, the connection between the client and the server is upgraded to a WebSocket connection. The server can generate a long-lived channel identifier (ChannelID) corresponding to the client and send the ChannelID to the client through the WebSocket connection's data frame.

[0084] After a WebSocket connection is established between the client and the gateway, they can maintain a heartbeat, periodically sending probe packets to the other end and receiving acknowledgment packets in return, thus monitoring the other end's status in real time. The gateway can forcibly close the connection if it detects a loss of client heartbeat or connection failure. Additionally, if the gateway determines that the WebSocket connection is idle after a preset period of inactivity, it can automatically disconnect the connection. The gateway can limit the maximum number of connections and the number of concurrent client connections to prevent malicious congestion.

[0085] exist Figure 5 The method shown uses the HTTP POST method for authentication based on the OAuth protocol, which avoids the limitation that the header cannot be customized during the "WebSocket handshake" process. This allows for more secure and cost-effective authentication, and the protocol can be upgraded after successful authentication.

[0086] Figure 6 In the embodiments described in this specification, the Agent server ( Figure 6 The diagram shows the method flowchart for calling client tools during the inference process (as shown in the image).

[0087] like Figure 6 As shown, in step S601, the client sends a query statement Query and the WebSocket connection identifier ChannelID corresponding to the client through the WebSocket connection.

[0088] Figure 7 This is a schematic diagram of the client structure in the embodiments of this specification. As described above, this client, for example, corresponds to CodeAgent and includes an editor module, an MCP Server module, and an MCP Client SDK module. The editor module is the client's user interface, where users can input queries. For the security of the user's device, the editor can be placed in a sandbox. A sandbox is a security isolation mechanism that imposes resource access restrictions on programs at the operating system / runtime level, ensuring they run within a controlled scope. The MCP Server module can receive tool invocation requests and execute tools based on these requests; for example, it can access local files by executing tools. The MCP Client SDK module acts as a local tool invocation proxy, routing tool invocation requests received from the Agent server to the local MCP Server, receiving tool invocation results from the MCP Server, encapsulating the results, and sending them back to the Agent server.

[0089] As mentioned above, the Agent corresponding to the client is used, for example, to modify the code. The user can enter a Query into the client's user interface to instruct the Agent to rewrite the code file "UserCode" stored locally on the client.

[0090] In step S603, the gateway sends the Query and ChannelID to the server via an HTTP connection.

[0091] If the gateway does not currently have an HTTP connection with the server, it can first initiate a three-way handshake to establish a TCP connection. After establishing the TCP connection, the gateway sends an HTTP request (referred to as an HTTP request) to the server based on this TCP connection. This HTTP request may use the POST method and include the server's URL to send the Query and ChannelID to the server. The ChannelID is used to locate the WebSocket connection between the client and the gateway when the Agent server sends a tool invocation request to the gateway. If the gateway already has a persistent HTTP connection with the server, it can directly use the existing HTTP connection to send the Query and ChannelID to the server.

[0092] After receiving the HTTP request, the server can reply with an acknowledgment (ACK) to the gateway. With HTTP version 1.0, the gateway and server can terminate the FTP connection using a four-way handshake. With persistent HTTP, the gateway and server can maintain the TCP connection for subsequent HTTP requests. In one implementation, the gateway can push messages to the Agent server via an SSE connection. It is understood that neither the gateway nor the server stores protocol state in either HTTP or SSE connections.

[0093] In step S605, the Agent server performs inference based on the Query using the Agent and generates a tool invocation request.

[0094] The Agent server includes an MCP client. When the client's query is the first query in a session between the client and the Agent, the Agent server generates a unique session ID. In response to the opening of a session, the MCP client can proactively generate a tool list request to retrieve the tool list from the MCP server. This tool list request can be in, for example, a JSON-RPC format like the following:

[0095] {

[0096] "jsonrpc":"2.0"

[0097] "id": Req_1

[0098] "method": "tools / list"

[0099] "params": {}

[0100] }

[0101] Here, "jsonrpc" indicates the JSON-RPC format version. "Req_1" represents the request identifier RequestID. In this embodiment of the specification, the RequestID can be obtained by appending a number to the sessionID. For example, assuming the sessionID is 123456, the RequestID of this request can be 123456_1.

[0102] The MCP Client can send a request for the MCP tool list to the gateway. The gateway can pre-obtain and store the tool list from the client's MCP Server, and then return the tool list to the MCP Client. This tool list may include, for example, the names of tools such as file reading tools and file writing tools.

[0103] During the inference process based on the query, the Agent may output a tool invocation request: to read the file "UserCode" stored locally on the client. The MCP Client can parse the Agent's output and generate a corresponding tool invocation request. This request, for example, is used to invoke a file reading tool from the tool list, and can take the following form:

[0104] {

[0105] “jsonrpc”: “2.0”,

[0106] "id": Req_2,

[0107] "method": "tools / call",

[0108] “params”: {

[0109] “name”:“read_file”

[0110] "arguments":"UserCode"}

[0111] }

[0112] Here, Req_2 is the RequestID of the tool call request. Similar to Req_1 above, it can be generated based on the session ID above, for example, Req_2=123456_2. "read_file" is the name of the tool to be called, i.e., the file reading tool. "arguments": "UserCode" indicates that the input parameter to the tool "read_file" is the file name "UserCode". It can be understood that the input parameter here is not limited to the file name, but can also be, for example, the file storage address, etc.

[0113] In step S607, the Agent server sends a tool invocation request and ChannelID to the gateway via an HTTP connection. The result of the tool invocation includes the requestID.

[0114] When a short HTTP connection is established between the gateway and the server, the agent server can re-establish a TCP connection with the gateway to send HTTP requests. When a persistent HTTP connection is established between the gateway and the server, the agent server can reuse the previous TCP connection to send HTTP requests. In one implementation, the agent server can push data to the gateway via an SSE connection.

[0115] After sending a tool invocation request, the Agent server can suspend the request to await the result. Specifically, the Agent server can store the requestID and the tool invocation request context in memory and release the current thread (or put the current thread to sleep). The tool invocation request context may include data such as callback functions and timeout timers.

[0116] In step S609, the gateway sends a tool invocation request via a WebSocket connection based on the ChannelID.

[0117] The gateway can look up the corresponding WebSocket connection handle based on the ChannelID. The connection handle is essentially a memory object reference created by the server framework for each connection. The gateway can generate a connection handle after creating a WebSocket connection and store the mapping relationship between the ChannelID and the connection handle in memory. Therefore, the gateway can obtain the connection handle corresponding to the ChannelID based on this mapping relationship, and use this connection handle to send tool invocation requests via the WebSocket connection.

[0118] After sending the tool invocation request, the gateway can suspend the request, similar to the agent, to await the result. Specifically, the gateway can store the requestID and the tool invocation request context (RequestContext) in memory and release the current thread. The tool invocation request context may include data such as callback functions and timeout timers.

[0119] In step S611, the client invokes the tool according to the tool invocation request and obtains the tool invocation result.

[0120] refer to Figure 7After the client receives the tool call request and ChannelID via WebSocket, the MCPClient SDK obtains the tool call request and sends it to the MCP Server.

[0121] The MCP Server retrieves the tool name, input parameters, and RequestID from the tool invocation request. Based on the input parameters, it executes the tool and obtains the tool invocation result, which may include, for example, the read file "UserCode" and the RequestID. The MCP Server then sends the tool invocation result to the MCP Client SDK for feedback.

[0122] In step S613, the client sends the tool call result to the gateway via a WebSocket connection.

[0123] After receiving the tool call result, the MCP Client SDK can encapsulate the tool call result into a standard JSON-RPC response according to the MCP protocol, and send the encapsulated tool call result to the gateway via a WebSocket connection.

[0124] In step S615, the gateway sends the tool call result to the Agent server via an HTTP connection.

[0125] The gateway can parse the RequestID corresponding to the tool invocation result from the tool invocation result, and obtain the context based on the pre-stored mapping between RequestID and tool invocation request context. Then, the gateway can perform subsequent operations based on this context, namely, sending the tool invocation result to the Agent server via an HTTP connection.

[0126] After receiving the tool invocation result, the agent server can similarly parse the corresponding RequestID from the result and retrieve the context based on the pre-stored mapping between RequestID and tool invocation request context. Then, the agent server can perform subsequent operations based on this context; that is, the agent modifies the UserCode file read according to the user's query to obtain optimized code. Afterwards, the agent server sends the optimized code and ChannelID to the gateway via an HTTP connection. The gateway, based on the ChannelID, finds the corresponding WebSocket connection for the client and sends the optimized code to the client through this WebSocket connection for display in an editor.

[0127] Figure 6In the tool invocation method shown, a WebSocket connection is established between the client and the gateway, and an HTTP connection is established between the gateway and the Agent server. The WebSocket connection and the HTTP connection are associated based on the ChannelID. This allows the Agent service to initiate multiple rounds of tool invocations without maintaining the connection state, saving Agent service resources. In addition, the client only needs to access the gateway based on the WebSocket connection standard and does not need to worry about the complexity of the MCP protocol. The Agent server only needs to focus on the MCP protocol and does not need to know the details of the WebSocket connection to realize the internal and external MCP data channels, thus achieving low-cost connection.

[0128] Figure 8 This is a system architecture diagram of another embodiment of this specification. Figure 4 The difference in the system architecture shown is that, Figure 8 The gateway in the system comprises two separate gateways: an external gateway and an internal gateway. The client communicates with the external gateway via a WebSocket persistent connection, while the internal gateway communicates with the Agent server via an HTTP connection. The external and internal gateways can communicate via, for example, a Remote Procedure Call (RPC). It's important to understand that communication between the external and internal gateways is not limited to RPC; other connection protocols can also be used, such as HTTP connections.

[0129] Figure 9 This is a flowchart illustrating the method by which the Agent server calls the client tool during the inference process in the embodiments of this specification.

[0130] like Figure 9 As shown, in step S901, the client sends a Query and a ChannelID to the external gateway via a WebSocket connection.

[0131] In step S903, the external gateway sends a Query and a ChannelID to the internal gateway via RPC.

[0132] Specifically, both the external and internal gateways can start an RPC service by running RPC code, providing the RPC service name and port. The RPC code includes both server-side (called) and client-side (initiating) logic. The external gateway can dynamically obtain the internal gateway's gateway address through service discovery and initiate RPC calls to the internal gateway based on that address.

[0133] In step S905, the internal gateway sends the Query and ChannelID to the Agent server via an HTTP connection.

[0134] In step S907, the Agent server performs inference based on the Query using the Agent and generates a tool invocation request.

[0135] In step S909, the Agent server sends a tool invocation request and ChannelID to the internal gateway via an HTTP connection.

[0136] In step S911, the internal gateway sends a tool call request and ChannelID to the external gateway via RPC.

[0137] In step S913, the external gateway sends a tool invocation request to the client via a WebSocket connection based on the ChannelID.

[0138] In step S915, the client invokes the tool according to the tool invocation request and obtains the tool invocation result.

[0139] In step S917, the client sends the tool invocation result to the external gateway via a WebSocket connection.

[0140] In step S919, the external gateway sends the tool call result to the internal gateway via RPC.

[0141] In step S921, the internal gateway sends the tool invocation result to the Agent server via an HTTP connection.

[0142] exist Figure 9 In the method shown, the external gateway is responsible for establishing WebSocket connections and data interaction with the client, while the internal gateway acts as a bridge between the external gateway and the Agent server to execute business logic. This decouples connection management from business logic and enables the server to call client tools in reverse based on the RPC interface.

[0143] Figure 9 Steps S901, S905-S909, S913-S917, and S921 in the method shown can be referred to Figure 6 The descriptions of each step are omitted here.

[0144] In yet another embodiment of this specification, such as Figure 10 As shown in the system architecture, the external gateway and internal gateway can be designed as distributed clusters. For example... Figure 10 As shown, multiple external gateways ( Figure 10 The diagram shows g11-g13 as an example, interconnected with each other, and multiple internal gateways ( Figure 10(As shown in the example, g21-g23 are interconnected. The client can establish a connection with any external gateway, and the Agent server establishes a connection with each internal gateway.)

[0145] exist Figure 10 In the system architecture shown, after the Agent server generates a tool call request (hereinafter referred to as an MCP request) based on the output of the Agent's calling tool from the MCP Client, it generates a RequestID (e.g., Req_2) for the current MCP request according to the existing RequestID corresponding to the current session (e.g., Session_1). The specific method for generating this RequestID can be as described above. Afterwards, the Agent server can send the MCP request containing the RequestID and the pre-acquired ChannelID together to any of the multiple internal gateways (e.g., internal gateway g22).

[0146] After receiving the MCP request from the Agent server, to ensure that the tool invocation result corresponding to the request can still be returned to internal gateway g22, internal gateway g22 can send its connection information along with the MCP request to the external gateway g12 corresponding to the ChannelID. This connection information can be an IP address or other information corresponding to an IP address. Specifically, internal gateway g22 can store the mapping relationship between ChannelIDs and external gateways, thus determining the external gateway g12 based on the ChannelID received from the Agent server, and sending the MCP request, the connection information of internal gateway g22, and the ChannelID together to external gateway g12, while suspending the MCP request.

[0147] External gateway g12 can send the MCP request and connection information to the client using the WebSocket connection corresponding to the ChannelID. The client, based on the MCP request, invokes the tool using the MCP Client SDK and MCP Server, obtains the tool invocation result, and returns the tool invocation result and connection information to external gateway g12 via the aforementioned WebSocket connection. The tool invocation result includes the RequestID. External gateway g12 then returns the MCP result to internal gateway g22 based on the connection information. Internal gateway g22 then retrieves the context of the pending MCP request based on the RequestID and returns the MCP result to the Agent server. The Agent server can retrieve the previously pending context based on the RequestID and continue the subsequent inference process based on the MCP result.

[0148] In one implementation, to ensure that requests within the same session are handled by the same internal gateway (e.g., internal gateway g22), the Agent server can generate a session identifier based on the IP address and unique identifier UUID of internal gateway g22 when initiating a session. For example, the session identifier can be generated by concatenating the IP address and UUID, i.e., Session_1 = IP + UUID, or it can be encoded or obfuscated using a preset encoding method after concatenation to obtain the session identifier. Then, similarly, the request identifier can be obtained by concatenating an auto-incrementing number into the session identifier, for example, Req_2 = (IP + UUID)_2. This configuration ensures that the IP address of internal gateway g22 is included in the RequestID. When internal gateway g22 sends an MCP request to external gateway g12, the MCP request includes the RequestID, which in turn contains the IP address; therefore, internal gateway g22 does not need to explicitly send its IP address to external gateway g12.

[0149] In response to Figure 10 During the tool invocation process in the illustrated architecture, the internal gateway sends its connection information (e.g., IP address) along with the MCP request to the external gateway. This allows the external gateway to accurately return the tool invocation result to the internal gateway that initially sent the MCP request based on the internal gateway's connection information, ensuring session stability. It is understood that the embodiments in this specification are not limited to an architecture with multiple external gateways and multiple internal gateways. For example, it may also include an architecture with a single external gateway and multiple internal gateways. In such an architecture, the tool invocation result can still be accurately returned to the internal gateway that initially sent the MCP request based on the internal gateway's connection information.

[0150] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0151] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0152] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0153] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0154] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in the same or more software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0155] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0156] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0158] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0159] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0160] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0161] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0163] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0164] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for invoking a client tool, executed by a gateway, wherein the gateway has established a first connection with the client, the first connection including a bidirectional long-lived connection, the method comprising: The query statement to be input into the server-side large model and the connection channel identifier of the first connection are received from the client through the first connection. The query statement and the connection channel identifier are sent to the server through a second connection, which transmits data based on a stateless protocol. The client receives a tool invocation request and the connection channel identifier from the server via the second connection. The tool invocation request includes a request identifier and the tool name of the tool provided by the client. The first connection is obtained based on the connection channel identifier; the tool invocation request is sent to the client through the first connection; The tool invocation result is received from the client via the first connection, and the tool invocation result includes the request identifier. The tool call result is sent to the server through the second connection.

2. The method according to claim 1, wherein the first connection includes a WebSocket connection, the WebSocket connection transmitting data based on the WebSocket protocol, and the second connection includes an HTTP connection, the HTTP connection transmitting data based on the HTTP protocol.

3. The method according to claim 2, wherein the gateway includes an external gateway and an internal gateway, and the client establishes a WebSocket connection with the external gateway. The step of receiving the tool invocation request and the connection channel identifier from the server through the second connection includes: The internal gateway receives a tool invocation request and the connection channel identifier from the server via the HTTP connection. The method further includes the internal gateway sending the tool invocation request and the connection channel identifier to the external gateway via Remote Procedure Call (RPC). The step of obtaining the first connection based on the connection channel identifier and sending the tool invocation request to the client through the first connection includes: the external gateway obtaining the WebSocket connection based on the connection channel identifier and sending the tool invocation request to the client through the WebSocket connection.

4. The method according to claim 2 or 3, wherein the gateway includes an external gateway and multiple internal gateways, and the client establishes a WebSocket connection with the external gateway. The step of receiving the tool invocation request and the connection channel identifier from the server through the second connection includes the first internal gateway among the plurality of internal gateways receiving the tool invocation request and the connection channel identifier from the server through the second connection; The method further includes: The first internal gateway sends the tool invocation request, the connection channel identifier, and the connection information of the first internal gateway to the external gateway; Receiving the tool call result from the client through the first connection includes the external gateway receiving the tool call result from the client through the WebSocket connection and sending the tool call result to the first internal gateway based on the connection information.

5. The method according to claim 2 or 3, wherein the gateway includes an external gateway and multiple internal gateways, and the client establishes a WebSocket connection with the external gateway. The step of receiving a tool invocation request and the connection channel identifier from the server through the second connection includes receiving the tool invocation request and the connection channel identifier from the server by a first internal gateway among the plurality of internal gateways, wherein the request identifier in the tool invocation request is generated based on the connection information of the first internal gateway; The method further includes: The first internal gateway sends the tool invocation request and the connection channel identifier to the external gateway; Receiving the tool call result from the client through the first connection includes the external gateway receiving the tool call result from the client through the WebSocket connection, obtaining the connection information from the request identifier in the tool call result, and sending the tool call result to the first internal gateway based on the connection information.

6. The method according to claim 2 or 3, wherein the gateway comprises multiple external gateways and multiple internal gateways, and the client establishes a WebSocket connection with a first external gateway among the multiple external gateways. The step of receiving a tool invocation request and the connection channel identifier from the server through the second connection includes receiving the tool invocation request and the connection channel identifier from the server by a first internal gateway among the plurality of internal gateways, wherein the request identifier in the tool invocation request is generated based on the connection information of the first internal gateway; The method further includes: The first internal gateway determines the first external gateway based on the connection channel identifier, and sends the tool invocation request and the connection channel identifier to the first external gateway. Receiving the tool call result from the client through the first connection includes the first external gateway receiving the tool call result from the client through the WebSocket connection, obtaining the connection information from the request identifier in the tool call result, and sending the tool call result to the first internal gateway based on the connection information.

7. The method according to claim 2, further comprising: Authentication information is received from the client via an HTTP connection; If authentication is successful based on the authentication information, a connection token corresponding to the client is sent. Based on the connection token, a WebSocket connection is established with the client; After establishing the WebSocket connection, the connection channel identifier is sent to the client through the WebSocket connection.

8. The method according to claim 2, wherein sending the tool invocation request to the client through the first connection based on the connection channel identifier comprises: finding the WebSocket connection handle corresponding to the connection channel identifier based on the connection channel identifier, and sending the tool invocation request to the client based on the WebSocket connection handle.

9. The method according to claim 1, further comprising: After receiving a tool invocation request from the server, the request identifier is stored in association with the context of the tool invocation request. Sending the tool invocation result to the server via the second connection includes obtaining the context based on the request identifier in the tool invocation result, and sending the tool invocation result to the server via the second connection according to the context.

10. The method of claim 1, wherein the tool includes a file reading tool, and the tool invocation result includes a file read from the device where the client is located by executing the tool.

11. A gateway comprising a memory and a processor, the memory storing executable code, wherein the processor, when executing the executable code, implements the method of any one of claims 1-10.

Images