Intelligent agent interaction processing method and system based on end-cloud cooperation and electronic system

Abstract

The application provides an agent interaction processing method and system based on end-cloud cooperation and an electronic system, and relates to the technical field of artificial intelligence. The method comprises the following steps: in response to a user interaction request from a client, driving a target agent to perform reasoning and determining a target tool from a unified tool pool that aggregates cloud tools and client tools according to a reasoning result; if the target tool is a cloud tool, executing the target tool on a server side and generating an execution result, and if the target tool is a client tool, generating a calling mark object for instructing the client to call a corresponding native function; converting the execution result or the calling mark object into a first standard instruction frame and sending the first standard instruction frame to the client, so that the client performs a corresponding local operation; receiving result data returned from the client and restoring a context state of the target agent according to the result data to drive subsequent reasoning. The scheme can solve the problem that end-side capabilities cannot be directly called by a cloud large model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to an intelligent agent interaction processing method, system, and electronic system based on edge-cloud collaboration. Background Technology

[0002] As Large Language Models (LLMs) penetrate mobile devices, edge-cloud collaboration has become the mainstream architecture for achieving intelligent interaction. In this architecture, the cloud-based LLM acts as the decision-making brain of the intelligent agent, while the mobile device serves as the execution terminal providing native hardware capabilities (such as cameras and positioning).

[0003] Currently, in the industry's edge-cloud collaboration framework implementation technology, the intelligent agent runs entirely on the cloud server. When a user request involves local device operations, the cloud-based intelligent agent generates a structured invocation instruction and sends it to the mobile client via a long-lived connection. The client needs to pre-write and deploy specific parsing and execution code to map this instruction to the corresponding native API to perform the operation, and then return the result. In this mode, edge capabilities cannot be scheduled by the cloud-based intelligent agent as simply and directly as calling a local function.

[0004] In developing this application, the inventors discovered that this problem arises because the calling paradigms of intelligent agents for cloud tools and client tools are fragmented, and the latter is opaque to the intelligent agent. For cloud tools, the intelligent agent can directly call them and synchronously obtain results; for client tools, the intelligent agent can only execute an asynchronous process of "generating instructions - sending - waiting." This fragmentation prevents the intelligent agent from forming a unified tool management view, directly resulting in high system coupling and cumbersome expansion. Specifically, adding a new native capability to the client (such as "reading NFC") requires simultaneous, tightly coupled code modifications and release coordination on both the cloud and client sides; any lag or error on either side will lead to call failure.

[0005] Therefore, how to provide an intelligent agent interaction processing method to overcome the above-mentioned fragmentation of the calling paradigm, so that cloud-based intelligent agents can schedule and execute tools located in the cloud and on the client in a unified and transparent manner, thereby reducing system coupling and improving the flexibility and maintainability of capability integration, has become an urgent technical problem to be solved. Summary of the Invention

[0006] The purpose of this application is to provide an intelligent agent interaction processing method, system, and electronic system based on edge-cloud collaboration, which enables cloud-based intelligent agents to schedule and execute tools located in the cloud and on the client in a unified and transparent manner, thereby reducing system coupling and improving the flexibility and maintainability of capability integration.

[0007] This application is implemented as follows:

[0008] In a first aspect, this application provides an intelligent agent interaction processing method based on edge-cloud collaboration, applied to a server. The method includes: responding to a user interaction request from a client, driving a target intelligent agent to perform inference, and determining a target tool to be invoked from a unified tool pool based on the inference result, wherein the unified tool pool aggregates cloud tools executed on the server side and client tools registered and executed by the client; if the target tool is a cloud tool, executing the target tool on the server side and generating an execution result; if the target tool is a client tool, generating a call marker object to instruct the client to invoke a corresponding native function; converting the execution result or call marker object into a first standard instruction frame and sending it to the client so that the client can perform a corresponding local operation; receiving result data from the client performing the corresponding local operation, and restoring the context state of the target intelligent agent accordingly to drive subsequent inference.

[0009] Secondly, this application provides an intelligent agent interaction processing system based on edge-cloud collaboration, applied to a server. The system includes: a reasoning decision module configured to: respond to a user interaction request from a client, drive a target intelligent agent to perform reasoning, and determine a target tool to be invoked from a unified tool pool based on the reasoning result, wherein the unified tool pool aggregates cloud tools executed on the server side and client tools registered and executed by the client; a tool execution module configured to: if the target tool is a cloud tool, execute the target tool on the server side and generate an execution result; if the target tool is a client tool, generate a call marker object to instruct the client to invoke the corresponding native function; an instruction issuance module configured to: convert the execution result or call marker object into a first standard instruction frame and send it to the client so that the client can perform corresponding local operations; and a reasoning recovery module configured to: receive the result data of the client performing the corresponding local operations, and restore the context state of the target intelligent agent accordingly to drive subsequent reasoning.

[0010] Thirdly, this application provides an electronic device including a memory for storing one or more programs; a processor; and, when the one or more programs are executed by the processor, implementing the method as described in any one of the first aspects above.

[0011] Compared with the prior art, this application has at least the following advantages or beneficial effects:

[0012] This application proposes an intelligent agent interaction processing method based on edge-cloud collaboration. It aggregates cloud-based and client-side tools by introducing a unified tool pool, providing the target intelligent agent with a unified calling interface that does not distinguish execution location. This achieves a unified and transparent calling paradigm, allowing the target intelligent agent to call tools using the same logic without needing to be aware of their actual location. This fundamentally eliminates conditional branches and complex adaptations in the intelligent agent code caused by different tool locations, reducing the complexity and error probability of intelligent agent development. Simultaneously, the cloud and client communicate through a standardized first-standard instruction frame, resulting in a single and stable interaction interface. When adding or modifying client tools, the main issue is client adaptation; the core decision-making logic and calling code of the cloud-based intelligent agent do not require frequent changes, improving the maintainability and scalability of the server system. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a flowchart of an embodiment of an intelligent agent interaction processing method based on edge-cloud collaboration according to this application;

[0015] Figure 2 This is a flowchart illustrating the steps of driving the target intelligent agent to perform inference in one embodiment of this application.

[0016] Figure 3 This is a data processing flowchart in one embodiment of the present application when the reasoning result indicates that a dynamic interactive interface needs to be generated;

[0017] Figure 4 This is a structural block diagram of an embodiment of an intelligent agent interaction processing system based on edge-cloud collaboration according to this application;

[0018] Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0019] Icons: 101, Inference Decision Module; 102, Tool Execution Module; 103, Command Issuance Module; 104, Inference Recovery Module; 201, Processor; 202, Memory; 203, Communication Interface. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0021] In this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, without necessarily requiring or implying any such actual relationship or order between these entities or operations.

[0022] In existing edge-cloud collaboration frameworks, when a device capability needs to be invoked, the cloud-based agent generates a structured invocation command and sends it to the client for execution via a persistent connection. Generally, when a user request involves local device operations, the agent generates a natural language command or a predefined JSON command and sends it to the client via a persistent connection. The client parses the command, invokes the corresponding system API, executes it, and then sends the result back to the cloud as text or fixed-format data. In this process, the agent's invocation of local capabilities is essentially a Remote Procedure Call (RPC).

[0023] In the process of implementing this application, the inventors discovered that under the existing model, client capabilities are difficult to be scheduled by the cloud-based intelligent agent as simply and directly as calling local functions. Therefore, every time the client adds a new capability (such as reading NFC), it is necessary to modify both the instruction generation code of the cloud-based intelligent agent and the parsing and execution code of the client, and coordinate the release and update of both ends, resulting in extremely high system coupling and extremely cumbersome expansion.

[0024] Furthermore, the inventors discovered that the reason client-side capabilities are difficult for cloud-based agents to schedule as simply and directly as calling local functions is because the agents' calling paradigms for cloud tools and client-side tools are fragmented. Specifically, for cloud tools, the agent can call them directly; however, for client-side tools, the agent can only execute an asynchronous process of "generating instructions - sending - waiting." This fragmentation prevents the agent from managing and scheduling all tools with a unified logical view. Consequently, every time a native capability is added to the client, tight-coupling code modifications and release coordination must be performed simultaneously on both the cloud and the client side. This directly leads to tight system coupling and cumbersome expansion, resulting in low flexibility and maintainability in integrating client-side capabilities.

[0025] To address these issues, this application proposes an intelligent agent interaction processing method based on edge-cloud collaboration. This method constructs a unified resource abstraction layer (specifically, a unified tool pool) for cloud-based intelligent agents, enabling them to schedule and execute tools that physically process cloud and client data separately using a unified and transparent calling paradigm. This fundamentally solves the system coupling problem caused by calling paradigm isolation.

[0026] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings. Unless otherwise specified, the various embodiments and features described below can be combined with each other.

[0027] Please see Figure 1 This intelligent agent interaction processing method based on edge-cloud collaboration is applied to a server, and the method includes:

[0028] Step S101: In response to a user interaction request from the client, drive the target agent to perform reasoning, and determine the target tool to be invoked from the unified tool pool based on the reasoning result. The unified tool pool aggregates cloud tools executed on the server side and client tools registered and executed by the client.

[0029] In step S101 above, when a user initiates a user interaction request on the client (e.g., "Take a picture and analyze the plants in it"), this user interaction request is sent to the server. Upon receiving it, the server drives the target agent to perform reasoning. Specifically, the target agent analyzes and plans based on the content of the user interaction request, combined with its internal logic, and outputs the reasoning result for the current stage. This reasoning result indicates the target tool that needs to be invoked next. Next, the server determines the target tool to be invoked from a pre-built unified tool pool based on this reasoning result. This unified tool pool is a data structure that aggregates cloud tools executed on the server side and client tools registered and executed by the client, thus providing the target agent with a logically unified tool view that does not distinguish between physical execution locations. In other words, by introducing a unified tool pool, the target agent does not need to care about the actual execution environment (cloud or client) of the tool when selecting tools, achieving decoupling between the invocation intent and the execution location, and simplifying the agent's decision-making logic.

[0030] For example, taking the interactive request "Take a picture of the plant in it and analyze it" as described earlier, at least initially, a "photo-taking tool" needs to be invoked. Therefore, the photo-taking tool (i.e., the target tool) to be invoked will be determined from a unified tool pool. Of course, if the photo-taking tool has already been invoked, and the current inference result indicates that a "plant analysis tool" needs to be invoked, then the plant analysis tool to be invoked will be determined from the unified tool pool. It should be noted that the photo-taking tool is a client-side tool, while the plant analysis tool is a cloud-based tool. Thus, for the server, whether invoking the photo-taking tool or the plant analysis tool, it does not need to concern itself with the actual execution environment of the tool (cloud or client), allowing for a unified and transparent invocation.

[0031] Step S102: If the target tool is a cloud tool, then the target tool is executed on the server side and the execution result is generated; if the target tool is a client tool, then a call marker object is generated to instruct the client to call the corresponding native function.

[0032] In step S102 above, cloud capabilities are executed synchronously on-premises, while client capabilities are executed by generating a call marker object that instructs the client to invoke the corresponding native function. It should be noted that this call marker object encapsulates a remote call request into a standard data object that can circulate within the system. In other words, this call marker object is a lightweight data structure whose function is to carry the call intent and instruct the client to invoke the corresponding native function on its device.

[0033] In summary, step S102, by generating a call marker object, simplifies the complex Remote Procedure Call (RPC) problem, which depends on specific device environments, into a problem of generating and transmitting an internal data object. This allows the cloud server to initiate calls to client capabilities in the same way it processes local data, which is the core technical means to achieve transparent scheduling.

[0034] For example, taking the interactive request "Take a picture of me and analyze the plants in it" as mentioned earlier, if the plant analysis tool is called, it will be executed locally in the cloud. If the photo tool is called, a call marker object will be generated to instruct the client to call the camera (the target tool is the client's camera), making the agent think that the photo tool has been executed. In fact, the process is suspended, and the subsequent reasoning will be carried out after the client returns the photo result data.

[0035] Step S103: Convert the execution result or call marker object into a first standard instruction frame and send it to the client so that the client can perform the corresponding local operation.

[0036] In step S103, the execution result (for cloud tools) or the call marker object (for client tools) generated in step S102 is encapsulated into a first standard instruction frame with a unified format through a conversion mechanism. Then, the first standard instruction frame is sent to the client via the communication link. The payload of the first standard instruction frame is explicit to the client; it is either content that can be directly displayed from the cloud execution result, or a call marker object that the client needs to parse and trigger a local API. In this way, the client only needs to adapt to one standard instruction frame format to handle two different types of requests (direct results and call marker objects) from the cloud, greatly reducing the complexity of protocol parsing for the client and improving the reliability and maintainability of the communication link.

[0037] Step S104: Receive the result data from the client performing the corresponding local operation, and restore the context state of the target agent accordingly to drive subsequent inference.

[0038] In step S104 above, after the client performs the corresponding local operation (i.e., parses the first standard instruction frame and executes the corresponding native function or processes the displayed content) and returns the corresponding result data to the client, the context state of the target agent will be restored using this result data. This context state includes the target agent's suspended dialogue history, internal variables, and other memory information during the waiting period. After the state is restored, the target agent can drive subsequent reasoning based on the new context (including user requests and operation results), thereby initiating the next round of interaction.

[0039] For example, taking the previously mentioned interactive request "Take a picture of me and analyze the plants in it" as an example, if the camera tool is invoked initially, after the client launches the camera and the user takes a picture, the photo result will be sent back to the server. The server will restore the context to the AI ​​state of waiting for the photo based on the photo result, and then hand the photo over to the corresponding tool. Then, the plant analysis tool will be invoked to complete the subsequent reasoning.

[0040] In other words, step S104 above enables the seamless execution of complex task chains that include client tool calls through the context recovery mechanism. The user perceives it as a continuous intelligent dialogue process, which can solve the problem of easy discontinuity in the interaction state in end-to-cloud collaboration and improve the overall user experience.

[0041] In summary, this application addresses the problem of fragmented tool invocation paradigms between the cloud and the client by constructing a logically unified tool pool and a standardized invocation forwarding mechanism. This enables cloud-based agents to consistently schedule capabilities located at different physical locations, thus resolving the issue of fragmented invocation paradigms. Specifically, through the aforementioned processing flow, the actual execution location of the tools is hidden from the agent. Whether in the cloud or on the client side, the agent selects tools in the same way through the unified tool pool, breaking down the original fragmented invocation paradigms. Secondly, by generating invocation marker objects and converting all issued content into first standard instruction frames, the interaction interface between the server and the client becomes singular and standardized, significantly reducing the coupling between the server and the client.

[0042] Based on the aforementioned solution, please refer to Figure 2 In some implementations of this application, the driving target agent to perform inference includes: Step S201: obtaining the current dialogue context, which is maintained based on historical dialogue records and the internal state information of the target agent; Step S202: obtaining the functional description information of all currently available agents from a dynamic registry, which records at least the built-in agents statically integrated into the server and the external agents dynamically loaded based on external configuration information; Step S203: determining the target agent to handle the user interaction request based on the user interaction request, the current dialogue context, and the functional description information; Step S204: if the target agent is an external agent, forwarding the user interaction request and the current dialogue context to the corresponding external service endpoint through an adaptation proxy, receiving the streaming response from the external service endpoint, and converting the streaming response into an internal standard message format as the inference result.

[0043] Understandably, the above implementation constructs a dynamic and scalable agent scheduling layer, which enables intelligent routing of user requests to the most suitable handler based on dialogue state and real-time available capability set, and unified integration of external services, thereby solving the problems of poor scalability and closed ecosystem of agent systems.

[0044] Specifically, when it's necessary to drive the target agent to perform inference, the current dialogue context is first obtained. This current dialogue context is not simply the chat log; it's maintained based on historical dialogue records and the target agent's own internal state information, containing the complete state of the current session. Simultaneously, an up-to-date list is retrieved from a dynamic registry, recording all currently available agents and their functional descriptions. It's important to note that this dynamic registry records not only the agents built into the system at the factory but also external agents dynamically loaded based on external configuration information.

[0045] Next, the routing agent, specifically responsible for task allocation, comprehensively analyzes the specific content of the user interaction request, the current dialogue context mentioned above, and the functional descriptions of all agents obtained from the dynamic registry. Based on this information, the routing agent determines which agent should be the target agent to handle the current user interaction request. This decision-making process is dynamic and intelligent, aiming to find the most suitable handler for each request.

[0046] If the identified target agent happens to be an external agent, i.e., a service provided by a third party, the server will not directly allow the core process to interface with it. In this case, the adaptation proxy component will package the user interaction request and the current dialogue context and forward it to the external service endpoint corresponding to this external agent. After the external service endpoint processes the request, it will typically return data sequentially as a streaming response. The adaptation proxy will continuously receive this streaming response and be responsible for converting it into a standard message format that the system can understand internally. This converted internal standard message format will then serve as the final inference result and be delivered to subsequent processes.

[0047] In summary, the above implementation achieves dynamic discovery and intelligent scheduling of agent capabilities through a dynamic registry and routing agents. New agents can be integrated into the server without downtime or restart (whether through built-in upgrades or third-party integration), significantly improving server scalability and business agility. Secondly, by introducing an adaptation proxy to unify the handling of external agent interactions, heterogeneous external service interfaces are standardized into a unified internal format, shielding the implementation differences of external systems. This allows upper-layer core processes to call both built-in and external agents without distinction, solving the rigidity problem of traditional monolithic or fixed-combination agent architectures.

[0048] To provide a more intuitive understanding of this application for those skilled in the art, a specific example will be used here. This example illustrates the technical means by which the above implementation method is specifically implemented during the program implementation phase. Specifically, to build the open and scalable intelligent agent ecosystem in the above implementation method, the following three components are mainly adopted:

[0049] (1) Unified Registration and Hot Reload: The server system maintains a component called AgentRegistry (Unified Registration Center), which manages a hybrid registry. This registry records two types of agents: one is the built-in agents statically integrated into the server, which are automatically registered when the service starts; the other is external agents dynamically loaded based on external configuration information, whose metadata (such as name, description, and remote service endpoint address) is stored in the database. The server system supports a hot reload mechanism, which can update the list of available agents in real time through methods such as loadDynamicAgentsFromDB (loading dynamic agents from the database) without restarting the service, thereby achieving seamless expansion of business capabilities.

[0050] (2) External Agent Proxy and Protocol Unification: For external agents registered in the dynamic registry, the server system instantiates a generic DynamicAgent class as its proxy. This adapter mainly performs two tasks: first, context pass-through, which encapsulates the current dialogue history, user intent, and server system context into a standard format (such as JSON) and forwards it to the configured third-party service endpoint via an HTTP POST request; second, streaming response bridging, which listens for streaming responses returned by third-party services through the SSE (Server-Sent Events) protocol and converts these heterogeneous responses into the server system's internal standard AIMessage (Artificial Intelligence Message) format in real time. This process shields the implementation differences of external services, making the underlying protocol stack of the server system unaware of the difference between "local" and "remote" agents.

[0051] (3) Semantic Intent-Based Intelligent Routing: A component called SupervisorAgent acts as the scheduling hub of the server system. Its workflow is as follows: First, it obtains the names and functional descriptions of all available agents from the AgentRegistry (unified registry center) in real time, and dynamically constructs server system prompts for large language model inference based on this information, ensuring that routing decisions are based on the latest capability set. Then, the large language model analyzes the semantics of the user's interaction request, matches and determines the most suitable agent from all available agents as the target agent. To improve efficiency, the server system can also introduce IntentCacheManager to cache high-frequency request intents and routing results, thereby significantly reducing the latency of routing decisions.

[0052] Based on the aforementioned scheme, in some implementations of this application, the construction steps of the unified tool pool include: initializing a unified tool pool during the server startup or initialization phase; loading and instantiating at least one cloud tool executed on the server side, and receiving tool registration requests from at least one client, wherein the tool registration requests carry description information of the client tools; based on the description information, creating a corresponding virtual tool wrapper for each client tool, and registering the virtual tool wrapper and the cloud tool together in the unified tool pool; wherein the virtual tool wrapper is configured to: when invoked by the target intelligent agent, execute the logic for generating the invocation marker object, without executing the actual functional logic of the client tool.

[0053] Understandably, in the above implementation, the construction of client tools involves creating a corresponding virtual tool wrapper for each client tool based on the received description information. These virtual tool wrappers, along with the previously instantiated cloud tools, are then registered in the pre-initialized unified tool pool. It's important to note that the virtual tool wrappers created here are configured with specific logic: when the target agent calls them, they do not execute the actual functional logic of the client tool, but instead execute the logic for generating a call marker object.

[0054] Thus, through the above implementation, this application shields the differences in the actual execution location of tools from the system initialization level. For the target agent at the upper layer, all tools in the unified tool pool, whether cloud-based or virtual tool wrappers, have a consistent callable interface. This eliminates the need for agents to write two different sets of calling code for cloud and client tools, fundamentally simplifying development logic and solving the problem of fragmented calling paradigms. Simultaneously, this mechanism of dynamically creating virtual wrappers based on description information allows new client capabilities to be included in the unified tool pool for agent use simply by registering and informing the server of their description information, achieving flexible and low-cost capability expansion and effectively reducing system coupling.

[0055] Based on the aforementioned scheme, in some implementations of this application, the step of generating a call marker object for instructing the client to call the corresponding native function includes: calling a virtual tool wrapper corresponding to the client tool; executing a marker generation logic through the virtual tool wrapper to generate the call marker object, wherein the call marker object includes a call identifier, a tool name, call parameters, and a specific type attribute for identifying it as a client tool call.

[0056] It's important to note that in the above implementation, the call identifier can be a unique number used to track this specific call request throughout the entire server system. The tool name can be used to explicitly indicate which native function the client needs to invoke, such as "take a picture" or "get location." The call parameters contain the specific input information required to execute the function. Additionally, the specific type attribute contained in the call marker object is an internal marker used to explicitly identify that this object is not a regular execution result, but a specific instruction for client-side tool invocation.

[0057] In summary, the above implementation encapsulates the action of "performing a function on the client side"—an action that depends on a specific device environment and typically requires complex network communication—into a pure data object that can be processed and transferred within the server. This makes the server's operation logic when initiating a client call very similar in form to generating a local data result, thus greatly simplifying the server-side control logic.

[0058] Based on the aforementioned solution, please refer to Figure 3 In some implementations of this application, after the driving target agent performs inference, if the inference result indicates that a dynamic interactive interface needs to be generated, the method further includes: step S301: matching a target template from a pre-set interface template library based on the interface configuration parameters in the inference result; step S302: injecting business logic data and client theme configuration information into the target template, dynamically assembling and generating an independently executable interface resource package, wherein the interface resource package encapsulates predefined cross-layer communication logic for encapsulating user operations on the dynamic interactive interface into standard format interactive events; step S303: converting the interface resource package into a second standard instruction frame and sending it to the client, so that the client renders the interface resource package and presents the dynamic interactive interface in a sandbox environment. The step of receiving result data from the client performing the corresponding local operation includes receiving the result data converted from the interactive event.

[0059] In the above implementation, when the inference result of the target agent indicates the need to generate a dynamic interactive interface, the server system matches a target template from a pre-set interface template library based on the interface configuration parameters in the inference result. Subsequently, business logic data and the client's theme configuration information are injected into the target template, dynamically assembling and generating an independently executable interface resource package. This interface resource package encapsulates predefined cross-layer communication logic, used to encapsulate user operations into standard-format interactive events. Next, the server system converts the interface resource package into a second standard instruction frame and sends it to the client. The client renders the interface resource package in a sandbox environment to present a dynamic interactive interface. Since user operations are transformed into interactive events and transmitted back through cross-layer communication logic, when receiving result data from the client performing corresponding local operations, the data source will include the result data transformed from these interactive events.

[0060] In other words, by implementing the above methods, the user interface (UI) can be dynamically generated and distributed in the cloud. This eliminates the need for client application version updates to launch new services and new interaction methods, greatly improving the agility and flexibility of business iteration. Furthermore, since the interface logic is pre-encapsulated within the interface resource package, the client only needs to provide a standard sandbox rendering environment, reducing reliance on native client development capabilities.

[0061] To enable those skilled in the art to understand this application more intuitively, a specific example will be provided here. This example illustrates the technical means by which the above implementation method is specifically implemented during the program implementation stage.

[0062] Specifically, the implementation process can be as follows:

[0063] The first step is intent recognition and template matching: When the target agent, through reasoning, recognizes that the user request requires complex interaction (e.g., "filling out a risk assessment form"), it doesn't output regular text but triggers an action to generate a dynamic interactive interface. A component called UIComponentTemplates (UI component template engine) receives interface configuration parameters (e.g., form title, option list, theme style, etc.) from the target agent's reasoning results. Based on these interface configuration parameters, the system matches a suitable target template from a pre-built interface template library.

[0064] The second step is dynamic resource assembly: UIComponentTemplates (the UI component template engine) dynamically injects two core pieces of information into the matched target template (which is a pre-built HTML / CSS / JS code skeleton). First, it injects business logic data, filling the template with specific business fields and content. Second, it injects theme configuration information (ThemeConfig) obtained from the client. For example, based on whether the client is currently in light or dark mode, it automatically sets the corresponding primary color (primaryColor) and background color (backgroundColor), ensuring that the generated UI resource package maintains consistency with the native visual style of the client application. During this process, the engine also generates and injects crucial JavaScript scripts containing calls to window.PA25054651rent.postMessage (an interface for communication between WebView and the native container), thus embedding standardized cross-layer communication logic within the UI resource package. This logic predefines how to encapsulate user actions (such as clicks and input) on the dynamic interactive interface into standard-format interactive events.

[0065] The third step is resource encapsulation and distribution: After assembly, the engine encapsulates the generated complete HTML code string into a UIResource object. Subsequently, a component called LegacyToAguiConverter converts this UIResource object into a specific type of AgUiEvent (AG-UI protocol event), whose type (EventType) is marked as CUSTOM and whose name (Name) is "mcp-ui". This AgUiEvent (AG-UI protocol event) is equivalent to the second standard instruction frame in the above implementation. After this second standard instruction frame is sent to the client, the client directly renders the received interface resource package in its secure sandbox environment (such as a WebView or feature card), thus presenting the user with a complete, interactive, and dynamic interface.

[0066] Based on the aforementioned scheme, in some implementations of this application, the step of converting the execution result or call marker object into a first standard instruction frame includes: real-time monitoring of heterogeneous message streams generated internally by the server, wherein the heterogeneous message stream includes at least one of text block events, tool call start events, and tool call parameter events; mapping and assembling the events in the monitored heterogeneous message stream into structured atomic events according to the event type; supplementing the mapped and assembled atomic event sequence with lifecycle control events based on a session state machine; and serializing the atomic event sequence and the supplemented lifecycle control events into a first standard instruction frame conforming to a predetermined protocol format, and streaming it to the client.

[0067] Understandably, this transformation process in the above implementation begins with real-time monitoring of heterogeneous message streams generated internally by the server. These message streams may contain various primitive events such as text block events and tool call start events. The server system maps and reassembles each event into atomic events with a clearer structural definition, based on its specific type. During this process, a session state machine continuously tracks the progress of the entire interactive session. Based on the state machine's judgment, the server system automatically inserts necessary lifecycle control events, such as signals to identify the start or end of the session, into the assembled sequence of atomic events. Finally, this complete sequence, containing both business atomic events and server system control events, is serialized into a first standard instruction frame conforming to a predetermined format and continuously streamed to the client.

[0068] In other words, through the protocol conversion process described above, the complex and potentially changing data model within the server is transformed into a stable, consistent, and easily parsed data stream for the client. This eliminates the need for the client to concern itself with the specific implementation details of the backend intelligent agent framework or tool calls; it only needs to adapt to a single protocol to respond to all types of server events in real time, significantly reducing client development and maintenance costs. Simultaneously, streaming delivery and embedded lifecycle events ensure that the client can perceive the server's processing status in real time (such as "thinking" or "waiting for input"), thereby achieving precise state synchronization and interactive guidance for the local user interface.

[0069] To provide a more intuitive understanding of this application for those skilled in the art, a specific example will be used here. This example illustrates the technical means by which the above implementation is achieved during the program implementation phase. Specifically, a core conversion module (LegacyToAguiConverter, protocol adapter) can normalize the complex streaming output and heterogeneous messages within the server into a standard event stream that is easy for mobile devices to process. Its working principle includes:

[0070] First, the server system internally uses the AG-UI protocol, encapsulated based on the SSE (Server-Sent Events) protocol, to generate raw heterogeneous message streams (SSEMessage streams), which define events of various types such as start, chunk, and tool_call_start. The protocol conversion and communication module (whose functionality can be implemented via LegacyToAguiConverter) intercepts this internal heterogeneous message stream in real time. For text chunk events, this module accumulates and converts them into atomic events of type TEXT_MESSAGE_CONTENT; for tool calls, the core conversion module assembles scattered tool_call_start and tool_call_args events into a single, structurally complete atomic event of type TOOL_CALL.

[0071] Secondly, this core conversion module internally maintains a session state machine (whose state can be represented by variables such as hasStarted and hasFinished) to automatically complete the session's lifecycle control events. For example, a RUN_STARTED event is automatically inserted when the first data packet is received, and a RUN_FINISHED event is ensured to be sent when the stream ends. This mechanism guarantees the completeness of the client's awareness of the server's processing state.

[0072] Finally, all transformed and supplemented atomic events and lifecycle control events will be uniformly serialized into a first standard instruction frame conforming to a predetermined protocol format (such as AgUiEvent in JSON format). In this way, the client only needs to adapt to this one first standard instruction frame format to interface with and understand all types of agent outputs, thereby shielding the differences between upstream models and frameworks.

[0073] Based on the aforementioned scheme, in some implementations of this application, the step of receiving result data from the client performing corresponding local operations and restoring and updating the context state of the target agent accordingly includes: while waiting for the result data, generating state snapshot data of the current running state of the target agent and asynchronously publishing it to a message queue for persistent storage; when the result data is received, extracting a session identifier from the result data; using the session identifier as a key, retrieving the corresponding latest state snapshot data from the persistent storage; based on the retrieved latest state snapshot data, reconstructing the complete context and execution state of the target agent, and injecting the result data into the reconstructed context to drive subsequent inference.

[0074] Understandably, the above implementation, by introducing session identifiers, enables the stateless server to reliably reconstruct and continue an interrupted agent session. This ensures the data continuity and logical correctness of long-term, multi-step interactive tasks, thereby resolving the issues of interrupted interaction processes and state loss caused by client operation delays or network instability. Furthermore, through asynchronous persistence of state snapshots, it ensures that even if the server instance restarts or experiences a network outage, the context of issued client calls will not be lost.

[0075] To enable those skilled in the art to understand this application more intuitively, a specific example will be provided here. This example illustrates the technical means by which the above implementation method is specifically implemented during the program implementation phase. Specifically, to achieve long-term process memory for a "stateless" server and solve the session loss problem caused by mobile network instability, this exemplary solution employs a checkpoint saving mechanism based on NATS and TimescaleDB. Its implementation follows a read-write separation architecture design.

[0076] To ensure high throughput in the write path, when the target agent's state changes (e.g., generating a new response or entering a state awaiting result data), the server system does not directly manipulate the database. Instead, a reliable publishing component (ReliableNatsPublisher) publishes the target agent's current running state, i.e., state snapshot data, as a message to a specific topic (Subject:CHECKPOINT_CREATE) in the NATS message queue. This design ensures that the main business logic can receive immediate responses in high-concurrency scenarios, avoiding blocking the dialogue process due to direct database I / O operations.

[0077] On the read path, in order to achieve low latency, when a session needs to be restored (such as when a user reconnects or when the result data of a client callback is received), the server system directly reads the latest state snapshot data corresponding to the session from TimescaleDB through the data access layer (drizzle-orm).

[0078] To achieve accurate state recovery, the server system uses the session identifier as a unique index key for retrieval in the database. Whenever the client re-establishes the connection, as long as its request carries the same session identifier, the checkpoint saver (NatsTimescaleSaver)'s query method (getTuple) can use this session identifier to accurately retrieve and restore the previously saved complete dialog context, internal variable state, and program execution pointer from the TimescaleDB, thus achieving a "resume interrupted download" interactive experience.

[0079] Based on the aforementioned scheme, in some implementations of this application, the step of asynchronously publishing to a message queue for persistent storage includes: publishing the state snapshot data to a message middleware; having an independent consumer service subscribe to and consume the state snapshot data from the message middleware, and writing it into a time-series database for persistence.

[0080] In the above implementation, by introducing a message broker and consumer services, the time-consuming operation of state saving is decoupled from the main business logic and made asynchronous, achieving a balance between high performance and high reliability. Specifically, by separating the core dialogue process (generating and issuing commands) from the time-consuming data storage operation in the architecture, the main service can return immediately after publishing a message and continue processing new requests, thus avoiding the delays or blocking that might occur from directly writing to the database, ensuring the overall response speed and throughput of the server system. Simultaneously, a dedicated consumer service is responsible for writing to the database. Even if the database experiences temporary performance fluctuations or brief unavailability, it will not directly affect the availability of the main service. The message broker ensures that state snapshot data is not lost, and the consumer service can continue processing after the database is restored, thereby enhancing the fault tolerance and reliability of the server system.

[0081] Please see Figure 4 This application also provides an intelligent agent interaction processing system based on edge-cloud collaboration, applied to a server. The system includes: a reasoning decision module 101, configured to: respond to a user interaction request from a client, drive the target intelligent agent to perform reasoning, and determine the target tool to be invoked from a unified tool pool based on the reasoning result, wherein the unified tool pool aggregates cloud tools executed on the server and client tools registered and executed by the client; a tool execution module 102, configured to: if the target tool is a cloud tool, execute the target tool on the server and generate an execution result; if the target tool is a client tool, generate a call marker object to instruct the client to invoke the corresponding native function; an instruction issuance module 103, configured to: convert the execution result or call marker object into a first standard instruction frame and send it to the client so that the client can perform the corresponding local operation; and a reasoning recovery module 104, configured to: receive the result data of the client performing the corresponding local operation, and restore the context state of the target intelligent agent accordingly to drive subsequent reasoning.

[0082] For the specific implementation process of the above system, please refer to the intelligent agent interaction processing method based on edge-cloud collaboration provided in the above embodiment, which will not be repeated here.

[0083] Based on the aforementioned scheme, in some implementations of this application, the driving of the target intelligent agent to perform inference includes: obtaining the current dialogue context, which is maintained based on historical dialogue records and the internal state information of the target intelligent agent; obtaining the functional description information of all currently available intelligent agents from a dynamic registry, which records at least the built-in intelligent agents statically integrated into the server and the external intelligent agents dynamically loaded based on external configuration information; determining the target intelligent agent to handle the user interaction request by a routing intelligent agent based on the user interaction request, the current dialogue context, and the functional description information; if the target intelligent agent is an external intelligent agent, forwarding the user interaction request and the current dialogue context to the corresponding external service endpoint through an adaptation proxy, receiving the streaming response from the external service endpoint, and converting the streaming response into an internal standard message format as the inference result.

[0084] Based on the aforementioned scheme, in some implementations of this application, the construction steps of the unified tool pool include: initializing a unified tool pool during the server startup or initialization phase; loading and instantiating at least one cloud tool executed on the server side, and receiving tool registration requests from at least one client, wherein the tool registration requests carry description information of the client tools; based on the description information, creating a corresponding virtual tool wrapper for each client tool, and registering the virtual tool wrapper and the cloud tool together in the unified tool pool; wherein the virtual tool wrapper is configured to: when invoked by the target intelligent agent, execute the logic for generating the invocation marker object, without executing the actual functional logic of the client tool.

[0085] Based on the aforementioned scheme, in some implementations of this application, the step of generating a call marker object for instructing the client to call the corresponding native function includes: calling a virtual tool wrapper corresponding to the client tool; executing a marker generation logic through the virtual tool wrapper to generate the call marker object, wherein the call marker object includes a call identifier, a tool name, call parameters, and a specific type attribute for identifying it as a client tool call.

[0086] Based on the aforementioned scheme, in some implementations of this application, after the driving target agent performs inference, if the inference result indicates that a dynamic interactive interface needs to be generated, the system further includes: a template matching module, configured to: match a target template from a pre-set interface template library based on the interface configuration parameters in the inference result; a resource package assembly module, configured to: inject business logic data and client theme configuration information into the target template, dynamically assemble and generate an independently executable interface resource package, wherein the interface resource package encapsulates predefined cross-layer communication logic for encapsulating user operations on the dynamic interactive interface into standard format interactive events; and a conversion and delivery module, configured to: convert the interface resource package into a second standard instruction frame and send it to the client, so that the client renders the interface resource package and presents the dynamic interactive interface in a sandbox environment. The inference recovery module 104 receives result data from the client performing corresponding local operations, including receiving the result data converted from the interactive events.

[0087] Based on the aforementioned scheme, in some implementations of this application, the step of converting the execution result or call marker object into a first standard instruction frame includes: real-time monitoring of heterogeneous message streams generated internally by the server, wherein the heterogeneous message stream includes at least one of text block events, tool call start events, and tool call parameter events; mapping and assembling the events in the monitored heterogeneous message stream into structured atomic events according to the event type; supplementing the mapped and assembled atomic event sequence with lifecycle control events based on a session state machine; and serializing the atomic event sequence and the supplemented lifecycle control events into a first standard instruction frame conforming to a predetermined protocol format, and streaming it to the client.

[0088] Based on the aforementioned scheme, in some implementations of this application, the step of receiving result data from the client performing corresponding local operations and restoring and updating the context state of the target agent accordingly includes: while waiting for the result data, generating state snapshot data of the current running state of the target agent and asynchronously publishing it to a message queue for persistent storage; when the result data is received, extracting a session identifier from the result data; using the session identifier as a key, retrieving the corresponding latest state snapshot data from the persistent storage; based on the retrieved latest state snapshot data, reconstructing the complete context and execution state of the target agent, and injecting the result data into the reconstructed context to drive subsequent inference.

[0089] Based on the aforementioned scheme, in some implementations of this application, the step of asynchronously publishing to a message queue for persistent storage includes: publishing the state snapshot data to a message middleware; having an independent consumer service subscribe to and consume the state snapshot data from the message middleware, and writing it into a time-series database for persistence.

[0090] Please see Figure 5 This application provides an electronic device including at least one processor 201 and at least one memory 202. The processor 201 and the memory 202 are directly connected to each other, or communicate with each other through a communication interface 203, or are electrically connected through one or more communication buses or signal lines to realize data transmission or interaction. The memory 202 stores program instructions that can be executed by the processor 201. The processor 201 can call the program instructions to execute the program instructions to realize any of the intelligent agent interaction processing methods based on edge-cloud collaboration provided by the above-described implementation methods.

[0091] The memory 202 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0092] The processor 201 can be an integrated circuit chip with signal processing capabilities. The processor 201 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0093] Understandable. Figure 5The structure shown is for illustrative purposes only; the electronic device may also include components that are more advanced than those shown. Figure 5 The more or fewer components shown, or having the same Figure 5 The different configurations shown. Figure 5 The components shown can be implemented using hardware, software, or a combination thereof.

[0094] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that this application can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered illustrative and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this application. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for intelligent agent interaction processing based on edge-cloud collaboration, characterized in that, Applied to a server, the method includes: In response to user interaction requests from clients, the target agent is driven to perform reasoning, and the target tool to be invoked is determined from a unified tool pool based on the reasoning results. The unified tool pool aggregates cloud tools executed on the server side and client tools registered and executed by clients. If the target tool is a cloud tool, then the target tool is executed on the server side and the execution result is generated; if the target tool is a client tool, then a call marker object is generated to instruct the client to call the corresponding native function. The execution result or call marker object is converted into a first standard instruction frame and sent to the client so that the client can perform the corresponding local operation; Receive the result data from the client to perform the corresponding local operation, and restore the context state of the target agent accordingly to drive subsequent inference; The construction steps of the unified tool pool include: initializing a unified tool pool during the server startup or initialization phase; loading and instantiating at least one cloud tool that executes on the server side, and receiving tool registration requests from at least one client, wherein the tool registration requests carry description information of the client tools; based on the description information, creating a corresponding virtual tool wrapper for each client tool, and registering the virtual tool wrapper and the cloud tool together in the unified tool pool; wherein the virtual tool wrapper is configured to: when invoked by the target intelligent agent, execute the logic for generating the invocation marker object, without executing the actual functional logic of the client tool.

2. The method according to claim 1, characterized in that, The driving force for the target intelligent agent to perform reasoning includes: Obtain the current dialogue context, which is maintained based on historical dialogue records and the internal state information of the target agent; The function description information of all currently available intelligent agents is obtained from a dynamic registry, which records at least the built-in intelligent agents that are statically integrated into the server, and the external intelligent agents that are dynamically loaded based on external configuration information. Based on the user interaction request, the current dialogue context, and the function description information, a routing agent determines the target agent to process the user interaction request. If the target agent is an external agent, the user interaction request and the current dialogue context are forwarded to the corresponding external service endpoint through an adaptation proxy, and the streaming response from the external service endpoint is received. The streaming response is then converted into an internal standard message format as the inference result.

3. The method according to claim 1, characterized in that, The step of generating a call marker object to instruct the client to invoke the corresponding native function includes: Invoke the virtual tool wrapper corresponding to the client tool; The virtual tool wrapper executes a tag generation logic to generate the call tag object, wherein the call tag object includes a call identifier, a tool name, call parameters, and a specific type attribute for identifying it as a client tool call.

4. The method according to claim 1, characterized in that, After the driving target agent performs inference, if the inference result indicates that a dynamic interactive interface needs to be generated, the method further includes: Based on the interface configuration parameters in the inference results, the target template is matched from the preset interface template library; Business logic data and client theme configuration information are injected into the target template to dynamically assemble and generate an independently executable interface resource package. The interface resource package contains predefined cross-layer communication logic, which is used to encapsulate user operations on the dynamic interactive interface into standard format interactive events. The interface resource package is converted into a second standard instruction frame and sent to the client so that the client can render the interface resource package and present the dynamic interactive interface in a sandbox environment. The step of receiving result data from the client performing the corresponding local operation includes receiving the result data transformed from the interaction event.

5. The method according to claim 1, characterized in that, The step of converting the execution result or call marker object into a first standard instruction frame includes: Real-time monitoring of heterogeneous message streams generated internally by the server, wherein the heterogeneous message streams include at least one of text block events, tool call start events, and tool call parameter events; The events in the heterogeneous message stream that are monitored are mapped and assembled into structured atomic events according to the event type; Based on the session state machine, lifecycle control events are added to the mapped and assembled atomic event sequence; The atomic event sequence and the supplementary lifecycle control events are serialized into a first standard instruction frame conforming to a predetermined protocol format and streamed to the client.

6. The method according to claim 1, characterized in that, The step of receiving result data from the client performing the corresponding local operation, and accordingly restoring and updating the context state of the target agent, includes: While waiting for the result data, the current running state of the target intelligent agent is generated as a state snapshot data and asynchronously published to the message queue for persistent storage; When the result data is received, the session identifier is extracted from the result data; Using the session identifier as the key, retrieve the corresponding latest state snapshot data from the persistent storage; based on the retrieved latest state snapshot data, reconstruct the complete context and execution state of the target agent, and inject the result data into the reconstructed context to drive subsequent inference.

7. The method according to claim 6, characterized in that, The step of asynchronously publishing to the message queue for persistent storage includes: Publish the state snapshot data to the message middleware; An independent consumer service subscribes to and consumes the state snapshot data from the message middleware and writes it to a time-series database for persistence.

8. A smart agent interaction processing system based on edge-cloud collaboration, characterized in that, The system, applied to a server, includes: The reasoning and decision-making module is configured to: respond to user interaction requests from the client, drive the target intelligent agent to perform reasoning, and determine the target tool to be invoked from a unified tool pool based on the reasoning results, wherein the unified tool pool aggregates cloud tools executed on the server side and client tools registered and executed by the client; The tool execution module is configured to: if the target tool is a cloud tool, execute the target tool on the server and generate an execution result; if the target tool is a client tool, generate a call marker object to instruct the client to call the corresponding native function. The instruction delivery module is configured to convert the execution result or call marker object into a first standard instruction frame and send it to the client so that the client can perform the corresponding local operation; The inference recovery module is configured to receive result data from the client performing corresponding local operations, and thereby restore the context state of the target agent to drive subsequent inference. The construction steps of the unified tool pool include: initializing a unified tool pool during the server startup or initialization phase; loading and instantiating at least one cloud tool that executes on the server side, and receiving tool registration requests from at least one client, wherein the tool registration requests carry description information of the client tools; based on the description information, creating a corresponding virtual tool wrapper for each client tool, and registering the virtual tool wrapper and the cloud tool together in the unified tool pool; wherein the virtual tool wrapper is configured to: when invoked by the target intelligent agent, execute the logic for generating the invocation marker object, without executing the actual functional logic of the client tool.

9. An electronic device, characterized in that, include: Memory, used to store one or more programs; processor; When the one or more programs are executed by the processor, the method as described in any one of claims 1-7 is implemented.

Images