Agent-based UI interaction method, apparatus, device, system, and medium

Abstract

The application discloses a kind of UI interaction method, device, equipment, system and medium based on agent provided in the application, belong to artificial intelligence field.The method comprises: in the case where user accesses target website, the business request prompt word of user is received through dialogue window and is forwarded to agent, and target website and dialogue window are integrated with SDK;The basic information of target WebMCP service fed back by agent is received through dialogue window, and the WebMCP service matched by the subtask of the user intent business according to business request prompt word obtained by agent is target WebMCP service;The service execution interface of the SDK of dialogue window is called, according to the page network address in the service information of target WebMCP service, the service execution callback function in the service information of target WebMCP service is executed, and execution result is obtained.According to the application embodiment, development difficulty can be reduced, and development efficiency is improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence, and in particular relates to a UI interaction method, device, equipment, system and medium based on intelligent agents. Background Technology

[0002] With the continuous development of electronic information technology, more and more services can be conducted through online business websites. Users can log in to these websites and operate through the user interface (UI) to execute business transactions. However, due to the increasing variety of services, the page hierarchy of these websites is becoming more complex, and the navigation paths are lengthy, requiring users to search multiple times to find the desired service. To improve the efficiency of finding services for users, artificial intelligence (AI) technology can be introduced to assist users by executing business transactions through conversational interaction between the user and an intelligent agent.

[0003] However, in order to enable the intelligent agent to have UI interaction capabilities, it is necessary to write complex code that is tightly coupled with the front-end UI for the AI ​​function of the intelligent agent, and to redevelop an interaction system adapted to the intelligent agent, which is difficult and inefficient. Summary of the Invention

[0004] This application provides a UI interaction method, device, equipment, system, and medium based on intelligent agents, which can reduce the development difficulty of UI interaction functions based on intelligent agents and improve development efficiency.

[0005] In a first aspect, embodiments of this application provide a UI interaction method based on an intelligent agent, comprising: when a user visits a target website, receiving a user's business request prompt through a dialog window, forwarding the business request prompt to an intelligent agent, wherein the target website integrates an SDK, and the dialog window integrates an SDK; receiving basic information of a target WebMCP service fed back by the intelligent agent through the dialog window, wherein the target WebMCP service is a WebMCP service matched by the intelligent agent in the WebMCP service list based on the sub-tasks contained in the user's intent business obtained according to the business request prompt, the interface of the WebMCP service includes a service execution interface, the service execution interface is used to execute a service execution callback function, and the service information of the WebMCP service includes a page network address and a service execution callback function; calling the service execution interface through the SDK of the dialog window, executing the service execution callback function in the service information of the target WebMCP service according to the page network address in the service information of the target WebMCP service, and obtaining an execution result.

[0006] Secondly, embodiments of this application provide a UI interaction device based on an intelligent agent, comprising: an interaction module, configured to receive a user's business request prompt through a dialog window when the user visits a target website, and forward the business request prompt to the intelligent agent; the target website integrates an SDK, and the dialog window integrates an SDK; a communication module, configured to receive basic information of a target WebMCP service fed back by the intelligent agent through the dialog window; the target WebMCP service is a WebMCP service matched in the WebMCP service list by the intelligent agent based on the sub-tasks contained in the user's intent business obtained according to the business request prompt; the interface of the WebMCP service includes a service execution interface, which is used to execute a service execution callback function; the service information of the WebMCP service includes a page network address and a service execution callback function; and an execution module, configured to call the service execution interface through the SDK of the dialog window, execute the service execution callback function in the service information of the target WebMCP service according to the page network address in the service information of the target WebMCP service, and obtain an execution result.

[0007] Thirdly, embodiments of this application provide a terminal device, including: a processor and a memory storing computer program instructions; the processor executes the computer program instructions to implement the agent-based UI interaction method of the first aspect.

[0008] Fourthly, embodiments of this application provide a UI interaction system based on an intelligent agent, comprising: a terminal device according to the third aspect; and an intelligent agent configured to infer, based on a service request prompt word sent by the terminal device, the sub-tasks included in the user intent service indicated by the service request prompt word, determine a target WebMCP service that matches the sub-task in a pre-synchronized WebMCP service list, and send basic information of the target WebMCP service to the terminal device.

[0009] Fifthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the agent-based UI interaction method of the first aspect.

[0010] Sixthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the agent-based UI interaction method of the first aspect.

[0011] This application provides a UI interaction method, apparatus, device, system, and medium based on an intelligent agent. When a user visits a target website, a dialog window can receive the user's business request prompt, which is then forwarded to the intelligent agent. The intelligent agent, based on the business request prompt, obtains the sub-tasks included in the user's intent task indicated by the prompt, thereby determining the target WebMCP service matching the sub-task. The intelligent agent feeds back basic information of the target WebMCP service to the dialog window, enabling the dialog window's SDK to identify the target WebMCP service. It then calls the extended service execution interface of the target WebMCP service, loads the page corresponding to the target WebMCP service, executes the service execution callback function in the service information of the target WebMCP service, reuses the resources and functions of the page corresponding to the target WebMCP service on the target website, and implements the sub-task, thus fulfilling the user's intent task. By loading the extended WebMCP service into the SDK of the business website and the dialog window, it eliminates the need to develop a new interactive system adapted to the intelligent agent or change the front-end development architecture. This allows reuse of the existing functions of the business website to achieve intelligent agent-based UI interaction capabilities, thereby reducing the development difficulty and improving development efficiency of intelligent agent-based UI interaction functions. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic diagram of an agent-based UI interaction system provided in an embodiment of this application; Figure 2 A flowchart illustrating an embodiment of the agent-based UI interaction method provided in this application; Figure 3 A flowchart illustrating an example of the service registration phase provided in this application embodiment; Figure 4 A flowchart illustrating an example of the service discovery and monitoring phase provided in an embodiment of this application; Figure 5 A flowchart illustrating an example of the user interaction and service matching phase provided in an embodiment of this application; Figure 6 A flowchart illustrating an example of the service execution phase provided in an embodiment of this application; Figure 7 A flowchart illustrating an example of result feedback and subsequent planning provided in an embodiment of this application; Figure 8This is a schematic diagram of the structure of a UI interaction device based on an intelligent agent provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application. Detailed Implementation

[0014] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the purpose, technical solution, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of the details in these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application. It should be noted that the acquisition, storage, use, and processing of information and data in the embodiments of this application are all authorized by users or relevant organizations and comply with the relevant provisions of national laws and regulations. In the embodiments of this application, certain software, components, models, and other existing solutions in the industry may be mentioned. These should be considered as exemplary, and their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or necessarily used such a solution.

[0015] With the continuous development of electronic information technology, more and more business operations can be conducted through online business websites. Users can log in to these websites and interact with the user interface to execute business requests. However, due to the increasing variety of business types, the page hierarchy of these websites is becoming more complex, and navigation paths are lengthy, requiring users to search multiple times to find the desired service. To improve the efficiency of finding business requests for users, AI technology can be introduced to assist them, enabling them to execute business requests through conversational interaction with an intelligent agent. However, intelligent agents cannot reuse the resources of existing pages on the business website, and their compatibility with the existing technical architecture and security system is poor. To enable the intelligent agent to have UI interaction capabilities, complex code tightly coupled with the front-end UI needs to be written for the AI ​​functions of the intelligent agent, requiring the development of a completely new interactive system adapted to the intelligent agent. All UI interaction-related functions need to be developed from scratch, resulting in high development difficulty and low efficiency. Furthermore, the strong coupling between the intelligent agent and the front-end UI makes the UI interaction based on the intelligent agent lack flexibility and reliability, making it difficult to support complex business scenarios.

[0016] This application provides a UI interaction method, device, equipment, system, and medium based on intelligent agents. It encapsulates the functions of a business website into a Web Model Context Protocol (WebMCP) service that can be perceived by intelligent agents. It extends the interfaces and data structures of the WebMCP service and its service information. The extended WebMCP service is implemented through an SDK loaded into the business website and the dialog window. This enables querying, monitoring, and execution of WebMCP services based on intelligent agents. It reuses the existing functions, authentication and authorization mechanisms, and security mechanisms of the browser loading the business website's pages. No additional adaptation or security control is required for the intelligent agent, and no new interactive system adapted to the intelligent agent needs to be developed. Through SDK settings and the extension of the WebMCP service interfaces and service information data structures, the intelligent agent, dialog window, and business website are decoupled and can collaborate in real time. This achieves intelligent agent-based UI interaction capabilities without changing the front-end development system, reducing the development difficulty of intelligent agent-based UI interaction functions, improving development efficiency, enhancing the efficiency and scenario adaptability of intelligent agent-based UI interaction, and ensuring system security and architectural scalability.

[0017] To facilitate understanding, a brief description of the agent-based UI interaction system provided in this application will be given first. Figure 1 A schematic diagram of an agent-based UI interaction system provided in an embodiment of this application is shown below. Figure 1 As shown, the agent-based UI interaction system may include a terminal device 11 and an agent 12.

[0018] Terminal device 11 has a target website 111 and a dialog window 112. The target website 111 can be accessed through a browser 113, and the browser 113 loads the pages of the target website 111. The target website 111 is a web site that provides business services. It can encapsulate the business functions of its pages into WebMCP services and complete the registration to provide existing resources to the dialog window 112 and the agent 12, so that the UI interaction based on the agent can reuse the existing resources of the target website 111.

[0019] Dialogue window 112 serves as the front-end window for user interaction with agent 12. It can discover WebMCP services, forward user input to agent 12, execute commands issued by agent 12, and provide feedback on the execution results. In some examples, the target website 111's page can be the main element, with dialogue window 112 embedded within it. In other examples, dialogue window 112 can be the main element, with the target website 111's page embedded within it. The target website 111's page and dialogue window 112 can also be displayed side-by-side; this is not a limitation.

[0020] Both target website 111 and dialog window 112 integrate a Software Development Kit (SDK). Target website 111 integrates SDK 114, which can also be considered as the SDK integrated by browser 113. Dialog window 112 integrates SDK 115. SDK 114 and SDK 115 have the same functionality, and SDK 114 and SDK 115 can communicate directly or indirectly, which is not limited here. SDK 114 and SDK 115 can be specifically implemented as JavaScript SDK, i.e., JSSDK, but are not limited to this. SDK 114 and SDK 115 can be lightweight SDKs that implement the protocol extension function of WebMCP service in the embodiments of this application. They are the core hub connecting target website 111, dialog window 112, and intelligent agent 12, and can provide the full-process application programming interface (API) and operating mechanism for registration, discovery, operation, and communication of WebMCP service. In this embodiment, SDK 114 integrated into target website 111 provides a standardized API for target website 111 to register and deregister WebMCP services; SDK 115 integrated into dialog window 112 provides a standardized API for dialog serial port to query and monitor WebMCP services; SDK 114 integrated into target website 111 and SDK 115 integrated into dialog window 112 also provide an execution mechanism for WebMCP services to support local and cross-page execution of WebMCP services; SDK 114 integrated into target website 111 and SDK 115 integrated into dialog window 112 also enable same-domain and cross-domain communication between target website 111 and dialog window 112.

[0021] The intelligent agent 12 can be an AI backend with intent recognition, service planning, and logical reasoning capabilities. For example, the intelligent agent 12 can be implemented through devices such as remote servers. The intelligent agent 12 can be used to parse user intents, match corresponding WebMCP services for user intents, issue execution instructions to the dialogue window 112, and plan and respond based on the execution results fed back by the dialogue window 112.

[0022] The specific details of the agent-based UI interaction methods executed by each part of the aforementioned agent-based UI interaction system will be described in detail below. The agent-based UI interaction methods, devices, equipment, media, and program products provided in this application will be described below.

[0023] This application provides a UI interaction method based on intelligent agents, which can be applied to scenarios where users execute business services by interacting with a dialog window and utilizing the functions of intelligent agents. This UI interaction method based on intelligent agents can be executed by intelligent agent-based UI interaction devices, terminal devices, systems, etc., and is not limited thereto. Figure 2 A flowchart of an agent-based UI interaction method provided in an embodiment of this application is shown below. Figure 2 As shown, the agent-based UI interaction method may include steps S201 to S203.

[0024] In step S201, when a user visits a target website, the system receives the user's business request prompt through a dialog window and forwards the prompt to the intelligent agent.

[0025] The target website integrates an SDK, and the dialog window also integrates an SDK. For details on the SDK's content, please refer to the relevant descriptions in the above embodiments; they will not be repeated here. When a user visits the target website, the terminal device can receive the user's service request prompts through the dialog window. These prompts reflect the user's intent; that is, the user can express the service they wish to perform through the prompts. The dialog window can forward the service request prompts to the intelligent agent, which then processes them based on the prompts.

[0026] In step S202, the basic information of the target WebMCP service is received from the agent through the dialog window.

[0027] The agent can infer the subtasks included in the user intent service indicated by the business request prompt based on the prompt. It then matches these subtasks with WebMCP services in a pre-synchronized list of WebMCP services and provides basic information about the matching WebMCP services to the dialog window. The target WebMCP service is the WebMCP service matched by the agent in the WebMCP service list based on the subtasks of the user intent service obtained from the business request prompt. The target WebMCP service includes WebMCP services capable of implementing the subtasks. The business request prompt includes information entered by the user in the dialog window requesting to process the service; this can include natural language information, as well as voice, images, forms, etc., and is not limited here. The WebMCP service list may include basic information about the WebMCP services supported by the target website. The WebMCP service list includes basic information on all WebMCP services of the target website. This list can be pre-configured on the backend agent via requests or backend configuration. This prevents the agent from only receiving basic information about WebMCP services for pages visited by the user, and from being unable to find the target WebMCP service if the page corresponding to the WebMCP service for a subtask has not yet been visited by the user. By pre-configuring the WebMCP service list, which includes basic information on all WebMCP services of the target website, on the backend agent, the agent can determine the target WebMCP service matching the subtask even if the page corresponding to the WebMCP service for a subtask has not yet been visited by the user.

[0028] In some examples, basic information may include, but is not limited to, service name, service description, and page network address; the service description is a natural language description of the WebMCP service, which can be recognized by the agent; the page network address can indicate the Uniform Resource Locator (URL) of the page corresponding to the WebMCP service, which can be used for page navigation. In some examples, subtasks are obtained by the agent decomposing the user intent business, and the target WebMCP service is obtained by the agent matching the subtasks with the service descriptions of WebMCP services in the WebMCP service list. The agent can infer the user intent task based on the business request prompt words, and decompose the user intent task into N subtasks, where N is a positive integer. For each subtask, the subtask is matched with the service description of the WebMCP service in the WebMCP service list, and the WebMCP service whose description information matches the subtask is identified as the target WebMCP service. For example, the subtask can be converted into a semantic vector, the service description of the WebMCP service can be converted into a semantic vector, and the similarity between the semantic vector of the subtask and the semantic vector of the service description can be calculated. The WebMCP service corresponding to the service description with the highest similarity above a preset similarity threshold can be identified as the target WebMCP service.

[0029] In some examples, if there is no target WebMCP service matching the subtask in the WebMCP service list, the agent can output natural language response information according to the normal dialogue logic, and feed the natural language response information back to the dialogue window, which then feeds it back to the user.

[0030] In step S203, the service execution interface is called through the SDK of the dialog window. Based on the page network address in the service information of the target WebMCP service, the service execution callback function in the service information of the target WebMCP service is executed to obtain the execution result.

[0031] The WebMCP service protocol declares the WebMCP service interface (API) and service information. The WebMCP service interface can be mounted under `navigator.modelContext`. In this embodiment, the WebMCP service interface includes not only basic service registration and deregistration interfaces, but also a service execution interface, used to execute service execution callback functions. The service execution interface can execute the corresponding WebMCP service based on its service name and return the execution result. For example, the service execution interface can be `navigator.modelContext.executeTool(name)`, where `name` is the service name. When this service execution interface is called with a given service name, the specified WebMCP service can be executed. The WebMCP service information may include, but is not limited to, the page network address and the service execution callback function. The explanation of the page network address can be found above and will not be repeated here. The service execution callback function can reuse the target website's business logic, UI rendering logic, etc., allowing reuse of existing functions and resources. During the execution of the WebMCP service, the service execution callback function can be executed to implement the corresponding webpage operation functions. The service information of the WebMCP service may include the basic information of the WebMCP service in the above embodiments. In some examples, the service information of the WebMCP service may be ModelContextTool.

[0032] The SDK in the dialog window can determine the target WebMCP service from among multiple registered WebMCP services based on the basic information of the target WebMCP service received from the agent. It then calls the service execution interface of the target WebMCP service. During this call, the SDK determines the page corresponding to the target WebMCP service based on the page's network address in the service information. After the page is loaded, the service execution callback function in the target WebMCP service's service information is executed. The execution result may include information about the execution of the service execution callback function, such as success, failure, or business feedback information, which is not limited here. The target WebMCP service runs on the browser side of the target website. It can directly share the target website's cookies, tokens, and other authentication information with the existing authentication and authorization mechanisms, and strictly adheres to the browser's same-origin policy, security sandbox, and other native security mechanisms. The agent does not need to independently handle hot authentication authorization and security management.

[0033] In some examples, if a user intent task includes N subtasks, where N is a positive integer greater than or equal to 2, and more than two of the N subtasks are sequentially executed (i.e., the execution of the (i+1)th subtask depends on the execution result of the ith subtask), then the service execution callback function of the target WebMCP service matched by the ith subtask can be executed. After obtaining the execution result, the execution result is sent to the agent through the dialog window. The agent infers the (i+1)th subtask based on the execution result of the ith subtask, determines the target WebMCP service matched by the (i+1)th subtask in the WebMCP service list, and feeds back the basic information of the target WebMCP service matched by the (i+1)th subtask to the dialog window. The SDK of the dialog window calls the service execution interface to execute the target WebMCP service and obtain the execution result.

[0034] In this embodiment, when a user visits a target website, a dialog window can receive the user's business request prompt and forward it to an intelligent agent. The intelligent agent then determines the sub-tasks included in the user's intent task indicated by the business request prompt, thereby identifying the target WebMCP service matching the sub-task. The intelligent agent feeds back the basic information of the target WebMCP service to the dialog window, enabling the dialog window's SDK to identify the target WebMCP service. It then calls the extended service execution interface of the target WebMCP service, loads the page corresponding to the target WebMCP service, executes the service execution callback function in the service information of the target WebMCP service, reuses the resources and functions of the page corresponding to the target WebMCP service on the target website, and implements the sub-task, thereby fulfilling the user's intent task. By implementing extended WebMCP services through an SDK loaded into the business website and dialog window, it is possible to reuse the existing functions of the business website without developing a new interaction system adapted to the intelligent agent or changing the front-end development architecture. This enables intelligent agent-based UI interaction capabilities, thereby reducing the development difficulty of intelligent agent-based UI interaction functions, improving development efficiency, enhancing the efficiency and scenario adaptability of intelligent agent-based UI interaction, and ensuring system security and architectural scalability. Moreover, in this embodiment, the target website, dialog window, and intelligent agent are independent of each other. A real-time decoupling and collaboration approach is used to achieve intelligent agent-based UI interaction, fully leveraging the technical advantages of each component, improving the flexibility and reliability of intelligent agent-based UI interaction, and supporting the UI interaction needs of demanding business scenarios.

[0035] In some embodiments, the operation to implement the subtask can be determined based on the page network address in the service information of the target WebMCP service, whether it is an operation on the current page or a cross-page operation, and then the corresponding operation can be executed. Specifically, it can be detected whether the page network address in the service information of the target WebMCP service matches the page network address of the currently loaded page on the target website; if the page network address in the service information matches the page network address of the currently loaded page on the target website, the service execution callback function in the service information of the target WebMCP service is executed; if the page network address in the service information does not match the page network address of the currently loaded page on the target website, navigation is performed to the page indicated by the page network address in the service information and the page is loaded, and the service execution callback function in the service information of the target WebMCP service is executed.

[0036] If the page network address in the service information of the target WebMCP service matches the page network address of the currently loaded page on the target website, it means that the page corresponding to the target WebMCP is the same page as the currently loaded page on the target website. Under the condition of the currently loaded page, the service execution callback function in the service information of the target WebMCP service can be executed to reuse the existing UI and business logic of the currently loaded page, so as to realize the web page operation function corresponding to the target WebMCP service executed on the currently loaded page.

[0037] If the page network address in the service information of the target WebMCP service does not match the page network address of the currently loaded page on the target website, it means that the page corresponding to the target WebMCP is a different page from the currently loaded page on the target website. The existing UI and business logic of the currently loaded page cannot support the target WebMCP service. It is necessary to navigate to the page indicated by the page network address in the service information of the target WebMCP service and load that page. After the page is loaded, execute the service execution callback function in the service information of the target WebMCP service to realize the web page operation function corresponding to the target WebMCP service on the loaded page.

[0038] In some embodiments, the service information of a WebMCP service may further include UI style attribute information. This UI style attribute information can indicate whether to hide or show one or more UI elements to adjust the UI style of the loaded page. Specifically, during the execution of the service execution callback function in the service information of the target WebMCP service, the UI style of the loaded page can be adjusted according to the UI style attribute information of the target WebMCP service to hide or show specified UI elements. A page may contain multiple functions, each with its corresponding UI elements. The target WebMCP service may only involve a portion of the UI elements on a page. In this case, UI elements unrelated to the target WebMCP service may interfere with the user, affecting the efficiency of the user in obtaining information from the page, thereby reducing the user experience. The UI style attribute information may include hidden element information or shown element information. Hidden element information indicates whether to hide specified UI elements, and shown element information indicates whether to show specified UI elements. For example, the data structure of UI style attribute information can be `{show: list of CSS selectors}` or `{hide: list of CSS selectors}`; `{show: list of CSS selectors}` represents the information for displayed elements, where the "CSS selector list" can be filled with the UI elements to be displayed; `{hide: list of CSS selectors}` represents the information for hidden elements, where the "CSS selector list" can be filled with the UI elements to be hidden. UI elements can include, but are not limited to, navigation bars, sidebars, footers, tooltips, switches, sliders, etc. When the UI style attribute information includes hidden element information, the displayed page hides the UI elements specified in the hidden element information, and the UI elements not hidden on the displayed page are UI elements related to the target WebMCP service. For example, UI style attribute information of `{hide: "nav", "side-bar", "footer"}` indicates that the navigation bar, sidebar, and footer are hidden. When the UI style attribute information includes displayed element information, the displayed page only displays the UI elements specified in the displayed element information, and the displayed page displays UI elements related to the target WebMCP service. By expanding the service information of WebMCP services and adding UI style attribute information, it is possible to personalize the UI style of the page, display only the core business functions of the business services required by the user, improve the immersive experience of the user's conversational interaction, and increase the efficiency of the user in obtaining information from the page.

[0039] In some embodiments, before receiving the user's business request prompt through the dialog window, the registration of WebMCP services can be completed before the user visits the target website, so that the SDK can call the WebMCP services subsequently. Specifically, when the user visits the target website, the SDK of the target website can call the global service registration interface to register global WebMCP services in batches, and save the service information of the global WebMCP services in the local storage of the browser loading the page of the target website, so that the SDK can obtain the global WebMCP services; the SDK of the target website can call the page service registration interface to register the page-level WebMCP services of the currently loaded page of the target website individually, and save the service information of the page-level WebMCP services in the local storage of the browser, so that the SDK can obtain the page-level WebMCP services.

[0040] WebMCP services can include global WebMCP services and page-level WebMCP services. Global WebMCP services are those common to all pages within the target website. For example, global WebMCP services may include, but are not limited to, services implementing login functionality, user information retrieval functionality, and basic query functionality. Global WebMCP services can be registered through a global service registration interface with batch registration capabilities. This interface can be mounted under `navigator.modelContext`. For example, the global service registration interface could be `navigator.modelContext.provideContext(options)`, where `options` is an object parameter that allows batch definition and replacement of tools that can be called by the agent. These tools constitute the service information of the WebMCP service, thus registering the global WebMCP service. The service information of global WebMCP services can be stored in the browser's local storage. Page-level WebMCP services are WebMCP services unique to individual pages within the target website. For example, page-level WebCMP services may include, but are not limited to, WebMCP services that implement institutional network access functions, WebMCP services that implement merchant account opening functions, and WebMCP services that implement specific business processing functions. For page-level WebMCP services, registration can be done through a page service registration interface with a separate registration function. The WebMCP service interface can include a page service registration interface, which can be mounted under `navigator.modelContext`. For example, the page service registration interface could be `navigator.modelContext.registerTool(tool)`, where `tool` is a definable tool that can be called by the agent. This tool represents the service information of the WebMCP service, used to register the page-level WebMCP service. The service information of the page-level WebMCP service can be stored in the browser's local storage.

[0041] In some embodiments, after a WebMCP service completes registration, the retrieval of all registered WebMCP services can be triggered when a user accesses the dialog window. The WebMCP service interface also includes a full service retrieval interface, which is an extended WebMCP service interface as described in this application embodiment. The full service retrieval interface is used to query all registered WebMCP services for a specified target website and return a list of all registered WebMCP services. The full service retrieval interface can be mounted under `navigator.modelContext`, for example, `navigator.modelContext.listTools(targetSite)`, where `targetSite` is the target website. Specifically, before receiving the user's business request prompt through the dialog window, after the WebMCP service completes registration, in response to the user's access to the dialog window, the full service retrieval interface is called through the dialog window's SDK to retrieve the registered WebMCP services; the service information of the retrieved registered WebMCP services is then synchronized to the intelligent agent. User access to the dialog window triggers a call to the full service retrieval interface to obtain all registered WebMCP services. The service information of these WebMCP services is then synchronized to the backend agent, making the agent aware of all registered WebMCP services. The target website and the dialog window may be on the same domain or cross-domain.

[0042] In some examples, when the dialog window and the target website are on the same domain, the full service retrieval interface is called to read the WebMCP service information from the browser's local storage. Since the dialog window and the target website are on the same domain, `targetSite` in the full service retrieval interface `navigator.modelContext.listTools(targetSite)` can be empty; that is, `targetSite` is not a required parameter, and the full service retrieval interface will naturally read the service information directly from the browser's local storage.

[0043] In other examples, when the dialog window and the target website are in different domains, the full service retrieval interface is called to obtain the WebMCP service information from the dialog window's SDK. The dialog window's SDK communicates with the target website's SDK through a bridging entity to obtain the WebMCP service information of the target website across domains. Since the dialog window and the target website are in different domains, `targetSite` in the full service retrieval interface `navigator.modelContext.listTools(targetSite)` cannot be empty; that is, the dialog window is a required parameter, and `targetSite` is the domain name of the target website. The dialog window's SDK and the target website's SDK need to communicate through a bridging entity that enables cross-domain communication to obtain the WebCMP service information of the target website across domains. The bridging entity can be a software entity, implemented as, but not limited to, a method (method, API, etc.). For example, the bridging entity may include a `postMessage` method, which allows asynchronous direct message sending between the dialog window and the target website in different domains, thus achieving cross-domain communication between the dialog window and the target website. The `postMessage` method is suitable for most web scenarios, has good compatibility, and low development difficulty and cost. The bridging entity can also be a system entity, which can be implemented as, but is not limited to, server processes, middleware, etc. For example, the bridging entity may include a reverse proxy component. The reverse proxy component can be configured in the backend system of the dialog window or the backend system of the target website. The reverse proxy component may include, but is not limited to, Nginx. The reverse proxy component can proxy the dialog port and the target website to the same domain name, realizing the same-domain communication effect, thereby realizing cross-domain communication between the dialog window and the target website. When using a reverse proxy component to realize cross-domain communication between the dialog window and the target website, the targetSite in the quantity service acquisition interface navigator.modelContext.listTools(targetSite) can be omitted.

[0044] In some embodiments, changes to the WebMCP service of the target website can be monitored in real time, thereby enabling timely synchronization of WebMCP service updates to the agent. The WebMCP service interface also includes a monitoring service feedback interface, which is an extended WebMCP service interface according to embodiments of this application. The monitoring service feedback interface is used to monitor changes to the WebMCP service of the target website. When changes occur, a callback is triggered, returning the latest full set of WebMCP service information. The monitoring service feedback interface can be mounted under `navigator.modelContext`. For example, the monitoring service feedback interface can be `navigator.modelContext.onToolsChange(targetSite)`, where `targetSite` is the target website. Specifically, changes to the WebMCP service of the target website can be monitored by calling the monitoring service feedback interface through the dialog window's SDK. When changes occur to the WebMCP service of the target website, the service information of the WebMCP service of the target website is obtained through the dialog window's SDK and synchronized to the agent. Changes to the WebMCP service of the target website may include, but are not limited to, adding, deregistering, or modifying WebMCP services. By monitoring the service feedback interface, when the WebMCP services of the target website change, the dialog window can obtain the latest full set of WebMCP service information and synchronize it to the agent in real time, thereby ensuring that the service information of the agent's WebMCP services is consistent with the service information of the actual WebMCP services on the target website.

[0045] In some embodiments, after receiving the execution result in the dialog window, the dialog window can interact with the agent, utilizing the agent's context of user interaction to perform subsequent business planning, summarization, etc. Specifically, the dialog window can provide feedback on the execution result to the agent, enabling the agent to generate business execution response information based on the execution result; the dialog window can also receive business execution response information. The agent can utilize its domain knowledge based on the execution result to generate business execution response information, which can be in natural language form. If the execution result indicates that the user's intention task was successfully executed, the business execution response information generated by the agent can include the result of the user's intention task execution. If the execution result indicates that the user's intention task failed, the business execution response information generated by the agent can include the reason for the failure and a solution. The business execution response information can also include guidance information, which can prompt the user to supplement input parameter information so that the dialog window, target website, SDK, and agent can re-execute the agent-based UI interaction method according to the user's supplemented input parameter information.

[0046] As can be seen from the above embodiments, based on the original API of the W3C WebMCP service, this application extends the core interfaces such as the service execution interface, the full service acquisition interface, and the service monitoring feedback interface, which can realize the query, monitoring, and execution of WebMCP services. The extended core interfaces can be mounted under navigator.modelContext along with the original WebMCP API, further reducing the development difficulty and cost. The basic service declaration of the service information of the WebMCP service is the ModelContextTool dictionary. The extension of the service information of the WebMCP service in this application embodiment is the extension of the ModelContextTool dictionary. The extended service information of the WebMCP service in this application embodiment may include the service name (DOMString name), the service description (DOMString description), the service input parameter structure (inputSchema), the service execution callback function (ToolExecuteCallback execute), the service annotations (ToolAnnotations annotations), the page network address (DOMString url), and the UI style attribute information (initStyle). Among them, the page network address and the UI style attribute information are newly added service information after the extension.

[0047] The agent-based UI interaction method in this application embodiment may include five stages: service registration, service discovery and monitoring, user interaction and service matching, service execution, result feedback and subsequent planning. These five stages are described below with reference to the accompanying drawings.

[0048] Figure 3 A flowchart illustrating an example of the service registration phase provided in this application embodiment is shown below. Figure 3 As shown, the service registration phase may include steps a1 to a3.

[0049] In step a1, the target website receives user access. The target website has already loaded the SDK.

[0050] In step a2, the target website uses the SDK to call the global service registration interface to register global WebMCP services in batches.

[0051] In step a3, the target website registers a page-level WebMCP service separately by calling the page service registration interface through the SDK.

[0052] The specific details of steps a1 to a3 above can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0053] Figure 4A flowchart illustrating an example of the service discovery and monitoring phase provided in an embodiment of this application is shown below. Figure 4 As shown, the service discovery and monitoring phase may include steps a4 to a8.

[0054] In step a4, the dialog window receives user access. The dialog window has already loaded the SDK.

[0055] In step a5, the dialog window uses the SDK to call the full service retrieval interface to query and retrieve all WebMCP services of the target website. Retrieving all WebMCP services of the target website can include retrieving WebMCP services within the same domain as the dialog window and WebMCP services across different domains.

[0056] In step a6, the dialog window sends the service information of the queried WebCMP service to the agent.

[0057] In step a7, the dialog window uses the SDK to call the monitoring service feedback interface to monitor changes in the target website's WebMCP service. If the target website's WebMCP service changes, a callback function is triggered to retrieve the latest full set of WebMCP services from the target website.

[0058] In step a8, the dialog window sends the updated full WebMCP service information to the agent.

[0059] The specific details of steps a4 to a8 above can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0060] Figure 5 A flowchart illustrating an example of the user interaction and service matching stage provided in an embodiment of this application is shown below. Figure 5 As shown, the user interaction and service matching stage may include steps a9 to a14.

[0061] In step a9, the dialog window receives the user's service request prompt.

[0062] In step a10, the dialog window sends a service request prompt to the agent.

[0063] In step a11, the agent performs intent recognition, business planning, and service matching. Intent recognition allows for the reasoning of intent-based business processes; business planning decomposes user intent-based business processes into sub-tasks; and service matching matches sub-tasks with WebMCP services.

[0064] In step a12, the agent determines whether a WebMCP service matches the subtask. If it does, step a13 is executed; otherwise, step a14 is executed.

[0065] In step a13, the agent sends basic information and execution instructions for the target WebMCP service to the dialog window.

[0066] In step a14, the agent sends a natural language response to the dialog window so that the dialog window can send a natural language response to the user.

[0067] The specific details of steps a9 to a14 above can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0068] Figure 6 A flowchart illustrating an example of the service execution phase provided in an embodiment of this application is shown below. Figure 6 As shown, the service execution phase may include steps a15 to a21.

[0069] In step a15, the dialog window responds to the execution command sent by the agent and executes the target WebMCP service by calling the service execution interface through the SDK.

[0070] In step a16, the SDK determines whether the web address of the target WebMCP service matches the web address of the currently loaded page on the target website; if they match, step a17 is executed; if they do not match, step a18 is executed.

[0071] In step a17, the SDK directly executes the service execution callback function of the target WebMCP service on the currently loaded page of the target website.

[0072] In step a18, the SDK controls the target website to navigate to the page corresponding to the web address of the target WebMCP service, and executes the service execution callback function of the target WebMCP service.

[0073] In step a19, the target website sends the execution result back to the SDK.

[0074] In step a20, the SDK sends the execution result back to the dialog window.

[0075] In step a21, the dialog window provides feedback on the execution result to the agent.

[0076] The specific details of steps a15 to a21 above can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0077] Figure 7 A flowchart illustrating an example of result feedback and subsequent planning provided in an embodiment of this application is shown below. Figure 7 As shown, the feedback of this result and subsequent planning phases may include steps a22 and a23.

[0078] In step a22, the agent performs subsequent business planning or generates business execution response information based on the execution result. The agent can perform subsequent business planning or output business execution response information based on the execution result and the context of user interaction.

[0079] In step a23, the agent sends a response to the business execution request to the dialog window.

[0080] The specific details of steps a22 and a23 above can be found in the relevant descriptions in the above embodiments, and will not be repeated here.

[0081] The agent-based UI interaction method provided in this application can directly reuse the existing functions, code, front-end components, business logic, and other resources of the target website. It eliminates the need to redevelop agent-based UI interaction and business functions from scratch, achieving 100% reuse of the target website's existing resources and fundamentally reducing development difficulty and costs. WebMCP services are essentially JavaScript functions with natural language descriptions and structured patterns. In this application embodiment, front-end development only requires exposing business functions in WebMCP form on top of the existing code. There is no need to change existing components, user-dialogue window interaction methods, or communication methods. The agent can plan the WebMCP service according to the logic of conventional MCP planning, without altering the existing front-end development system or requiring additional agent adaptation. This reduces the development difficulty of agent-based UI interaction and improves development efficiency. The WebMCP service in this embodiment runs on the browser side of the target website, directly sharing the target website's authentication information and existing authentication and authorization mechanisms. It strictly adheres to the browser's same-origin policy, security sandbox, and other native security mechanisms. This allows the intelligent agent to leverage the browser's robust security features without requiring independent authentication, authorization, and security management, further reducing the development difficulty of agent-based UI interactions and improving development efficiency. The target website is responsible for encapsulating and registering the WebMCP service, the dialog window is responsible for discovering and communicating the WebMCP service, and the backend intelligent agent is responsible for business planning and service matching. The target website, dialog window, and intelligent agent are decoupled and collaborate, fully leveraging their respective technical advantages to enhance the flexibility and reliability of agent-based UI interactions. The agent-based UI interaction in this embodiment is compatible with both same-domain and cross-domain scenarios between the target website and the dialog window. This enables fine-grained management of both global and page-level WebMCP services. Personalized control of the page's UI style is achieved through UI style attributes in the WebMCP service's service information, showcasing core business functions and enhancing the immersive user experience during dialogue. If new business functions are added, they only need to be encapsulated as a new WebMCP registration function to be used, without modifying the existing framework, thus improving adaptability and scalability.

[0082] This application also provides a UI interaction device based on an intelligent agent, which corresponds to the UI interaction method based on an intelligent agent in the above embodiments. Figure 8 This is a schematic diagram of the structure of a UI interaction device based on an intelligent agent provided in an embodiment of this application, as shown below. Figure 8 As shown, the agent-based UI interaction device 300 may include an interaction module 301, a communication module 302, and an execution module 303.

[0083] The interaction module 301 can be used to receive the user's business request prompts through a dialog window when the user visits the target website, and forward the business request prompts to the intelligent agent. The target website has an integrated SDK, and the dialog window has an integrated SDK.

[0084] The communication module 302 can be used to receive basic information of the target WebMCP service from the agent through the dialog window. The target WebMCP service is the WebMCP service that the agent matches in the WebMCP service list for the sub-tasks contained in the user intent business obtained by the agent based on the business request prompt. The interface of the WebMCP service includes a service execution interface, which is used to execute the service execution callback function. The service information of the WebMCP service includes the page network address and the service execution callback function.

[0085] The execution module 303 can be used to call the service execution interface through the SDK of the dialog window, execute the service execution callback function in the service information of the target WebMCP service according to the page network address in the service information of the target WebMCP service, and obtain the execution result.

[0086] In some embodiments, the execution module 303 may be used to: detect whether the page network address in the service information of the target WebMCP service matches the page network address of the currently loaded page of the target website; if the page network address in the service information matches the page network address of the currently loaded page of the target website, execute the service execution callback function in the service information of the target WebMCP service; if the page network address in the service information does not match the page network address of the currently loaded page of the target website, navigate to the page indicated by the page network address in the service information and load it, and execute the service execution callback function in the service information of the target WebMCP service.

[0087] In some embodiments, the service information of the WebMCP service also includes UI style attribute information. The execution module 303 can also be used, during the execution of the service execution callback function in the service information of the target WebMCP service, to adjust the UI style of the loaded page according to the UI style attribute information of the target WebMCP service, so as to hide or show specified UI elements.

[0088] In some embodiments, the WebMCP service list includes basic information about the WebMCP services supported by the target website, including service name, service description, and page web address. Subtasks are obtained by the agent decomposing the user's intent, and the target WebMCP service is obtained by the agent matching the subtasks with the service descriptions of the WebMCP services in the WebMCP service list.

[0089] In some embodiments, the WebMCP service includes a global WebMCP service and a page-level WebMCP service. The execution module 303 can also be used to: before receiving the user's business request prompt through the dialog window, when the user visits the target website, call the global service registration interface through the target website's SDK to register global WebMCP services in batches, and save the service information of the global WebMCP services in the local storage of the browser loading the page of the target website, so that the SDK can obtain the global WebMCP services; call the page service registration interface through the target website's SDK to register the page-level WebMCP services of the currently loaded page of the target website individually, and save the service information of the page-level WebMCP services in the local storage of the browser, so that the SDK can obtain the page-level WebMCP services.

[0090] In some embodiments, the WebMCP service interface further includes a full service acquisition interface. The execution module 303 can also be used to, in response to the user's access to the dialog window, call the full service acquisition interface through the dialog window's SDK to acquire the registered WebMCP services before receiving the user's service request prompt through the dialog window. The communication module 302 can also be used to synchronize the acquired service information of the registered WebMCP services to the intelligent agent.

[0091] In some examples, execution module 303 can be used to: when the dialog window and the target website are in the same domain, call the full service retrieval interface to read the service information of the WebMCP service from the browser's local storage; when the dialog window and the target website are in different domains, call the full service retrieval interface to retrieve the service information of the WebMCP service from the dialog window's SDK, and the dialog window's SDK communicates with the target website's SDK through a bridging entity to retrieve the service information of the cross-domain target website's WebMCP service.

[0092] In some examples, the bridging entity may include the postMessage method or a reverse proxy component.

[0093] In some embodiments, the WebMCP service interface further includes a monitoring service feedback interface. The execution module 303 can be used to: monitor changes in the target website's WebMCP service by calling the monitoring service feedback interface through the dialog window's SDK; and, when changes occur in the target website's WebMCP service, obtain service information about the target website's WebMCP service through the dialog window's SDK. The communication module 302 can be used to: synchronize the service information of the target website's WebMCP service to the intelligent agent.

[0094] In some embodiments, the communication module 302 can be used to: call the service execution interface of the target WebMCP service through the SDK of the dialog window, execute the service execution callback function in the target WebMCP service according to the page network address of the target WebMCP service, and after obtaining the execution result, feed back the execution result to the agent through the dialog window so that the agent can generate business execution response information based on the execution result; and receive business execution response information through the dialog window.

[0095] It should be noted that the agent-based UI interaction device 300 is a device corresponding to the agent-based UI interaction method described above. All implementation methods in the above method embodiments are applicable to the embodiments of this device and can achieve the same technical effect.

[0096] This application also provides a terminal device. Figure 9 This is a schematic diagram of the structure of a terminal device provided in an embodiment of this application, as shown below. Figure 9 As shown, the terminal device 400 includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

[0097] In some examples, the processor 402 described above may include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits that may be configured to implement the embodiments of this application.

[0098] Memory 401 may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform operations described with reference to the agent-based UI interaction method according to embodiments of this application.

[0099] The processor 402 runs a computer program corresponding to the executable program code by reading the executable program code stored in the memory 401, so as to implement the agent-based UI interaction method in the above embodiments.

[0100] In some examples, terminal device 400 may also include communication interface 403 and bus 404. For example, Figure 9 As shown, the memory 401, processor 402, and communication interface 403 are connected through bus 404 and complete communication with each other.

[0101] The communication interface 403 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application. Input devices and / or output devices can also be connected through the communication interface 403.

[0102] Bus 404 includes hardware, software, or both, that couples the components of terminal device 400 together. For example, and not limitingly, bus 404 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Where appropriate, bus 404 may include one or more buses. Although specific buses are described and illustrated in the embodiments of this application, this application considers any suitable bus or interconnection.

[0103] This application also provides a UI interaction system based on an intelligent agent. This UI interaction system may include a terminal device and an intelligent agent. The terminal device can execute the UI interaction method based on the intelligent agent described in the above embodiments. The intelligent agent is used to infer the sub-tasks included in the user intent service indicated by the service request prompts sent by the terminal device, determine the target WebMCP service matching the sub-tasks in a pre-synchronized WebMCP service list, and send the basic information of the target WebMCP service to the terminal device.

[0104] For details regarding the terminal device and the intelligent agent, please refer to the relevant descriptions of the intelligent agent-based UI interaction system and the intelligent agent-based UI interaction method in the above embodiments, which will not be repeated here.

[0105] This application also provides a computer-readable storage medium storing computer program instructions. When these computer program instructions are executed by a processor, they can implement the UI interaction method of the intelligent agent in the above embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The aforementioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., and is not limited thereto.

[0106] This application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the UI interaction method of the intelligent agent in the above embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0107] It should be clarified that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. For the device embodiments, equipment embodiments, system embodiments, computer-readable storage medium embodiments, and computer program product embodiments, the relevant parts can be referred to the description section of the method embodiments. This application is not limited to the specific steps and structures described above and shown in the figures. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application. Furthermore, for the sake of brevity, detailed descriptions of known methods and techniques are omitted here.

[0108] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0109] Those skilled in the art will understand that the above embodiments are exemplary and not restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on a study of the drawings, specification, and claims, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the claims, the term "comprising" does not exclude other means or steps; the quantifier "a" does not exclude a plurality; the terms "first" and "second" are used to identify names and not to indicate any particular order. No reference numerals in the claims should be construed as limiting the scope of protection. The functionality of multiple parts appearing in the claims can be implemented by a single hardware or software module. The appearance of certain technical features in different dependent claims does not mean that these technical features cannot be combined to achieve beneficial effects.

Claims

1. A UI interaction method based on intelligent agents, characterized in that, include: When a user visits a target website, the system receives the user's business request prompt through a dialog window and forwards the business request prompt to the intelligent agent. The target website integrates an SDK, and the dialog window also integrates an SDK. The dialog window receives basic information about the target WebMCP service from the agent. The target WebMCP service is the WebMCP service that the agent matches in the WebMCP service list for the subtasks included in the user intent service obtained by the agent based on the business request prompt. The interface of the WebMCP service includes a service execution interface, which is used to execute the service execution callback function. The service information of the WebMCP service includes the page network address and the service execution callback function. The service execution interface is called through the SDK of the dialog window. Based on the page network address in the service information of the target WebMCP service, the service execution callback function in the service information of the target WebMCP service is executed to obtain the execution result. The step of executing the service execution callback function in the service information of the target WebMCP service based on the page network address in the service information of the target WebMCP service, and obtaining the execution result, includes: Detect whether the page network address in the service information of the target WebMCP service matches the page network address of the currently loaded page on the target website; If the page network address in the service information matches the page network address of the currently loaded page on the target website, execute the service execution callback function in the service information of the target WebMCP service; If the page network address in the service information does not match the page network address of the currently loaded page on the target website, navigate to the page indicated by the page network address in the service information and load it, and execute the service execution callback function in the service information of the target WebMCP service; The WebMCP service interface also includes a full service acquisition interface; before receiving the user's business request prompt through the dialog window, it also includes: In response to a user's access to the dialog window, the registered WebMCP services are obtained by calling the full service acquisition interface through the dialog window's SDK. The service information of the registered WebMCP services obtained will be synchronized to the intelligent agent; The step of obtaining registered WebMCP services by calling the full service acquisition interface through the SDK in the dialog window includes: In the case where the dialog window and the target website are in cross-domain, the full service acquisition interface is called to obtain the service information of the WebMCP service from the SDK of the dialog window. The SDK of the dialog window communicates with the SDK of the target website through the bridging entity to obtain the service information of the WebMCP service of the target website in the cross-domain.

2. The method according to claim 1, characterized in that, WebMCP service information also includes UI style attribute information; The process of executing the service execution callback function in the service information of the target WebMCP service also includes: Based on the UI style attribute information of the target WebMCP service, adjust the UI style of the loaded page to hide or show specified UI elements.

3. The method according to claim 1, characterized in that, The WebMCP service list includes basic information about the WebMCP services supported by the target website, including service name, service description, and page web address. The subtask is obtained by the agent decomposing the user intent service, and the target WebMCP service is obtained by the agent matching the subtask with the service description of the WebMCP service in the WebMCP service list.

4. The method according to claim 1, characterized in that, WebMCP services include global WebMCP services and page-level WebMCP services; Before receiving the user's service request prompt through the dialog window, the following is also included: When a user visits the target website, the SDK of the target website calls the global service registration interface to register global WebMCP services in batches. The service information of the global WebMCP services is stored in the local storage of the browser that loads the page of the target website, so that the SDK can obtain the global WebMCP services. By calling the page service registration interface through the SDK of the target website, the page-level WebMCP service of the currently loaded page of the target website is registered separately, and the service information of the page-level WebMCP service is saved in the local storage of the browser so that the SDK can obtain the page-level WebMCP service.

5. The method according to claim 1, characterized in that, The step of obtaining registered WebMCP services by calling the full service acquisition interface through the SDK in the dialog window also includes: If the dialog window is in the same domain as the target website, the full service retrieval interface is invoked to read the service information of the WebMCP service from the browser's local storage.

6. The method according to claim 1, characterized in that, The bridging entity includes the postMessage method or a reverse proxy component.

7. The method according to claim 1, characterized in that, The WebMCP service interface also includes a monitoring service feedback interface; The method further includes: The SDK in the dialog window is used to call the monitoring service feedback interface to monitor changes in the WebMCP service of the target website; If the WebMCP service of the target website changes, the service information of the WebMCP service of the target website is obtained through the SDK of the dialog window and synchronized to the intelligent agent.

8. The method according to claim 1, characterized in that, After the SDK in the dialog window calls the service execution interface of the target WebMCP service, executes the service execution callback function in the target WebMCP service according to the page network address of the target WebMCP service, and obtains the execution result, the process further includes: The execution result is fed back to the intelligent agent through the dialogue window, so that the intelligent agent can generate business execution response information based on the execution result; Receive the business execution response information through the dialog window.

9. A UI interaction device based on intelligent agents, characterized in that, include: An interaction module is used to receive a user's business request prompt through a dialog window when the user visits a target website, and forward the business request prompt to the intelligent agent. The target website integrates an SDK, and the dialog window integrates an SDK. The communication module is used to receive basic information of the target WebMCP service fed back by the agent through the dialog window. The target WebMCP service is the WebMCP service matched by the agent in the WebMCP service list based on the sub-tasks of the user intent service obtained by the agent according to the service request prompt words. The interface of the WebMCP service includes a service execution interface, which is used to execute the service execution callback function. The service information of the WebMCP service includes the page network address and the service execution callback function. The execution module is used to call the service execution interface through the SDK of the dialog window, execute the service execution callback function in the service information of the target WebMCP service according to the page network address in the service information of the target WebMCP service, and obtain the execution result; The execution module is configured to: detect whether the page network address in the service information of the target WebMCP service matches the page network address of the currently loaded page of the target website; if the page network address in the service information matches the page network address of the currently loaded page of the target website, execute the service execution callback function in the service information of the target WebMCP service; if the page network address in the service information does not match the page network address of the currently loaded page of the target website, navigate to the page indicated by the page network address in the service information and load it, and execute the service execution callback function in the service information of the target WebMCP service. The WebMCP service interface also includes a full service acquisition interface; the execution module is further configured to: before receiving the user's business request prompt through the dialog window, in response to the user's access to the dialog window, call the full service acquisition interface through the dialog window's SDK to obtain the registered WebMCP service. The communication module is also used to: synchronize the service information of the acquired registered WebMCP services to the intelligent agent; The execution module is configured to: in the case that the dialog window and the target website are in cross-domain, call the full service acquisition interface to obtain the service information of the WebMCP service from the SDK of the dialog window, and the SDK of the dialog window communicates with the SDK of the target website through the bridging entity to obtain the service information of the WebMCP service of the target website in the cross-domain.

10. A terminal device, characterized in that, include: Processor and memory storing computer program instructions; When the processor executes the computer program instructions, it implements the agent-based UI interaction method as described in any one of claims 1 to 8.

11. A UI interaction system based on intelligent agents, characterized in that, include: The terminal device as described in claim 10; An intelligent agent is used to infer the sub-tasks included in the user intent service indicated by the service request prompt words sent by the terminal device, determine the target WebMCP service that matches the sub-tasks in the pre-synchronized WebMCP service list, and send the basic information of the target WebMCP service to the terminal device.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the agent-based UI interaction method as described in any one of claims 1 to 8.

13. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the agent-based UI interaction method according to any one of claims 1 to 8.

Images