Page interaction method, electronic equipment, storage medium and program product

Abstract

The invention provides a page interaction method, electronic equipment, a storage medium and a program product, and belongs to the technical field of artificial intelligence. The method comprises the following steps: identifying a control intention from a control statement of a target account; based on the control intention, target metadata is obtained from a webpage tool database, the target metadata is used for indicating field formats of different fields in a target function corresponding to the target function name, and the target function is an execution function of the target function component; the webpage tool database comprises metadata of different functional components on an application page and function names of execution functions of the metadata; based on the target metadata, performing format conversion on the control intention to obtain a target parameter; and sending a control instruction to the client so as to control the application page. According to the method, the control statement of the target account can be converted into the control instruction which can be executed by the client, so that the application page is driven based on the control instruction, and the interaction experience of a user is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a page interaction method, electronic device, storage medium, and program product. Background Technology

[0002] In recent years, with the development of artificial intelligence technology, users expect to have a more natural, intelligent, and personalized interactive experience when interacting with application pages. For example, they want to be able to directly complete complex tasks such as searching, comparing, filtering, and even combining products through natural language commands.

[0003] However, the front-end architecture of mainstream e-commerce platforms and other information display and service delivery platforms has undergone long-term iteration and development, forming a highly stable and complex technical system. These front-end systems are usually based on fixed programming paradigms, and their page structure, interaction flow, and data logic are deeply coupled.

[0004] Therefore, how to drive page interaction based on users' natural language commands has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a page interaction method, electronic device, storage medium, and program product that can convert the control statements of a target account into control commands executable by the client, thereby driving the application page based on the control commands and improving the user's interactive experience. The technical solution is as follows: Firstly, a page interaction method is provided, the method comprising: Identify the control intent from the control statements of the target account, wherein the control statements are the statements entered by the target account when interacting with the intelligent agent on the application page; Based on the control intent, target metadata is obtained from the web tool database. The target metadata is the metadata of the target functional component related to the control intent on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. Based on the target metadata, the manipulation intent is format-converted to obtain target parameters; A control command is sent to the client. The control command includes the target function name and the target parameters. The control command is used to instruct the client to manipulate the application page based on the target function name and the target parameters.

[0006] Secondly, a page interaction method is provided, the method comprising: Display an application page, the application page including a first area for interacting with the intelligent agent; Obtain the control statements entered by the target account in the first area; The control statement is sent to the server. The control statement is used by the server to obtain target metadata from the web tool database based on the control intent identified from the control statement. Based on the target metadata, the control intent is format-converted to obtain target parameters. The target metadata is the metadata of the target functional component related to the control intent on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. In response to the control commands sent by the server, the application page is manipulated based on the target function name and the target parameters.

[0007] Thirdly, a page interaction device is provided, the device comprising: The identification module is used to identify the control intent from the control statements of the target account, wherein the control statements are the statements entered by the target account when interacting with the intelligent agent on the application page; The first acquisition module is used to acquire target metadata from the web tool database based on the control intention. The target metadata is the metadata of the target functional component related to the control intention on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. The conversion module is used to convert the control intent into a format based on the target metadata to obtain target parameters; The sending module is used to send control instructions to the client. The control instructions include the target function name and the target parameters. The control instructions are used to instruct the client to manipulate the application page based on the target function name and the target parameters.

[0008] Fourthly, a page interaction device is provided, the device comprising: A first display module is used to display an application page, the application page including a first area for interacting with the intelligent agent; The acquisition module is used to acquire the control statements entered by the target account in the first area; A sending module is used to send the control statement to the server. The control statement is used by the server to obtain target metadata from the web tool database based on the control intent identified from the control statement. Based on the target metadata, the server performs format conversion on the control intent to obtain target parameters. The target metadata is the metadata of the target functional component related to the control intent on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. The control module is used to respond to control commands sent by the server and control the application page based on the target function name and the target parameters.

[0009] Fifthly, an electronic device is provided, including a processor and a memory; the memory stores at least one piece of program code; the at least one piece of program code is used to be called and executed by the processor to implement the page interaction method described in the first aspect, or the page interaction method described in the second aspect.

[0010] In a sixth aspect, a computer-readable storage medium is provided, wherein at least one computer program is stored therein, and the at least one computer program, when executed by a processor, is capable of implementing the page interaction method described in the first aspect or the page interaction method described in the second aspect.

[0011] In a seventh aspect, a computer program product is provided, the computer program product comprising a computer program, which, when executed by a processor, is capable of implementing the page interaction method described in the first aspect, or the page interaction method described in the second aspect.

[0012] The beneficial effects of the technical solutions provided in this application are: The page interaction method provided in this application embodiment pre-constructs a webpage tool database, which stores metadata of multiple functional components on the application page. The metadata of each functional component indicates the field format of different fields in the execution function of the corresponding functional component. When a control statement input by the target account is obtained, intent recognition is performed on the control statement to obtain the target account's control intent. Then, based on this control intent, target metadata of the target functional component related to the control intent is retrieved from the webpage tool database. Since the target metadata indicates the execution function of the target functional component, i.e., the field format of different fields in the target function, target parameters can be obtained by format conversion of the target account's control intent based on the target metadata. These target parameters are the function parameters of the target function and are input into the target function as input parameters, enabling the target function to execute. Based on the target function name and target parameters indicated in the target metadata, a control command is generated and then sent to the client providing the application page. After receiving the control command, the client calls the target function corresponding to the target function name and inputs the target parameters into the target function to obtain an executable function. By executing the executable function, the application page can be manipulated, thereby realizing the driving of the application page by natural language commands and improving the user's interactive experience. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of the implementation environment involved in a page interaction method provided in an embodiment of this application; Figure 2 This is a flowchart of a page interaction method provided in an embodiment of this application; Figure 3 This is a flowchart of a page interaction method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating a method for constructing a webpage tool database according to an embodiment of this application; Figure 5 This is a schematic diagram of an application page provided in an embodiment of this application; Figure 6 This is a flowchart of a page interaction method provided in an embodiment of this application; Figure 7 This is a flowchart of a page interaction method provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a page interaction device provided in an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a page interaction device provided in an embodiment of this application; Figure 10 A structural block diagram of an electronic device provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0016] It is understood that the terms "each," "multiple," and "any" used in the embodiments of this application, etc., mean that "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the corresponding multiples. For example, multiple words include 10 words, and "each word" refers to each of the 10 words, while "any word" refers to any one of the 10 words.

[0017] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0018] Before implementing the embodiments of this application, the names involved in the embodiments of this application will be explained first.

[0019] WebTool is a standardized front-end interaction tool protocol that allows AI (artificial intelligence) agents to directly drive the UI (User Interface) state and interactive behavior of application pages through declarative instructions.

[0020] A deterministic page is a web page built using the traditional symbolic programming paradigm, in which the interaction logic and UI state changes are predictable, reproducible, and reliable.

[0021] An intelligent agent is an intelligent decision-making unit built on a large language model that can understand user intent and generate structured tool invocation instructions.

[0022] A format converter is an intelligent data adaptation engine that can automatically map and convert raw data into standard parameters that conform to the target WebTool input protocol (such as JSON Schema).

[0023] A schema generator is a development aid that can automatically generate a JSON schema with Chinese field descriptions based on TypeScript, JavaScript, or JSON examples provided by the developer.

[0024] JSON Schema is used to describe the format of JSON data structures. Simply put, it's a set of rules that specifies how JSON data should be organized and validated, thereby ensuring that the data conforms to specific formats and requirements.

[0025] With the development of artificial intelligence technology, especially the breakthroughs in natural language understanding, generation, and reasoning capabilities of large language models, AI-native applications centered on intelligent dialogue and intent understanding have become an important development direction for the next generation of human-computer interaction. These applications, driven by large language models, emphasize efficient, intelligent, and context-aware user interaction through natural language, and are widely used in customer service, content generation, and intelligent assistants. Against this backdrop, deeply integrating large language models into e-commerce platforms to build AI-driven shopping interfaces with intelligent shopping guides, dynamic recommendations, semantic search, and natural language interaction capabilities has become a key path to improving user experience and conversion efficiency.

[0026] However, the front-end pages of existing e-commerce platforms and some information display and service provision platforms are deterministic pages. Their page structure is fixed during the development phase, following strict programming paradigms and predefined user flows. While this architecture is highly stable and maintainable, it is essentially a static reactive design. This means that user actions must strictly match pre-defined event handling mechanisms and DOM (Document Object Model) update logic, making it difficult to support dynamic, adaptive, and context-sensitive interaction modes driven by large language models. To improve the user experience, the following methods are currently used: The first approach involves building entirely new AI-native applications from scratch. However, this method requires repeatedly rebuilding core assets such as users, products, and transactions. Existing e-commerce platforms and some information display and service providers have complex services, large user bases, and high compatibility requirements, resulting in high reconstruction costs. Furthermore, due to the inherent illusion problem of large language models, key operations (such as filtering and order placement) in the reconstructed AI-native applications lack reliability, leading to a poor user experience.

[0027] The second approach involves calling backend tools of the application based on MCP (ModelContext Protocol). While protocols like MCP allow large language models to call predefined backend tools (such as APIs) to obtain information or perform operations, this method does not involve the frontend UI layer. The large language model cannot perceive or change the visual state and interaction flow of the page. Even if the called backend tool operates successfully, the frontend still needs to passively update the UI through polling or event listening, failing to achieve direct and real-time AI-driven UI.

[0028] The third approach uses RPA (Robotic Process Automation) to simulate user actions (such as mouse clicks and keyboard input) to automate UI interactions. However, this method heavily relies on XPath (Xml Path Language) or CSS (Cascading Style Sheets) selectors for UI elements, making the scripts prone to failure if the page is redesigned. Furthermore, this method is based on coordinates or element IDs (identity markers) rather than service-related semantics, making it difficult to align with the intent of a large language model. Additionally, this method typically requires the installation of additional proxy software on the client side, making it complex to operate.

[0029] The fourth method involves embedding a separate AI chat box within the application page to answer user questions. However, this chat box operates independently of the application page's context, requiring users to switch between the two, resulting in complex operations and a poor user experience. Furthermore, this AI chat box can only perform question-and-answer interactions and cannot proactively alter the application page's logic or state.

[0030] In summary, the relevant technical solutions are either too costly, have limited capabilities, or lack front-end control, and none of them can solve the problem of the disconnect between the large language model and the application page.

[0031] To address the problems existing in related technical solutions, this application provides a page interaction method. This method inputs TypeScript, JavaScript, or JSON examples of functional components (such as filtering and refresh components) of the front-end page into a schema generator, automatically generating a standardized schema data JSON Schema. This schema data JSON Schema is also the metadata of the WebTool tool described in subsequent embodiments. It can be considered an entity that can be called by the large language model, solving the problem of the large language model's inability to communicate with the application page. Based on this, a format converter is used to convert the user's input natural language commands into function parameters of the function to be called, eliminating tedious manual adaptation work and providing a communication foundation for natural language commands to drive the application page. In short, this application uses the large language model as the "intelligent brain" and the page as the "reliable limbs." Through protocol decoupling, it achieves deep coupling between the large language model and the application page, improving the flexibility of the large language model while ensuring service reliability.

[0032] The method in this application embodiment can drive application pages with natural language commands without rebuilding a new AI-native application, saving reconstruction costs. Furthermore, the large language model can directly manipulate functional components on the application page, filling the gap in front-end driving fields for protocols such as MCP, and realizing a closed loop from "data acquisition" to "UI execution." Further, the large language model in this application embodiment can directly call functional components on the application page instead of using the underlying DOM path. Therefore, as long as the services provided by the application remain unchanged, even if the application page is redesigned, driving the application page can still be achieved by updating the generated schema through the schema generator. Furthermore, this application embodiment incorporates the large language model as part of the application page, and the decisions of the large language model can be directly and instantly reflected on the application page, providing users with an integrated intelligent experience of "what you see is what you control."

[0033] Please refer to Figure 1 It illustrates the implementation environment involved in the page interaction method provided in the embodiments of this application, which includes: terminal 101 and server 102.

[0034] The terminal 101 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal 101 has a target application installed and running, which can provide any service such as shopping or information retrieval. To facilitate interaction with the user, the target application provides an application page, which can be a graphical user interface or a language user interface. To improve the smoothness of interaction, a conversational robot agent can be displayed on the application page. This conversational robot agent is the front-end interaction agent of the intelligent agent described in this embodiment, capable of interacting with the user in any form, such as text, audio, or images. During the interaction between the application page and the conversational robot agent, the user can input control statements for the application page, which are natural language commands. After receiving the control statement, the conversational robot agent sends it to the server 102, so that the server 102 can generate client-executable control commands based on the control statement, and then manipulate the application page by executing the control commands.

[0035] Server 102 can provide a natural language command to executable control command conversion service. After receiving a control statement, server 102 performs intent recognition on the operation statement to obtain the user's control intent. Based on the control intent, it retrieves the target metadata of the target functional component related to the control intent from the web tool database. Based on the target metadata, it performs format conversion on the control intent to obtain the target parameters, and then generates a control command including the target function name and target parameters. Finally, it sends the control command to the client of the target application.

[0036] The terminal 101 and server 102 described above can be directly or indirectly connected via network 103, and this embodiment of the application does not impose any limitations on this connection. The network 103 described above can be a wired network or a wireless network. Wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0037] Figure 2 A flowchart illustrating a page interaction method according to an embodiment of this application is shown. See also: Figure 2 The page interaction method can be provided by Figure 1 The target application is executed by the client and server 102 of the terminal 101.

[0038] The client provides the front-end page, i.e., the application page. This application page includes a conversational robot agent, which acts as the front-end interaction agent for the intelligent agent, used for language interaction with the user. Specifically, after receiving the user's input commands, the conversational robot agent sends them to the intelligent agent. The application page also includes various functional components, such as filtering, refreshing, highlighting, and dropshipping / free shipping components. Depending on the actual application requirements, all or part of these functional components can be encapsulated into a WebTool. In this embodiment, the WebTool is not a real tool on the front-end page, but rather a protocol tool for the functional components on the front-end page. Based on this protocol tool, the front-end functional components to be called can be inferred from the user's commands, and how to convert the user's commands into control instructions recognizable by the front-end can be determined. The metadata of the WebTool includes multiple fields such as toolName, toolDescription, inputSchema, and executor. To facilitate the large language model's recognition of the meaning of different fields, each field is labeled with a field description. For example, the toolName field is described as a unique identifier for the tool, the toolDescription field is described as a natural language description of the tool's function, the inputSchema field is described as a JSON schema of the parameters required to call this tool, and the executor field is described as the name of the function that is actually executed in the front-end SDK (Software Development Kit).

[0039] A front-end SDK is deployed on the front-end page. During the development phase of the WebTool, the front-end SDK is responsible for registering the WebTool to the local WebTool marketplace, i.e., the web tool database. The WebTool registration process essentially involves obtaining the function names of the functional components' execution functions and filling these names into the execution function name field, which is included in the metadata of the corresponding WebTool. This allows the SDK to determine the actual execution function after retrieving the metadata of the WebTool related to the user's control intent. During the WebTool's runtime, the front-end SDK receives control commands including the function parameters and function name of the execution function. It validates the function parameters, and if the validation passes, calls the execution function corresponding to the function name. The function parameters are then input into the execution function to obtain the executable function, which is then executed to drive changes to elements on the front-end page.

[0040] The server-side components include a schema generator, a WebTool marketplace, an agent, a runtime gateway engine, and a format converter. The schema generator generates JSON schema data with field descriptions for functional components based on their TypeScript, JavaScript, or JSON examples. The WebTool marketplace stores metadata for different functional components on the application page. The agent receives user input commands, performs intent recognition to determine the user's intent, and then, based on this intent and the WebTool marketplace, infers the function to be invoked and the field formats of different fields within that function. The runtime gateway engine acts as a gateway, enabling communication between the agent and other components on the front-end page and the server (such as the WebTool marketplace and format converter). The format converter matches the data format of the agent's output intent (rawData) with the field formats of corresponding fields in the function. If the data format of the intent does not match the corresponding field format in the function, it converts the intent's data format to obtain function parameters that match the field formats in the function.

[0041] Figure 2 The interaction process between the client and server is also shown above. The specific interaction process is as follows: 1. Developers input the TypeScript, JavaScript, or JSON examples of the functional components on the front-end page into the pattern generator.

[0042] 2. The schema generator generates a JSON schema with field descriptions, which is the metadata of the WebTool tool.

[0043] 3. The front-end SDK in the front-end page obtains the name of the execution function of the functional component input by the user, i.e., the WebTool function name, and fills the WebTool function name into the corresponding field of the WebTool tool's metadata to complete the registration of the WebTool function.

[0044] 4. During the conversational robot agent's language interaction with the user on the front-end page, it obtains the user's input commands and sends them to the intelligent agent. The intelligent agent receives the user's input commands, performs intent recognition on the commands, and obtains the user's control intent.

[0045] 5. Optionally, during the process of server-driven changes to the front-end page, the service back-end system can obtain the original information stream that matches the user's control intention from the back-end database based on the user's input control statement, and send the original information stream to the runtime gateway engine so that the original information stream can be controlled on the front-end page later.

[0046] 6. The intelligent agent queries the WebTool marketplace for the metadata of the WebTool corresponding to the user's control intent through the runtime gateway engine.

[0047] 7. The WebTool marketplace sends the metadata of the retrieved WebTools to the runtime gateway engine, which then sends the metadata to the agent.

[0048] 8. The agent inputs the metadata and manipulation intent of the WebTool queried through the runtime gateway engine into the format converter.

[0049] 9. The format converter converts the data format of the manipulation intent based on the metadata of the WebTool tool, and obtains the standard function parameters of the actual function executed by the front-end SDK.

[0050] 10. The intelligent agent sends control commands to the front-end page through the runtime gateway engine. These control commands include the function name and standard function parameters of the function actually executed by the front-end SDK.

[0051] 11. Upon receiving a control command, the agent calls the front-end SDK, which in turn calls the execution function corresponding to the function name. The standard function parameters are then input into the execution function to obtain an executable function. The content of the front-end page is then updated by executing the executable function.

[0052] This application provides a page interaction method, which can be generated by... Figure 1 The target application is executed by the client and server 102 installed on terminal 101. See also Figure 3 The method flow provided in this application embodiment includes: 301. The client displays an application page, which includes a first area for interacting with the agent.

[0053] The application page is the display page of the target application. This application page includes multiple areas: the first area, the second area, and the third area. The first area is used for interaction between the target account and the agent; the target account is the user's account used to log in to the target application. The second area displays checkboxes for functional components, which have two states: selected and unselected. A selected checkbox indicates that the corresponding functional component has been invoked, while an unselected checkbox indicates that the corresponding functional component has not been invoked. The third area displays the result of manipulation of the application page, that is, the result of refreshing the application page based on the user's natural language commands.

[0054] 302. The client obtains the control statements entered by the target account in the first area.

[0055] During interaction with the intelligent agent in the first area of ​​the application page, the user can input control statements for the objects displayed on the application page in the first area. These control statements are natural language commands and can be in any form, such as text or voice. When the target application is a shopping application, if the user wants to filter the products searched on the application page, the user can input the control statement in the first area: "Please help me find an item with free shipping."

[0056] 303. The client sends the control statements to the server.

[0057] As mentioned earlier, the application page can display the front-end interaction agent of the intelligent agent, namely the conversational robot agent. After the conversational robot agent obtains the control statement entered by the target account, it sends the control statement to the intelligent agent on the server.

[0058] 304. The server identifies the control intent from the control statements of the target account.

[0059] After receiving a control statement from a target account, the server-side agent can identify the target account's control intent from the statement. If the control statement is in text format, intent recognition can be performed directly. If the control statement is not in text format, it needs to be converted to text format before intent recognition. When converting non-text control statements to text format, the non-text control statements can be input into a multimodal conversion model. Once the server-side agent identifies the control intent from the control statement, it can input the control statement into an intent recognition model to obtain the control intent output by the intent recognition model; it can also extract keywords from the control statement and generate control intents based on the extracted keywords.

[0060] 305. Based on the manipulation intent, the server retrieves the target metadata from the web tool database.

[0061] The webpage tool database includes metadata for multiple functional components on the application page. Each component's metadata includes several fields, such as the component name (toolName field), the function name of the corresponding executable function (executor field), and the field format of different fields within the executable function (inputSchema field). To facilitate the Big Data language model's identification of the meaning of different fields, each field is annotated with a description. For example, the toolName field description is a unique identifier for the tool, the inputSchema field description is a JSON schema of the parameters required to call this tool, and the executor field description is the name of the actual executable function in the front-end SDK.

[0062] The webpage tool database is the foundation upon which the embodiments of this application can be implemented. Before proceeding with subsequent steps, the construction process of the webpage tool database will be described. See [link to documentation]. Figure 4 This application provides a method for constructing a webpage tool database, which can be executed by a server. The method includes: 401. Obtain the source code of multiple functional components on the application page.

[0063] In this embodiment, the application page displays multiple functional components, each capable of performing different functions. Developers can select multiple functional components from the application page according to their actual needs; these components may be some or all of the functional components displayed on the application page. For the selected functional components, developers can obtain their original code from a code database and then encapsulate them into a WebTool through subsequent steps. The original code of the functional components can be TypeScript code, JavaScript code, or JSON examples, etc.

[0064] 402. Input the original code of multiple functional components into the format converter and obtain the pattern data of multiple functional components output by the format converter.

[0065] In this embodiment, the schema data for each functional component describes the data structure of the corresponding functional component, and the field value of the function name field in the schema data of each functional component is empty. In this application embodiment, the server can also provide a webpage tool to generate a page. This webpage tool generates a page displaying multiple editing controls for the functional components, including a source code upload control, a function name input control, and an upload control. Developers can input the source code of multiple functional components into the source code upload control on the webpage tool's generated page, and then input the source code of multiple functional components into the format converter by touching the upload control. The format converter receives the source code of multiple functional components input by the developer, processes the source proxies of the multiple functional components, and obtains the schema data of the multiple functional components.

[0066] 403. Obtain the function names of the execution functions of multiple functional components.

[0067] In this embodiment, the developer can input the function names of the execution functions of each functional component on the execution function name input control of the web page generated by the web tool, and send the function names of the execution functions of multiple functional components to the server through the touch upload control.

[0068] 404. Fill the function name of the execution function of each functional component into the execution function name field of the schema data to obtain the metadata of each functional component.

[0069] After the server obtains the function name of the execution function of each functional component, it fills the execution function name field of the schema data with the function name of the execution function of each functional component to obtain the metadata of each functional component.

[0070] 405. Store the metadata of multiple functional components to obtain the web tool database.

[0071] It should be noted that the metadata of the functional components stored in the web tool database constructed in this application embodiment is not fixed and can be dynamically added or removed as application requirements change. For any functional component, if the function of the functional component changes after the target application page is redesigned, the metadata corresponding to the functional component needs to be updated.

[0072] In this embodiment, based on the constructed web tool database, after obtaining the control intent of the target account, the intelligent agent can retrieve target metadata from the web tool database based on the control intent. Specifically, the intelligent agent can input the control intent into a text vector extraction model to obtain the target text vector output by the text vector extraction model. Then, the intelligent agent inputs the component names of different functional components into the text vector extraction model to obtain multiple text feature vectors output by the text vector extraction model. Next, the intelligent agent calculates the similarity between the target text vector and the multiple text feature vectors respectively, obtaining the similarity calculation results between the target text vector and the multiple text feature vectors. Based on the similarity calculation results between the target text vector and the multiple text feature vectors, the intelligent agent obtains the component name corresponding to the text feature vector with the highest similarity to the target text vector, and then determines the metadata including the obtained component name as the target metadata.

[0073] 306. The server converts the manipulation intent based on the target metadata to obtain the target parameters.

[0074] The target metadata includes field descriptions of different fields in the target function. After acquiring the target data, the agent inputs the control intent and target metadata into a format converter. Based on the field descriptions of different fields in the target function, the format converter identifies the target fields that are semantically similar to the control intent from multiple fields in the target function. Then, it performs format conversion on the control intent according to the field format of the target fields to obtain the target parameters. These target parameters are the function parameters of the target function and are input as input parameters to the target function, enabling the target function to execute.

[0075] Furthermore, after obtaining the target parameters, the target parameters will be numerically cleaned to avoid incomplete or inaccurate target parameters being input into the target function.

[0076] Optionally, to reduce the resource consumption of the format converter, after obtaining the target data and control intent, the format converter in this embodiment will also determine whether the data format of the control intent conforms to the field format of the target field. When the data format of the control intent does not conform to the field format of the target field, the control intent and target metadata are input into the format converter; when the data format of the control intent conforms to the field format of the target field, the control intent and target metadata are no longer input into the format converter, but the control intent is directly used as the target parameter.

[0077] 307. The server sends control commands to the client.

[0078] The control instructions include the target function name and target parameters. After obtaining the target parameters and target function name, the server-side agent generates control instructions carrying the target parameters and target function name, and sends these control instructions to the client through the runtime gateway engine.

[0079] 308. In response to control commands sent by the server, the client manipulates the application page based on the target function name and target parameters.

[0080] In this embodiment, the client maintains a tool container that stores the execution functions of multiple functional components on the application page. Upon receiving a control command, the client invokes the front-end SDK. The front-end SDK, based on the target function name, retrieves the corresponding target function from the tool container, calls the target function, inputs the target parameters into the target function, obtains an executable function, and then executes the executable function to manipulate the application page. For example, if the target function includes a filter field, and the target parameter is the field value of the filter field (e.g., "one-piece dropshipping with free shipping"), then the field value "one-piece dropshipping with free shipping" is filled into the "filter field" of the target function to obtain the executable function. By executing this executable function, products eligible for "one-piece dropshipping with free shipping" can be filtered from the existing products on the application page.

[0081] Optionally, the front-end SDK can also validate the target parameters before executing the control command. If the target parameters pass the validation, the target function corresponding to the target function name is called to improve the success rate of controlling the application page.

[0082] In another embodiment of this application, multiple checkboxes for functional components are displayed on the second area of ​​the application page. In traditional page interaction, if a user wants to drive changes in the content displayed on the application page, they need to manually select a checkbox for a specific functional component. In response to the user's selection of a checkbox for a functional component, the client controls the content on the application page to change. In this embodiment, since the user does not directly select any functional components in the second area, to intuitively demonstrate the functional components called by the client, the front-end SDK automatically selects the checkboxes for the target functional components in the second area when executing executable functions to manipulate the application page.

[0083] For example, Figure 5As shown in the application page, the user enters "I want dropshipping with free shipping" in the first area. The chatbot agent on the application page obtains the user's input "I want dropshipping with free shipping" and sends it to the server. Based on the user's input "I want dropshipping with free shipping," the server determines that the required functional component is the dropshipping with free shipping component, generates the corresponding control command, and then sends the control command to the client. The client calls the front-end SDK to manipulate the application page and automatically checks the dropshipping with free shipping component in the second area, so that the user knows that the application page is being manipulated based on this dropshipping with free shipping component.

[0084] In another embodiment of this application, in response to the user's control statement, not only are the functional components invoked on the application page automatically selected, but the server also manipulates the existing objects on the application page based on the control statement to obtain multiple target objects, which are then displayed in the third area. For example, if multiple products are displayed in the second area of ​​the application page, and the user enters "dropshipping with free shipping" in the first area, the server will filter out the products that qualify for dropshipping with free shipping from the multiple products. Then, the front-end SDK updates the product list on the application page based on the server's filtering results to display the filtered products on the application page.

[0085] Furthermore, after the server manipulates existing objects on the application page based on user commands to obtain multiple target objects, it can also sort the target objects across different dimensions based on their attribute information. This sorting result is then used to generate multiple target tags for each target object. When multiple target objects are displayed in the third area, these tags are displayed at their respective display positions. These different dimensions can be comprehensive, price-based, after-sales-based, etc. Correspondingly, the target tags can be "AI Priority Recommendation" and "AI Alternative Recommendation" tags derived from the comprehensive multi-dimensional sorting results, "Lowest Price" based on the price dimension, "Good After-Sales Service" based on the after-sales-based dimension, and so on.

[0086] For example, users in Figure 5After entering "dropshipping with free shipping" in the first area, the dropshipping with free shipping component in the second area is automatically selected. At the same time, the third area displays product information for multiple microscopes that meet the dropshipping with free shipping requirements, and displays corresponding labels in the display positions of multiple microscopes. For example, if the first microscope is obtained from the results of sorting based on multiple dimensions, then the label "AI Priority Recommendation" will be displayed in the display position of the first microscope; if the third microscope is obtained from the after-sales service dimension, then the label "Good After-sales Service" will be displayed in the display position of the third microscope.

[0087] The page interaction method flow provided in this application embodiment can be divided into a developer flow and a runtime flow. See [link to relevant documentation]. Figure 6 The developer-side process includes: the developer obtains the TypeScript, JavaScript, or JSON examples of functional components on the application page, and inputs these codes into a schema generator; the schema generator outputs the metadata (JSON Schema with field descriptions) of the WebTool corresponding to the functional component, and then registers and stores the WebTool function for that component in the WebTool marketplace. The runtime-side process includes: the user inputs a control statement (raw data) on the application page; the server-side agent identifies the control intent of the statement and, based on this intent, retrieves the metadata of the WebTool from the WebTool marketplace; then, inputs the WebTool metadata and the control intent into a format converter to obtain the standard function parameters output by the format converter; the agent generates control instructions based on the standard function parameters and function names, and sends these instructions to the client to manipulate the application page.

[0088] This application provides a page interaction method, see [link to relevant documentation]. Figure 7 The interaction method includes the following steps: 1. During the interaction with the conversational robot agent, the user enters "Help me filter for free shipping items shipped from Hangzhou" on the application page. The conversational robot agent obtains the user's input statement "Help me filter for free shipping items shipped from Hangzhou" and sends the statement to the intelligent agent.

[0089] 2. The intelligent agent obtains the user's input statement "help me filter for free shipping products shipped from Hangzhou", identifies the control intent "dropshipping with free shipping" and "shipped from Hangzhou" from the statement, and then obtains the metadata of the WebTool tool corresponding to the user's control intent from the WebTool tool market through the runtime gateway engine.

[0090] 3. The intelligent agent inputs the control intent and metadata of the WebTool into the format converter through the runtime gateway engine, and obtains the standard function parameters {"tags":["free shipping"],"location":"Hangzhou"} output by the format converter.

[0091] 4. The agent generates control instructions based on parameters. These instructions include function parameters and the function name of the function to be executed from the metadata of the WebTool tool. The control instructions are then sent to the client via the runtime gateway engine.

[0092] 5. The client's front-end SDK calls the corresponding execution function based on the function name, inputs the function parameters into the execution function to obtain the executable function, and then executes the executable function. The dropshipping and free shipping component on the page is automatically selected, and the products displayed on the application page are updated according to the filtering results of the products in the product list on the application page by the server, so as to display products that meet the requirements of Hangzhou delivery and dropshipping and free shipping, and at the same time display the corresponding tags of each filtered product.

[0093] Using the method provided in the embodiments of this application, the large language model can directly call functional components on the application page, realizing the driving of the application page by natural language commands. This significantly reduces user operation steps and achieves a "one-sentence direct access to the goal" interactive experience. Furthermore, the format converter and pattern generator are deployed on the backend, resulting in lower integration costs and improved development efficiency. All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0094] Please refer to Figure 8 The diagram illustrates a page interaction device provided in an embodiment of this application. This device can be implemented through software, hardware, or a combination of both, and can become all or part of an electronic device. The device includes: The identification module 801 is used to identify the control intention from the control statements of the target account, wherein the control statements are the statements entered by the target account when interacting with the intelligent agent on the application page. The first acquisition module 802 is used to acquire target metadata from the web tool database based on the control intention. The target metadata is the metadata of the target functional component related to the control intention on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. The conversion module 803 is used to convert the control intention into a format based on the target metadata to obtain target parameters; The sending module 804 is used to send control instructions to the client. The control instructions include the target function name and the target parameters. The control instructions are used to instruct the client to manipulate the application page based on the target function name and the target parameters.

[0095] In another embodiment of this application, the apparatus further includes: The second acquisition module is used to acquire the original code of multiple functional components on the application page; The third acquisition module is used to input the original code of the multiple functional components into the format converter, acquire the pattern data of the multiple functional components output by the format converter, the pattern data of each functional component is used to describe the data structure of the corresponding functional component, and the field value of the function name field in the pattern data of each functional component is empty; The fourth acquisition module is used to acquire the function names of the execution functions of the multiple functional components; The populate module is used to populate the function name of the execution function of each functional component into the execution function name field of the schema data to obtain the metadata of each functional component; A storage module is used to store the metadata of the multiple functional components to obtain the web tool database.

[0096] In another embodiment of this application, the metadata of the different functional components includes the component names of the functional components. The first acquisition module 802 is used to input the control intent into the text vector extraction model to obtain the target text vector output by the text vector extraction model; input the component names of the different functional components into the text vector extraction model to obtain multiple text feature vectors output by the text vector extraction model; calculate the similarity between the target text vector and the multiple text feature vectors respectively; obtain the component name corresponding to the text feature vector with the highest similarity to the target text vector; and determine the metadata including the obtained component names as the target metadata.

[0097] In another embodiment of this application, the target metadata includes field descriptions of different fields in the target function. The conversion module 803 is used to input the control intent and the target metadata into a format converter to obtain the target parameters output by the format converter. The target parameters are parameters obtained by the format converter from the control intent by format conversion according to the field format of the target field. The target field is a field in the target function whose field description is semantically similar to the control intent.

[0098] In another embodiment of this application, the apparatus further includes: The judgment module is used to determine whether the data format of the control intention conforms to the field format of the target field; The conversion module 803 is used to perform a step of converting the control intention based on the target metadata to obtain the target parameters when the data format of the control intention does not conform to the field format of the target field.

[0099] Please refer to Figure 9 The diagram illustrates a page interaction device provided in an embodiment of this application. This device can be implemented through software, hardware, or a combination of both, and can become all or part of an electronic device. The device includes: The first display module 901 is used to display an application page, the application page including a first area for interacting with the intelligent agent; The acquisition module 902 is used to acquire the control statements entered by the target account in the first area; The sending module 903 is used to send the control statement to the server. The control statement is used by the server to obtain target metadata from the web tool database based on the control intent identified from the control statement. Based on the target metadata, the server performs format conversion on the control intent to obtain target parameters. The target metadata is the metadata of the target functional component related to the control intent on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. The control module 904 is used to control the application page in response to the control command sent by the server, based on the target function name and the target parameters.

[0100] In another embodiment of this application, the control module 904 is used to call the target function corresponding to the target function name; input the target parameters into the target function to obtain an executable function; and execute the executable function to control the application page.

[0101] In another embodiment of this application, the application page further includes a second area for displaying checkboxes for functional components, and the control module 904 is used to automatically select the checkboxes for the target functional components in the second area.

[0102] In another embodiment of this application, the apparatus further includes: The second display module is used to display multiple target objects on the third area. Multiple target tags are added to the multiple target objects. The multiple target tags are generated by the server based on the sorting results of the multiple target objects in different dimensions. The multiple target objects are objects obtained by manipulating the original objects on the application page based on the executable function.

[0103] Figure 10 This diagram illustrates a structural block diagram of an electronic device 1000 provided in an exemplary embodiment of this application. Typically, the electronic device 1000 includes a processor 1001 and a memory 1002.

[0104] Memory 1002 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0105] The processor 1001, coupled to the memory 1002, is used to execute the computer program in the memory 1002 to implement the interactive interface generation method provided in the foregoing embodiments.

[0106] Furthermore, such as Figure 10 As shown, the electronic device also includes: a communication component 1003, a display 1004, a power supply component 1005, an audio component 1006, and other components. Figure 10 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 10 The components shown are as follows. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server device such as a conventional server, cloud server, or server array.

[0107] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0108] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0109] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0110] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0111] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0112] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium. Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A page interaction method, characterized in that, The method includes: Identify the control intent from the control statements of the target account, wherein the control statements are the statements entered by the target account when interacting with the intelligent agent on the application page; Based on the control intent, target metadata is obtained from the web tool database. The target metadata is the metadata of the target functional component related to the control intent on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. Based on the target metadata, the manipulation intent is format-converted to obtain target parameters; A control command is sent to the client. The control command includes the target function name and the target parameters. The control command is used to instruct the client to manipulate the application page based on the target function name and the target parameters.

2. The method according to claim 1, characterized in that, Before retrieving target metadata from the web tool database based on the stated manipulation intent, the process also includes: Obtain the original code of multiple functional components on the application page; The original code of the multiple functional components is input into the format converter to obtain the pattern data of the multiple functional components output by the format converter. The pattern data of each functional component is used to describe the data structure of the corresponding functional component, and the field value of the function name field in the pattern data of each functional component is empty. Obtain the function names of the execution functions of the multiple functional components; The function name of the execution function of each functional component is populated into the execution function name field of the schema data to obtain the metadata of each functional component; The metadata of the multiple functional components is stored to obtain the web page tool database.

3. The method according to claim 1, characterized in that, The metadata of multiple functional components in the web tool database includes the component names of multiple functional components. The step of obtaining target metadata from the web tool database based on the control intent includes: The control intent is input into the text vector extraction model to obtain the target text vector output by the text vector extraction model; Input the component names of the multiple functional components into the text vector extraction model to obtain multiple text feature vectors output by the text vector extraction model; Calculate the similarity between the target text vector and the plurality of text feature vectors respectively; Obtain the component name corresponding to the text feature vector that has the highest similarity to the target text vector; The metadata including the acquired component names is determined as the target metadata.

4. The method according to claim 1, characterized in that, The target metadata also includes field descriptions of different fields in the target function. Based on the target metadata, the manipulation intent is format-converted to obtain target parameters, including: The control intent and the target metadata are input into the format converter to obtain the target parameters output by the format converter. The target parameters are parameters obtained by the format converter by converting the control intent according to the field format of the target field. The target field is a field in the target function whose field description is semantically similar to the control intent.

5. The method according to claim 4, characterized in that, Before converting the control intent based on the target metadata to obtain the target parameters, the method further includes: Determine whether the data format of the control intent conforms to the field format of the target field; When the data format of the control intent does not conform to the field format of the target field, the step of converting the control intent based on the target metadata to obtain the target parameters is performed.

6. A page interaction method, characterized in that, The method includes: Display an application page, the application page including a first area for interacting with the intelligent agent; Obtain the control statements entered by the target account in the first area; The control statement is sent to the server. The control statement is used by the server to obtain target metadata from the web tool database based on the control intent identified from the control statement. Based on the target metadata, the control intent is format-converted to obtain target parameters. The target metadata is the metadata of the target functional component related to the control intent on the application page. The target metadata is used to indicate the field format of different fields in the target function corresponding to the target function name. The target function is the execution function of the target functional component. The web tool database includes the metadata of multiple functional components on the application page. In response to the control commands sent by the server, the application page is manipulated based on the target function name and the target parameters.

7. The method according to claim 6, characterized in that, The manipulation of the application page based on the target function name and the target parameters includes: Call the target function corresponding to the target function name; The target parameters are input into the target function to obtain an executable function; The executable function is executed to manipulate the application page.

8. The method according to claim 7, characterized in that, The application page also includes a second area for displaying checkboxes for functional components, and executing the executable function to manipulate the application page includes: Automatically select the checkboxes for the target functional components described in the second area.

9. The method according to claim 7, characterized in that, The application page also includes a third area for displaying the control results. After executing the executable function to control the application page, the process further includes: Multiple target objects are displayed in the third area, and multiple target tags are added to the multiple target objects. The multiple target tags are generated by the server based on the sorting results of the multiple target objects in different dimensions. The multiple target objects are objects obtained by manipulating the original objects on the application page based on the executable function.

10. An electronic device, characterized in that, It includes a processor and a memory; the memory stores at least one piece of program code; the at least one piece of program code is called and executed by the processor to implement the page interaction method as described in any one of claims 1 to 5, or the page interaction method as described in any one of claims 6 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which, when executed by a processor, is capable of implementing the page interaction method as described in any one of claims 1 to 5, or the page interaction method as described in any one of claims 6 to 9.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, can implement the page interaction method as described in any one of claims 1 to 5, or the page interaction method as described in any one of claims 6 to 9.

Images