Page interaction method and device and storage medium

Abstract

The invention relates to a page interaction method and device and a storage medium. The method comprises the steps that in response to operation of a user, a first operation process is run on a first page of a browser, and the first operation process is associated with a scene of a target task; based on the first operation process, calling the server to process and return a processing result, the processing including natural language processing; and completing the target task on the first page based on a processing result returned by the server. According to the embodiment of the invention, the AI and the RPA can be combined, so that the AI has action ability, a user does not need to operate a page according to a processing result, but automatically operates the browser and the page according to the operation process, so that the AI processing result is automatically applied on the page, the target task is efficiently completed, the working efficiency is improved, and the user experience is improved. And the type of work can be automatically completed on the page, so that the task throughput is not limited by manpower any more, and the application scene can be expanded on a large scale.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to a page interaction method, device and storage medium. Background Art

[0002] In recent years, artificial intelligence (AI) technology has achieved rapid development. The existing AI interaction mode is mainly based on dialogue. Users need to interact with AI by asking questions and complete subsequent task operations according to AI's responses. In this mode, AI is only an information auxiliary tool and cannot directly participate in the execution of tasks. The completion of tasks is still highly dependent on human operation. This "task and AI separation" model limits the efficiency of AI applications and the possibility of large-scale deployment. Therefore, in the scenario of completing workflows based on web pages, a new technical solution combining artificial intelligence and process automation (RPA, Robotic Process Automation) is urgently needed to achieve full process automation and improve the stability and efficiency of workflows in page interaction scenarios. Summary of the invention

[0003] In view of this, the present disclosure proposes a page interaction method, device and storage medium.

[0004] According to one aspect of the present disclosure, a page interaction method is provided. The method comprises:

[0005] In response to the user's operation, a first operation process is executed on a first page of the browser, where the first operation process is associated with a scenario of a target task;

[0006] Based on the first operation flow, calling the server to process and return the processing result, the processing including natural language processing;

[0007] Based on the processing results returned by the server, the target task is completed on the first page.

[0008] In a possible implementation, based on the first operation flow, calling the server to perform processing and return the processing result includes:

[0009] Based on the first operation process, determining the knowledge source information;

[0010] The structural information and knowledge source information of the first page are sent to the server, so that the server performs processing based on the structural information and knowledge source information of the first page to obtain a processing result.

[0011] In a possible implementation, based on the first operation process, determining the knowledge source information includes:

[0012] Based on the first operation process, opening a second page and / or a file uploaded by the user on the browser, the second page including a third-party web page;

[0013] The content in the second page and / or the file uploaded by the user is read as the knowledge source information.

[0014] In a possible implementation, the first operation flow includes one or more operation nodes, and based on the processing result returned by the server, completes the target task on the first page, including:

[0015] Determine a target operation node in the first page according to the first operation process;

[0016] The target operation node is operated based on the processing result.

[0017] In a possible implementation, the server performs processing based on the structure information and knowledge source information of the first page to obtain a processing result, including:

[0018] The server runs the second operation process;

[0019] Based on the second operation flow, the server uses the large language model to process the structural information and knowledge source information of the first page to obtain a processing result.

[0020] In a possible implementation, the second operation flow includes a data processing method at the server and a type of large language model used.

[0021] In a possible implementation, the first page includes a form page, and operating the target operation node based on the processing result includes:

[0022] Fill the processing results into the form page corresponding to the target operation node.

[0023] In a possible implementation, the first page includes an account operation page, an operation node is associated with an account, and an operation is performed on a target operation node based on a processing result, including:

[0024] Based on the processing results, an operation is performed in the target account corresponding to the target operation node, and the operation includes any one or more of account information management, content management, and account interaction.

[0025] According to another aspect of the present disclosure, a page interaction device is provided. The device includes:

[0026] An operation module, configured to execute a first operation process on a first page of a browser in response to an operation of a user, wherein the first operation process is associated with a scenario of a target task;

[0027] A calling module, configured to call a server to process and return a processing result based on the first operation process, wherein the processing includes natural language processing;

[0028] The completion module is used to complete the target task on the first page based on the processing results returned by the server.

[0029] In one possible implementation, a calling module is used to:

[0030] Based on the first operation process, determining the knowledge source information;

[0031] The structural information and knowledge source information of the first page are sent to the server, so that the server performs processing based on the structural information and knowledge source information of the first page to obtain a processing result.

[0032] In a possible implementation, based on the first operation process, determining the knowledge source information includes:

[0033] Based on the first operation process, opening a second page and / or a file uploaded by the user on the browser, the second page including a third-party web page;

[0034] The content in the second page and / or the file uploaded by the user is read as the knowledge source information.

[0035] In a possible implementation, the first operation flow includes one or more operation nodes and a completion module, which is used to:

[0036] Determine a target operation node in the first page according to the first operation process;

[0037] The target operation node is operated based on the processing result.

[0038] In a possible implementation, the server performs processing based on the structure information and knowledge source information of the first page to obtain a processing result, including:

[0039] The server runs the second operation process;

[0040] Based on the second operation flow, the server uses the large language model to process the structural information and knowledge source information of the first page to obtain a processing result.

[0041] In a possible implementation, the second operation flow includes a data processing method at the server and a type of large language model used.

[0042] In a possible implementation, the first page includes a form page, and operating the target operation node based on the processing result includes:

[0043] Fill the processing results into the form page corresponding to the target operation node.

[0044] In a possible implementation, the first page includes an account operation page, an operation node is associated with an account, and an operation is performed on a target operation node based on a processing result, including:

[0045] Based on the processing results, an operation is performed in the target account corresponding to the target operation node, and the operation includes any one or more of account information management, content management, and account interaction.

[0046] According to another aspect of the present disclosure, a page interaction device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0047] According to another aspect of the present disclosure, a non-volatile computer-readable storage medium is provided, on which computer program instructions are stored, wherein the computer program instructions implement the above method when executed by a processor.

[0048] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0049] According to an embodiment of the present disclosure, by responding to the user's operation, a first operation process associated with the scenario of the target task is run on the first page of the browser, and a server is called based on the first operation process to perform processing and return a processing result, wherein the processing includes natural language processing, and the target task is completed on the first page based on the processing result returned by the server. This allows AI to be combined with RPA to give AI the ability to act, without the user having to operate the page according to the processing result. Instead, the browser and page are automatically operated according to the operation process to automatically apply the AI ​​processing results on the page, efficiently complete the target task, and improve work efficiency. The first operation process can be used to standardize work tasks so that this type of work can be automatically completed on the page, so that the task throughput is no longer limited by manpower, and the application scenarios can be expanded on a large scale.

[0050] Further features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the disclosure and, together with the description, serve to explain the principles of the disclosure.

[0052] Figure 1A schematic diagram showing an application scenario according to an embodiment of the present disclosure.

[0053] Figure 2 A flow chart of a page interaction method according to an embodiment of the present disclosure is shown.

[0054] Figure 3 A schematic diagram of the overall flow of the page interaction method according to an embodiment of the present disclosure is shown.

[0055] Figure 4 A structural diagram of a page interaction device according to an embodiment of the present disclosure is shown.

[0056] Figure 5 It is a block diagram of a device 1900 for page interaction according to an exemplary embodiment. DETAILED DESCRIPTION

[0057] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0058] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0059] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present disclosure.

[0060] In recent years, artificial intelligence (AI) technology has achieved rapid development. The existing AI interaction mode is mainly based on dialogue. Users need to interact with AI by asking questions and complete subsequent task operations according to AI's responses. In this mode, AI is only an information auxiliary tool and cannot directly participate in the execution of tasks. The completion of tasks is still highly dependent on human operation. This "task and AI separation" model limits the efficiency of AI applications and the possibility of large-scale deployment. Therefore, in the scenario of completing workflows based on web pages, a new technical solution combining artificial intelligence and process automation (RPA, Robotic Process Automation) is urgently needed to achieve full process automation and improve the stability and efficiency of workflows in page interaction scenarios.

[0061] In view of this, the present disclosure provides a page interaction method, device and storage medium. The method of the embodiment of the present disclosure runs a first operation process associated with the scenario of the target task on the first page of the browser in response to the user's operation, and calls the server to process and return the processing result based on the first operation process, the processing includes natural language processing, and the target task is completed on the first page based on the processing result returned by the server. It can realize the combination of AI and RPA, so that AI has the ability to act, and there is no need for the user to operate the page according to the processing result, but to automatically operate the browser and page according to the operation process, so as to automatically apply the AI ​​processing results on the page, efficiently complete the target task, and improve work efficiency. The work tasks can be standardized through the first operation process, so that this type of work can be automatically completed on the page, so that the task throughput is no longer limited by manpower, and the application scenarios can be expanded on a large scale.

[0062] Figure 1 A schematic diagram showing an application scenario according to an embodiment of the present disclosure is shown. Figure 1 As shown, the page interaction method disclosed in the present invention can be applied to a terminal device, for example, it can be used in a browser plug-in on a terminal device. The browser plug-in runs a pre-configured operation process on the browser page, and calls a server (for example, a cloud platform) for processing to return the processing result, so that the corresponding work task can be completed on the page.

[0063] The method of the embodiments of the present disclosure can be applied to any scenario where a workflow is completed based on a browser page, such as form filling, financial report generation, customer relationship management, account operation, etc., and the present disclosure does not impose any restrictions on this.

[0064] The terminal device involved in the present disclosure may be any one or more of a mobile phone, a foldable electronic device, a tablet computer, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), and a vehicle-mounted device. The terminal device may be a device including a touch screen, or a device that can obtain user input through other means such as a keyboard and a mouse. The embodiments of the present disclosure do not impose any special restrictions on the specific type of the terminal device, and the terminal device may have a wired or wireless communication function.

[0065] Figure 2 A flow chart of a page interaction method according to an embodiment of the present disclosure is shown. Figure 2 As shown, the method may include:

[0066] Step S201, in response to a user operation, executing a first operation process on a first page of a browser.

[0067] The user's operation may include an operation on a plug-in in the browser to invoke the plug-in. The plug-in may also be automatically invoked when the first page is opened, and the user's operation is an operation to open the first page. The first page may be opened by the plug-in based on the first operation process, or may be opened by the user before the plug-in is invoked. The first page may include one or more pages, and multiple pages may be configured to be opened in sequence in a logical order in the first operation process, and the first operation process is completed through multiple pages in collaboration.

[0068] The first operation process may be a pre-configured process associated with the scenario of the target task. The target task may be, for example, a form filling task, an account operation task, a text parsing task, etc. The target task may be a task corresponding to the first page, such as a task that needs to be completed through the first page. For example, if the first page is a form filling page, the target task is the form filling task.

[0069] The first operation flow may include one or more operation nodes. The operation node may represent a step that needs to be operated or interacted when executing the target task. The operation node may include, for example, a node for filling in or selecting content, a node for triggering an action (such as clicking a control), a node for reading page content, a node for performing logical judgment, etc.

[0070] The operation nodes included in the first operation flow can be preset according to the needs of the target task.

[0071] For example, when the target task is a form filling task, the first page may include a form page, and the operation nodes in the first operation process may include, for example, nodes for filling in form fields in the form page (such as "name", "contact information", etc.), nodes for submitting buttons (used to save or submit the completed form page), etc.

[0072] For example, when the target task is an operational task, the first page may also include an account operation page, and the operation nodes may be associated with the account, such as nodes for logging into the account, nodes for content management (such as nodes for generating content and filling in the title and text areas, uploading pictures / videos and attaching relevant tags, and clicking the "Publish" or "Save Draft" button), and nodes for interactive operations (such as nodes for automatically replying to user messages and liking specified comments).

[0073] The first operation flow can be pre-configured by the R&D personnel. For a simple first operation flow, it can also be configured by the user, wherein the user can customize the operation nodes in the first operation flow in the browser plug-in. For example, when the target task is a form filling task, operation nodes such as "open page", "fill in form", and "click button" can be preset. When the target task is an operation task, operation nodes such as "login account", "generate title and text content", "upload pictures / videos", "like comments", and "publish content" can be preset. The user can complete the custom design of the first operation flow by selecting the operation node and arranging the execution order between the operation nodes.

[0074] Step S202: Based on the first operation flow, call the server to perform processing and return the processing result.

[0075] Among them, the server can be a cloud platform. In the first operation process, a specific operation node can be set for calling the server for processing. For example, the operation nodes of the above-mentioned first operation process can also include operation nodes for obtaining structural information in the first page, and operation nodes for obtaining knowledge source information, so as to call the server for processing based on the obtained information. It is also possible to automatically trigger the operation of calling the server for processing when executing a specific operation node in the first operation process. For example, when the first operation process executes to specific nodes such as "fill in the form" or "log in to the account", the structural information in the first page and the operation node for obtaining knowledge source information can be obtained, so as to call the server for processing based on the obtained information.

[0076] In step S202, it is possible to:

[0077] Based on the first operation process, determining the knowledge source information;

[0078] The structural information and knowledge source information of the first page are sent to the server, so that the server performs processing based on the structural information and knowledge source information of the first page to obtain a processing result.

[0079] By automatically acquiring the structural information and knowledge source information of the first page based on the first operation process, and processing the structural information and knowledge source information of the first page by the server, RPA and AI can be combined to automatically provide the data required for the first operation process to the server, and use the processing capabilities of the server to obtain the required processing results.

[0080] The structural information of the first page may be obtained based on specific rules or related parsing techniques. The structural information of the first page may include form fields, controls, text boxes, etc. corresponding to the current operation node, indicating the currently interactive content.

[0081] The knowledge source information can be used to execute the corresponding operation node of the first operation process. For example, when the current operation node is to fill in the form field content, the knowledge source information can include the information to be filled in the form field; when the current operation node is to log in to the account, the knowledge source information can include the account name and account password of the account to be logged in.

[0082] Among them, it is possible to: based on the first operation process, open the second page and / or the file uploaded by the user on the browser; read the content of the second page and / or the file uploaded by the user as the knowledge source information.

[0083] By reading the content of the second page and / or the file uploaded by the user as the knowledge source information, the knowledge source information can be enriched, and when AI capabilities are subsequently utilized, the AI ​​output can be made more stable and accurate, thereby further improving the efficiency and stability of the entire process.

[0084] The second page may include one or more third-party web pages, and the files uploaded by the user may include documents, tables, etc. The user may upload files through the first page or other interfaces. The second page and the files uploaded by the user may be web pages or files for providing relevant information to support the browser plug-in to complete the automated tasks in the first operation process. For example, the second page may be a third-party web page of the type of encyclopedia, film review, e-commerce, social media, etc.

[0085] The URL of the second webpage associated with the operation node and / or the storage address of the file associated with the operation node (which may be a local storage address or a cloud storage address) may be configured in the operation node of the first operation flow. When the first operation flow is executed to the corresponding operation node, the corresponding second webpage and / or the file uploaded by the user may be opened based on the URL and / or file storage address configured in the operation node.

[0086] Among them, the second page and / or the file uploaded by the user can be opened through the plug-in, and the data of the specified field in the second page and / or file can be read according to the task specified by the operation node in the first operation process. For example, when the second page is a page of the film review type, and the operation node is "Extract movie ratings and number of reviews", the plug-in can locate the "rating" and "number of reviews" elements of the page through the selector, and extract the data under the corresponding fields. For another example, when the file uploaded by the user includes the account name, account password, etc. of the account to be logged in, and the operation node is "Login Account", the plug-in can read the "Account Name" and the corresponding "Account Password" fields respectively, and extract the data under the corresponding fields respectively.

[0087] The plug-in can open and display the second page and / or file, or open the second page and / or file in the background (i.e., not display it). For example, when the user's private information is involved (such as when the file includes the account name and password of the account to be logged in), the second page and / or file can be opened in the background to ensure information security. In addition, when the file includes user private information, information security can also be ensured by encrypting the file, setting read and write permissions for the file, and other mechanisms.

[0088] After obtaining the structural information and knowledge source information of the first page, the browser plug-in may send the information to the server to call the server to process the information, and the processing may include calling the server to perform natural language processing.

[0089] The server may: run a second operation process, and based on the second operation process, use a large language model to process the structural information and knowledge source information of the first page to obtain a processing result.

[0090] By using a large language model to process the structural information and knowledge source information of the first page, it is possible to use AI capabilities to further improve the efficiency and stability of the entire process and make it easy to expand on a large scale.

[0091] The second operation process can be pre-configured by the user, and the process of natural language understanding with the server can be managed, which can include the data processing method of the server and the type of large language model used. The data processing method can include choosing whether to use relevant tools to pre-process the relevant data of the structural information and knowledge source information of the first page, and the pre-processed objects include the structural information and knowledge source information of the first page. The type of pre-processing can be configured in the second operation process based on task requirements, such as data cleaning, data formatting, data deduplication and compression, context completion, data screening, privacy desensitization, etc. For example, before inputting the film review content extracted from the second page into the large language model, you can choose to pre-process the film review content to format the data and convert it into a specific format.

[0092] Among them, for different operation nodes, corresponding second operation processes can be pre-configured, and different data processing methods and large language models can be selected for processing based on the configuration of the second operation processes corresponding to different operation nodes, so that the selected data processing methods and large language models are suitable for processing the data of the corresponding operation nodes (such as the structural information and knowledge source information of the first page) and completing the operation target of the operation node. For example, when the current operation node is to fill in the form field content, the multimodal language model can be used to identify the content of the corresponding form field in the structural information of the first page in the knowledge source information to ensure that the content in the knowledge source information is correctly filled into the corresponding form field. At this time, the processing result returned by the server can be the content to be filled that matches the form field; when the current operation node is to publish the reply content, the generative language model can be used to generate the content copy based on the knowledge source information to fill in the text box in the corresponding first page for publishing. At this time, the processing result returned by the server can be the generated reply content.

[0093] The disclosed embodiment uses the processing results output by the large language model to complete the target task on the first page, which can significantly improve the efficiency and stability of the workflow and is easy to expand on a large scale.

[0094] Step S203, completing the target task on the first page based on the processing result returned by the server.

[0095] According to an embodiment of the present disclosure, by responding to the user's operation, a first operation process associated with the scenario of the target task is run on the first page of the browser, and a server is called based on the first operation process to perform processing and return a processing result, wherein the processing includes natural language processing, and the target task is completed on the first page based on the processing result returned by the server. This allows AI to be combined with RPA to give AI the ability to act, without the user having to operate the page according to the processing result. Instead, the browser and page are automatically operated according to the operation process to automatically apply the AI ​​processing results on the page, efficiently complete the target task, and improve work efficiency. The first operation process can be used to standardize work tasks so that this type of work can be automatically completed on the page, so that the task throughput is no longer limited by manpower, and the application scenarios can be expanded on a large scale.

[0096] In step S203, you can:

[0097] Determine a target operation node in the first page according to the first operation process; and operate the target operation node based on the processing result.

[0098] By operating the target operation node based on the processing results returned by the server, AI can be given the ability to act, directly help operate the browser and pages, complete related tasks, standardize the target tasks, and be able to expand on a large scale.

[0099] The target operation node may be the operation node currently being executed by the first operation process. For example, if the second operation process is currently being executed to the operation node of "filling in a form", the target operation node is "filling in a form".

[0100] For example, when the first page is a form page, in the process of operating the target operation node based on the processing result, the processing result can be filled into the form page corresponding to the target operation node.

[0101] In this way, an automated workflow for form filling tasks can be realized, so that the throughput of form filling tasks is no longer limited by manpower, and the work efficiency of form filling work is greatly improved.

[0102] At this time, the target operation node can be to fill in the form field content, and the to-be-filled content matching the form field returned by the server can be filled into the corresponding form field in the form page.

[0103] After the processing result is filled into the form page corresponding to the target operation node, the form can be submitted and the first operation flow can be ended. Before submitting the form, the user can confirm whether the content filled in the form page is correct, and submit it after the user confirms that it is correct. When the user confirms that the filled content is incorrect, the user can manually modify the corresponding erroneous field, or the user can adjust the operation nodes in the first operation flow and / or the second operation flow, and then re-execute the first operation flow to obtain a re-filled form page.

[0104] For another example, when the first page is an account operation page, the operation node is associated with the account, and in the process of operating the target operation node based on the processing result, you can: perform an operation in the target account corresponding to the target operation node based on the processing result.

[0105] In this way, the automated workflow of operational tasks can be realized, so that the throughput of operational tasks is no longer limited by manpower, and the work efficiency of related operational work can be greatly improved.

[0106] Operational operations may include any one or more of account information management, content management, and account interaction. Account information management may include, for example, updating basic information such as account name, avatar, profile, and contact information; content management may include posting reply content, reviewing published content, checking metrics such as clicks, likes, shares, and comments on published content, and editing published content; account interaction may include interactive behaviors with other accounts, such as liking, commenting, and forwarding content posted by other accounts.

[0107] For example, when the target operation node is to publish the reply content, the generated reply content returned by the server can be published under the corresponding account. Before publishing the corresponding reply content, the user can confirm whether the reply content can be published, and publish it after the user confirms. When the user wants to modify the reply content, the user can manually modify the reply content, or the user can adjust the operation nodes in the first operation process and / or the second operation process, and then re-execute the first operation process to obtain the regenerated reply content.

[0108] The first operation flow corresponding to the target task can also be associated with the first operation flow corresponding to other tasks. After the first operation flow corresponding to the target task is executed, the first operation flow corresponding to other associated tasks can also be continued to be executed. For example, the form filling task can be associated with the account operation task. After the form filling task is executed, the account operation task can be continued based on the completed form page, and the first operation flow corresponding to the account operation task can be run.

[0109] Figure 3 The overall flow diagram of the page interaction method according to the embodiment of the present disclosure is shown. Figure 3 As shown, in an overall process example of an embodiment of the present disclosure, a user can open the first page that needs to be operated, and invoke the browser plug-in to run the first operation process under the configured target task. The browser plug-in operates the browser to open the second page or read the file uploaded by the user to obtain the knowledge source information. The browser plug-in can send the structure of the first page and the knowledge source information to the server; the server runs the second operation process, uses a large language model and related tools to understand the content and return the final processing result, and can also choose whether to perform data preprocessing in this process. The browser plug-in can obtain the processing result returned by the server, determine the corresponding operation node, and operate the first page according to the processing result under the operation node to complete the target task.

[0110] Figure 4 FIG. 2 shows a structural diagram of a page interaction device according to an embodiment of the present disclosure. Figure 4 As shown, the device may include:

[0111] An operation module 401 is used to execute a first operation process on a first page of a browser in response to an operation of a user, wherein the first operation process is associated with a scenario of a target task;

[0112] A calling module 402 is used to call the server to process and return the processing result based on the first operation process, and the processing includes natural language processing;

[0113] The completion module 403 is used to complete the target task on the first page based on the processing result returned by the server.

[0114] In a possible implementation, calling module 402 is used to:

[0115] Based on the first operation process, determining the knowledge source information;

[0116] The structural information and knowledge source information of the first page are sent to the server, so that the server performs processing based on the structural information and knowledge source information of the first page to obtain a processing result.

[0117] In a possible implementation, based on the first operation process, determining the knowledge source information includes:

[0118] Based on the first operation process, opening a second page and / or a file uploaded by the user on the browser, the second page including a third-party web page;

[0119] The content in the second page and / or the file uploaded by the user is read as the knowledge source information.

[0120] In a possible implementation, the first operation flow includes one or more operation nodes, and the completion module 403 is used to:

[0121] Determine a target operation node in the first page according to the first operation process;

[0122] The target operation node is operated based on the processing result.

[0123] In a possible implementation, the server performs processing based on the structure information and knowledge source information of the first page to obtain a processing result, including:

[0124] The server runs the second operation process;

[0125] Based on the second operation flow, the server uses the large language model to process the structural information and knowledge source information of the first page to obtain a processing result.

[0126] In a possible implementation, the second operation flow includes a data processing method at the server and a type of large language model used.

[0127] In a possible implementation, the first page includes a form page, and operating the target operation node based on the processing result includes:

[0128] Fill the processing results into the form page corresponding to the target operation node.

[0129] In a possible implementation, the first page includes an account operation page, an operation node is associated with an account, and an operation is performed on a target operation node based on a processing result, including:

[0130] Based on the processing results, an operation is performed in the target account corresponding to the target operation node, and the operation includes any one or more of account information management, content management, and account interaction.

[0131] According to an embodiment of the present disclosure, by responding to the user's operation, a first operation process associated with the scenario of the target task is run on the first page of the browser, and a server is called based on the first operation process to perform processing and return a processing result, wherein the processing includes natural language processing, and the target task is completed on the first page based on the processing result returned by the server. This allows AI to be combined with RPA to give AI the ability to act, without the user having to operate the page according to the processing result. Instead, the browser and page are automatically operated according to the operation process to automatically apply the AI ​​processing results on the page, efficiently complete the target task, and improve work efficiency. The first operation process can be used to standardize work tasks so that this type of work can be automatically completed on the page, so that the task throughput is no longer limited by manpower, and the application scenarios can be expanded on a large scale.

[0132] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0133] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0134] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0135] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above method.

[0136] Figure 5 1 is a block diagram of an apparatus 1900 for page interaction according to an exemplary embodiment. For example, the apparatus 1900 may be provided as a terminal device. Figure 5 , the apparatus 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions, such as an application, that can be executed by the processing component 1922. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0137] The device 1900 may also include a power supply component 1926 configured to perform power management of the device 1900, a wired or wireless network interface 1950 configured to connect the device 1900 to a network, and an input / output interface 1958 (I / O interface). The device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server 2000. TM , MacOS X TM , Unix TM ,Linux TM , FreeBSD TM or similar.

[0138] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the device 1900 to perform the above method.

[0139] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0140] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0141] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0142] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0143] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0144] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0145] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0146] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0147] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A page interaction method, characterized in that: The method comprises: In response to the user's operation, a first operation process is executed on a first page of the browser, wherein the first operation process is associated with a scenario of a target task; Based on the first operation flow, calling the server to perform processing and return the processing result, wherein the processing includes natural language processing; Based on the processing result returned by the server, the target task is completed on the first page.

2. The method according to claim 1, characterized in that The calling of the server to process and return the processing result based on the first operation process includes: Based on the first operation process, determining knowledge source information; The structural information of the first page and the knowledge source information are sent to a server, so that the server performs processing based on the structural information of the first page and the knowledge source information to obtain a processing result.

3. The method according to claim 2, characterized in that The determining of knowledge source information based on the first operation process includes: Based on the first operation process, opening a second page and / or a file uploaded by the user on the browser, wherein the second page includes a third-party web page; The content of the second page and / or the file uploaded by the user is read as the knowledge source information.

4. The method according to claim 2, characterized in that: The first operation process includes one or more operation nodes, and completing the target task on the first page based on the processing result returned by the server includes: Determine a target operation node in the first page according to the first operation process; The target operation node is operated based on the processing result.

5. The method according to claim 2, characterized in that: The server performs processing based on the structural information of the first page and the knowledge source information to obtain a processing result, including: The server runs a second operation process; Based on the second operation process, the server uses a large language model to process the structural information of the first page and the knowledge source information to obtain a processing result.

6. The method according to claim 5, characterized in that The second operation process includes the data processing method of the server and the type of large language model used.

7. The method according to claim 4, characterized in that The first page includes a form page, and the operating the target operation node based on the processing result includes: The processing result is filled into the form page corresponding to the target operation node.

8. The method according to claim 4, characterized in that The first page includes an account operation page, the operation node is associated with an account, and the operation on the target operation node based on the processing result includes: Based on the processing result, an operation is performed in the target account corresponding to the target operation node, and the operation includes any one or more of account information management, content management, and account interaction.

9. A page interaction device, characterized in that: The device comprises: An operation module, configured to execute a first operation process on a first page of a browser in response to an operation of a user, wherein the first operation process is associated with a scenario of a target task; A calling module, configured to call a server to perform processing and return a processing result based on the first operation process, wherein the processing includes natural language processing; A completion module is used to complete the target task on the first page based on the processing result returned by the server.

10. A page interaction device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to implement the method described in any one of claims 1 to 8 when executing the instructions stored in the memory.

11. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images