Browser application operation method cooperating with intelligent marketing intelligent agent and related device

Abstract

The one or more cases provided in the present application provide a browser application operation method and related device cooperating with a smart marketing agent, the method is applied to a first process deployed in a first device, comprising: receiving a first request sent by a first agent, the first agent being a smart marketing agent; converting the first request into a first instruction sequence matched with a first browser application, and based on a reading permission, reading at least one browser instance of the first device from a local file of the first device, the at least one browser instance comprising a session state associated with the first browser application; based on the session state associated with the first browser application, executing the first instruction sequence on the first browser application to obtain an operation result of a first operation. In view of the problem that the automation operation stability of the browser application is poor, the accuracy and stability of the browser application operation are improved by performing the automation operation on the browser application in a real browser session environment.

Description

Technical Field

[0001] One or more of the above relate to a browser application operation method in collaboration with an intelligent marketing agent, a browser application operation device in collaboration with an intelligent marketing agent, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] A browser application (web application) can be understood as application software that can be accessed via the internet without requiring local client installation. In some scenarios, there is a need to automate the operations of browser applications; for example, when testing a browser application, there is a need to automate its operations.

[0003] Therefore, achieving accurate and stable automated operations for browser applications is particularly important. Summary of the Invention

[0004] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] This document provides at least one scenario for a browser application operation method in collaboration with an intelligent marketing agent, applied to a first process deployed in a first device, the first process having read permissions for local files on the first device. The method includes: receiving a first request sent by a first agent, wherein the first request is for requesting the execution of a first operation on a first browser application, the first agent being the intelligent marketing agent; converting the first request into a first instruction sequence matching the first browser application; and, based on the read permissions, reading at least one browser instance of the first device from local files on the first device, wherein the at least one browser instance includes a session state associated with the first browser application; and, based on the session state associated with the first browser application, executing the first instruction sequence on the first browser application to obtain the operation result of the first operation.

[0006] This document provides at least one scenario a browser application operation device that collaborates with an intelligent marketing agent. A first process deployed in a first device, having read permissions for local files on the first device, is included. The device comprises: a communication module configured to receive a first request sent by a first agent, wherein the first request requests the execution of a first operation on a first browser application, the first agent being the intelligent marketing agent; a processing module configured to convert the first request into a first instruction sequence matching the first browser application, and based on the read permissions, read at least one browser instance from local files on the first device, wherein the at least one browser instance includes a session state associated with the first browser application; and an execution module configured to execute the first instruction sequence on the first browser application based on the session state associated with the first browser application, to obtain the operation result of the first operation.

[0007] At least one embodiment of this document provides an electronic device comprising: at least one processor; and at least one memory including one or more computer program instructions; wherein the one or more computer program instructions are executed by the processor to perform a browser application operation method in collaboration with an intelligent marketing agent provided in at least one embodiment of this document.

[0008] At least one aspect of this document provides a computer-readable storage medium that non-transitory stores computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, a browser application operation method in collaboration with an intelligent marketing agent provided by at least one aspect of this document is implemented.

[0009] At least one aspect of this document provides a computer program product, including a computer program that, when executed by a processor, implements a browser application operation method in collaboration with an intelligent marketing agent, as provided in at least one aspect of this document.

[0010] In one of the browser application operation methods provided in this paper, which collaborates with an intelligent marketing agent, a first process resides on the terminal device. For requests sent by the agent to operate the browser application, the first process converts the request into a sequence of instructions executable within the browser application. Since the first process is deployed on the first device and has local file read permissions, it can also obtain a browser instance. Based on reusing the session state associated with the browser application, it executes the instruction sequence to automate the browser application's operations. Thus, the automated operations of the browser application are performed in a real browser session environment, improving the accuracy and stability of the automated operations. Furthermore, semantic requests generated by the agent (e.g., semantic requests related to intelligent marketing generated by the intelligent marketing agent) are automatically converted into atomic instruction sequences at the browser's underlying level, collaborating with the agent to efficiently drive the automated operation process of the browser application. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the various scenarios described herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0012] Figure 1 The illustration shows an application scenario diagram of a browser application operation method in collaboration with an intelligent marketing agent, provided in at least one of the cases described in this article.

[0013] Figure 2 The illustration shows a flowchart of a browser application operation method in collaboration with an intelligent marketing agent, provided in at least one scenario of this paper.

[0014] Figure 3 The illustration shows a flowchart of a browser application operation method in collaboration with an intelligent marketing agent, provided in at least one scenario of this paper.

[0015] Figure 4 The illustration shows a flowchart of a method for generating an instruction template, provided in at least one of the scenarios described herein.

[0016] Figure 5 The schematic diagram illustrates the structure of a browser application operating device that collaborates with an intelligent marketing agent, as provided in at least one scenario of this article.

[0017] Figure 6 A schematic diagram of the structure of an electronic device suitable for implementing at least one of the situations described herein is shown. Detailed Implementation

[0018] One or more scenarios described herein will now be described in more detail with reference to the accompanying drawings. While some scenarios are shown in the drawings, it should be understood that this document can be implemented in various forms and should not be construed as limited to the scenarios set forth herein; rather, these scenarios are provided to provide a more thorough and complete understanding of this document. It should be understood that the accompanying drawings and scenarios are for illustrative purposes only and are not intended to limit the scope of this document.

[0019] It should be understood that the steps described in the method embodiments herein may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one situation" means "at least one situation"; the term "another situation" means "at least one additional situation"; the term "some situations" means "at least some situations". Definitions of other terms will be given in the following description.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this article are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "one" and "more" used in this document are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of the messages or information exchanged between the various devices in the embodiments herein are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0024] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition, use, storage or deletion of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0025] It is understood that before using the technical solutions disclosed in each scenario in this article, relevant users should be informed of the type, scope of use, and usage scenarios of the information involved in this article and their authorization should be obtained through appropriate means in accordance with relevant laws and regulations. Relevant users may include any type of rights holder, such as individuals, enterprises, or groups.

[0026] For example, in response to receiving an active request from a user, a prompt message is sent to the relevant user to clearly inform the user that the requested operation will require obtaining and using the user's information, thereby enabling the relevant user to choose whether to provide information to the software or hardware such as electronic devices, applications, servers, or storage media that perform the operation of any of the technical solutions described herein.

[0027] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, such as a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide information to the electronic device.

[0028] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation method described in this article. Other methods that comply with relevant laws and regulations may also be applied to the implementation method described in this article.

[0029] A browser application can be understood as an application software that runs on a browser and is developed based on web technologies. Browser applications can be accessed over the Internet based on Hypertext Transfer Protocol (HTTP) or Hypertext Transfer Protocol Secure (HTTPS) without the need to install a client locally.

[0030] In some scenarios, there is a need to automate operations on browser applications. For example, when testing a browser application, there is a need to automate its operations; another example is when acquiring operational data from a browser application, there is a need to automate its operations.

[0031] In practical applications, the above-mentioned process of automating browser applications is usually implemented through script-driven automation programs. For example, firstly, the automation program can programmatically start a brand new, independent browser process (i.e., create a new browser instance). This browser process is an isolated session environment that does not include any history, cache, logged-in website sessions, or other session states. Then, the automation program sends a series of atomic operation instructions to the browser. After the operation instructions are executed, the browser process is closed and destroyed.

[0032] However, the above-mentioned method of automating browser applications through scripts has at least the following problems: Since the automation program starts a brand new, unmarked browser process, it often needs to go through complex login processes such as identity authentication during the execution of operation instructions, which reduces the success rate of automating browser applications.

[0033] Furthermore, with the continuous development of artificial intelligence technology, intelligent agents have emerged. An intelligent agent can be understood as a system centered around an artificial intelligence model (such as a large-scale AI model), possessing autonomous understanding, perception, planning, memory, and tool usage capabilities, and capable of performing complex tasks. How to apply intelligent agents to automate browser applications, and how to collaborate with intelligent agents to drive the entire process of automating browser application operations, has become a pressing issue.

[0034] To at least partially solve the aforementioned technical problem, this paper provides a browser application operation method in collaboration with an intelligent marketing agent in at least one scenario. This method is applied to a first process deployed in a first device, the first process having read permissions for local files on the first device. The method includes: receiving a first request sent by a first agent (an intelligent marketing agent) requesting the execution of a first operation on a first browser application; converting the first request into a first instruction sequence matching the first browser application; and, based on the read permissions, reading at least one browser instance from local files on the first device, the at least one browser instance including a session state associated with the first browser application; and, based on the session state associated with the first browser application, executing the first instruction sequence on the first browser application to obtain the operation result of the first operation.

[0035] Based on the browser application operation method for cooperating with intelligent marketing agents provided in at least one of the embodiments described herein, at least one of the embodiments described herein also provides a browser application operation device, electronic device, computer-readable storage medium, and computer program product for cooperating with intelligent marketing agents.

[0036] In one of the browser application operation methods provided in this paper, which collaborates with an intelligent marketing agent, a first process resides on the terminal device. For requests sent by the agent to operate the browser application, the first process converts the request into a sequence of instructions executable within the browser application. Since the first process is deployed on the first device and has local file read permissions, it can also obtain a browser instance. Based on reusing the session state associated with the browser application, it executes the instruction sequence to automate the browser application's operations. Thus, the automated operations of the browser application are performed in a real browser session environment, improving the accuracy and stability of the automated operations. Furthermore, semantic requests generated by the agent (e.g., semantic requests related to intelligent marketing generated by the intelligent marketing agent) are automatically converted into atomic instruction sequences at the browser's underlying level, collaborating with the agent to efficiently drive the automated operation process of the browser application.

[0037] The following detailed description, with reference to the accompanying drawings, illustrates one or more scenarios and some examples thereof.

[0038] Figure 1 The illustration shows an application scenario diagram of a browser application operation method in collaboration with an intelligent marketing agent, provided in at least one of the cases described in this article.

[0039] like Figure 1 As shown, the application scenario provided in this case may include a first device 11 and a first intelligent agent 12. The first device 11 can be various electronic devices used by users who have a need to automate the operation of browser applications. For example, the first device 11 can be a terminal device (e.g., a test machine) used for testing browser applications. The first device 11 can be a smart wearable device, smart appliance, smart car, mobile phone, tablet computer, laptop computer, or desktop computer, etc.

[0040] The first device 11 may include a first process 111, a browser instance 112, and a browser application 113. The first process 111 can interact with the browser instance 112 and communicate with the browser application 113. For example, the first process 111 may be deployed in the operating system of the first device 11, the browser instance 112 may be a browser instance generated in the browser of the first device 11 during the current or historical time period, and the browser application 113 may be any browser application running in the browser of the first device 11.

[0041] The first intelligent agent 12 can be an interactive, schedulable, and iterative task execution system. For example, the first intelligent agent 12 can rely on an artificial intelligence model (such as a large model) to achieve steps such as natural language intent parsing, long task decomposition, multi-step logical reasoning, and external tool collaboration. The first intelligent agent 12 can be of different types. For example, the first intelligent agent 12 can be an intelligent marketing agent, that is, in an intelligent marketing scenario, the intelligent marketing agent is used to automate the operation of browser applications.

[0042] The first device 11 can communicate with the first intelligent agent 12. For example, the first intelligent agent 12 can communicate with the first process 111 in the first device 11. That is, the first process 111 can act as a communication bridge between the first intelligent agent 12 and the browser application 113. Without direct communication between the first intelligent agent 12 and the browser application 113, automated operation of the browser application 113 driven by the first intelligent agent 12 can be realized.

[0043] For example, since the first intelligent agent 12 can divide the overall task objective into multiple sub-tasks during task execution, and different sub-tasks can be executed by calling different tools, the automated operation performed on the browser application 113 can be a sub-task of the first intelligent agent 12. The first process 111 can be used as a tool called by the first intelligent agent 12, and the first intelligent agent 12 and the first process 111 can realize the sub-task of automating the operation on the browser application 113.

[0044] For example, the browser application operation methods for collaborating with intelligent marketing agents provided in one or more scenarios of this paper can be implemented in software, hardware, firmware, or any combination thereof.

[0045] For example, the browser application operation method in collaboration with the intelligent marketing agent provided in one or more scenarios of this paper is applicable to the first process 111 in the first device 11. The first process 111 in the first device 11 can load and execute the browser application operation method in collaboration with the intelligent marketing agent. For example, the first process 111 in the first device 11 may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), neural network processing unit (NPU), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, storage units, etc. The first process 111 is also equipped with an operating system and various types of application programming interfaces (APIs), and implements the browser application operation method in collaboration with the intelligent marketing agent provided in one or more scenarios of this paper by running code or instructions.

[0046] The following will combine Figures 2 to 4 This paper provides a detailed description of a browser application operation method that collaborates with an intelligent marketing agent in at least one scenario.

[0047] Figure 2 The illustration shows a flowchart of a browser application operation method in collaboration with an intelligent marketing agent, provided in at least one scenario of this paper.

[0048] like Figure 2 As shown, the browser application operation method in collaboration with the intelligent marketing agent in this scenario includes steps S201 to S203. In some cases, the executing entity of this browser application operation method in collaboration with the intelligent marketing agent can be a first process deployed in the first device.

[0049] The first process can be understood as a process that runs resident on the first device. The first process can be configured to communicate with and operate on the browser application running on the first device.

[0050] One or more scenarios described herein do not restrict the way the first process is deployed on the first device. For example, first, write the program file for the first process, which may include a script file, a service file, an edited binary file, etc. Then, copy the program file of the first process to the file directory of the first device, and then execute a resident command (such as the nohup command or the systemd command) on the first device to make the first process run resident on the first device.

[0051] The first process has read permissions for local files on the first device; that is, the first process is able to read local files on the first device.

[0052] The steps for using this browser application are explained below:

[0053] Step S201: Receive the first request sent by the first intelligent agent.

[0054] The first intelligent agent can be understood as any intelligent agent connected to the first device. For example, the first intelligent agent can provide services in the form of cloud services, and the first device can deploy the cloud services to use the first intelligent agent on the first device.

[0055] The first request can be understood as a request sent by the first intelligent agent. The first request can be used to request the first browser application to perform a first operation.

[0056] For example, the first request may carry content in natural language form, which may be used to describe "performing a first operation on the first browser application", such as "searching for product A and adding it to the shopping cart".

[0057] The first browser application can be understood as any browser application that runs on the first device and has a need for automated operation. For example, the first browser application can be different types of browser applications, such as e-commerce browser applications, content sharing browser applications, etc.

[0058] The first operation can be understood as the operation that needs to be performed on the first browser application. For example, the first operation can be a page navigation operation, an element interaction operation, a data extraction operation, an input operation, etc. The first operation can have different operation types. For example, the operation type of the first operation can be a search operation, an order placement operation, a publishing operation, etc.

[0059] The first agent can generate the first request in different ways. For example, the first agent can receive a user request (e.g., a request to test the first browser application) and generate the first request based on that user request.

[0060] In some cases, the first intelligent agent can be an intelligent marketing agent, which can be understood as an agent that provides intelligent marketing services. For example, the intelligent marketing agent can provide intelligent marketing services to the first browser application. In the process of the intelligent marketing agent providing intelligent marketing services to the first browser application (for example, in the task of generating intelligent marketing strategies), there is a need to obtain operation data related to the first browser application. Therefore, the intelligent marketing agent can drive the first process to perform automated operations of the browser application.

[0061] For example, the intelligent marketing agent needs to generate an intelligent marketing strategy that is highly compatible with the first browser application based on the operation data related to the first browser application. Therefore, during the process of the intelligent marketing agent executing the task of generating the intelligent marketing strategy, the task of generating the intelligent marketing strategy can be planned into multiple sub-tasks. These sub-tasks may include a sub-task of obtaining operation data related to the first browser application and a sub-task of generating the intelligent marketing agent based on the operation data related to the first browser application. When the intelligent marketing agent executes the sub-task of obtaining operation data related to the first browser application, it generates a first request and sends the first request to the first process to drive the first process to operate on the first browser application running on the first device.

[0062] In other cases, the first intelligent agent can also be a testing intelligent agent. A testing intelligent agent can be understood as an intelligent agent that provides testing services for browser applications. For example, a testing intelligent agent can provide testing services for the first browser application. In the process of the testing intelligent agent providing testing services for the first browser application, there is a need to perform various testing operations on the first browser application. Therefore, the testing intelligent agent can also drive the first process to perform automated operations on the browser application.

[0063] Thus, in one or more scenarios described herein, the first device collaborates with a first intelligent agent (e.g., an intelligent marketing intelligent agent) to drive the execution of a first operation on a first browser application.

[0064] Step S202: Convert the first request into a first instruction sequence matching the first browser application, and read at least one browser instance of the first device from a local file of the first device based on read permissions.

[0065] The first instruction sequence matching the first browser application can be understood as an instruction sequence that can be executed in the first browser application to implement the first operation. In other words, after receiving the first request, the first process converts the high-level, semantic request generated by the first agent into the low-level, atomic operation instruction sequence of the first browser application, realizing the conversion between the upper-level semantic logic and the low-level browser control logic that conforms to the first browser application.

[0066] In this way, the conversion steps between the first request and the first instruction sequence are executed by the first process, which decouples the adaptation logic for the first browser application from the general browser control capabilities. The first intelligent agent does not need to pay attention to the conversion logic, thereby improving the success rate of automated operations on the first browser application.

[0067] Since the first process is deployed on the first device, and at least one browser instance is also stored on the first device, the first process can obtain at least one browser instance on the first device. At least one browser instance on the first device can be used to record the runtime data of browser applications running on the browser in the current and historical phases of the first device. In other words, at least one browser instance can be understood as a runtime container for browser applications, and as a complete browser runtime environment, it provides the runtime environment for browser applications to run on the first device.

[0068] A browser instance may include browser session state, which may include login state, cookies, session identifiers, site permissions, site configurations, login retention data, cached files, etc. In one or more scenarios described herein, at least one browser instance includes session state associated with the first browser application. For example, at least one browser instance includes the browser fingerprint of the first browser application, account information logged in to the first browser application, historical behavior data of the first browser application, etc.

[0069] In some cases, the first process can obtain all browser instances of the first device and then filter out at least one browser instance associated with the first browser application from all browser instances of the first device; in other cases, the first process can also first filter all browser instances of the first device and then obtain at least one browser instance associated with the first browser application from the filtered ones.

[0070] A browser instance can be stored as a local file on the first device. Since the first process has read permissions for the local files on the first device, it can read the local files on the first device. Therefore, the first process can use these read permissions to read at least one browser instance from the local files on the first device.

[0071] For example, the first process can send the file path of at least one browser instance of the first device to the operating system of the first device. The operating system of the first device can return a file descriptor to the first process. The first process uses the file descriptor to initiate a read operation. The operating system of the first device reads at least one browser instance of the first device into a memory buffer and then copies it to the first process. In this way, the first process can obtain at least one browser instance of the first device.

[0072] Thus, by establishing and maintaining a persistent control session with a local browser instance, including identity credentials and session state, through a first process resident on the first device, at least one browser instance on the first device can be reused during the operation of the first browser application. Under the premise of the session state associated with the first browser application, the first operation of the first request instruction issued by the first agent is executed, thereby realizing the reuse of browser application automation operations in the real browser session environment and improving the accuracy and stability of browser application automation operations.

[0073] Step S203: Based on the session state associated with the first browser application, execute the first instruction sequence on the first browser application to obtain the operation result of the first operation.

[0074] Executing the first instruction sequence on the first browser application based on the session state associated with the first browser application can be understood as: reusing the session state associated with the first browser application, executing the first instruction sequence on the first browser application in a real, long-term active browser session environment. That is, the browser session environment for automating the operation of the first browser application is highly consistent with the real browser session environment of the first device in the current or historical stage.

[0075] The result of the first operation can be understood as the relevant data generated after the first operation is performed on the first browser application. For example, the result of the first operation may include the page data and metadata of the first browser application after the first operation is performed (such as operation time, operation status, error code, error details, etc.).

[0076] In this way, while preserving the real session state of the browser on the first device, the first process automatically performs the first operation on the first browser application. During the execution of the first operation, there is no need to perform complex login steps and verification processes, thereby improving the success rate and stability of the automated operation tasks on the first browser application.

[0077] In some cases, the first process provides a first interface (API), in which case it receives a first request sent by the first agent, including: receiving the first request in response to the first agent's call to the first interface.

[0078] In other words, the first intelligent agent and the first process can communicate and interact through API calls. By having the first process provide the first interface, the first intelligent agent can quickly and efficiently send the first request to the first process by calling the API, thereby quickly and efficiently driving automated operations on the first browser application.

[0079] Thus, on the one hand, for the first process, the function of communicating and interacting with external intelligent agents is solidified and reused in the form of a first interface, providing a standardized interaction entry point for external intelligent agents and reducing the interaction complexity between the first process and the first intelligent agent; on the other hand, for the first intelligent agent, the process of the first intelligent agent driving the automated operation of the first browser application can be simplified to the simple step of "API call". That is, the first intelligent agent only needs to call the first interface of the first process to start the automated operation of the first browser application. The first intelligent agent does not need to perform complex steps and operations, reducing the coordination complexity and stability between the first device and the first intelligent agent.

[0080] Figure 3 The illustration shows a flowchart of a browser application operation method in collaboration with an intelligent marketing agent, provided in at least one scenario of this paper.

[0081] In some cases, such as Figure 3 As shown, after the first process receives the first request, it converts the first request into a first instruction sequence that matches the first browser application, including: matching the application information of the first browser application with a set of applications; in response to the first browser application matching the set of applications, converting the first request into a first instruction sequence based on the instruction template in the set of applications that matches the first browser application.

[0082] For example, the application set is configured to include instruction templates that are matched with at least one browser application, and each instruction template is matched with the technical implementation of the corresponding browser application.

[0083] In other words, the application set can be understood as a pre-configured set of instructions that corresponds to different browser applications. The application set provides matching instruction templates for different browser applications.

[0084] Because different browser applications have different technical implementation methods (such as front-end implementation, back-end implementation, front-end language, back-end language, etc.), the first process has difficulty in performing a unified instruction sequence conversion for different browser applications with different technical implementation methods. Therefore, by providing corresponding instruction templates for different browser applications to form a set of applications, the first process can match the browser application that needs to be automated (such as the first browser application) with the set of applications. From the set of applications, it can filter out the instruction template that matches the browser application that needs to be automated (such as the first browser application), and then use the instruction template that matches the browser application that needs to be automated (such as the first browser application) to perform instruction conversion.

[0085] Since the instruction template matching the browser application (e.g., the first browser application) that needs to perform automated operations matches the technical implementation of the browser application (e.g., the first browser application), the instruction sequence (e.g., the first instruction sequence) obtained by converting the instruction template matching the browser application (e.g., the first browser application) that needs to perform automated operations can be successfully executed in the browser application (e.g., the first browser application).

[0086] In this way, the first request sent by the first intelligent agent can be accurately mapped to a first instruction sequence that matches the first browser application. This provides highly matched, standardized instruction templates for different browser applications with different technical implementations, improving the success rate of automating operations on the first browser application. Furthermore, the first intelligent agent does not need to consider the specific technical implementation of the browser application that needs to be automated, reducing the complexity of automating operations on the browser application driven by the first intelligent agent.

[0087] In some possible implementations, the first request is converted into a first instruction sequence based on an instruction template that matches the first browser application in the set of applications. This includes: obtaining a first instruction template from the instruction templates that match the first browser application, and converting the first request into a first instruction sequence based on the first instruction template.

[0088] For example, the first instruction template can correspond to the operation type of the first operation.

[0089] In other words, the instruction templates that match each browser application include instruction templates corresponding to different operation types. For example, the instruction templates that match each browser application may include instruction templates corresponding to search type, instruction templates corresponding to order type, etc.

[0090] For a first operation that needs to be performed in the first browser application, by finding the instruction template corresponding to the operation type of the first operation from the instruction templates that match the first browser application, the first request can be accurately converted into a first instruction sequence using the instruction template corresponding to the operation type of the first operation, so that the first instruction sequence can instruct the first operation to be performed in the first browser application.

[0091] Thus, by providing different instruction templates for different operation types to various browser applications, the first request can be quickly, efficiently, and accurately converted into instructions, improving the accuracy of the first process in converting the semantic first request into an atomic sequence of instructions. Furthermore, providing different instruction templates based on operation type allows for the expansion of instruction templates that match various browser applications. When a browser application is updated, the instruction template corresponding to the operation type can be modified to match the updated browser application, improving compatibility.

[0092] Furthermore, in response to a mismatch between the first browser application and the set of applications, the system can also receive an instruction template sent by the first intelligent agent that matches the first browser application, and add the instruction template that matches the first browser application to the set of applications.

[0093] In other words, when the first process encounters an unfamiliar and unknown first browser application, since the set of applications does not include an instruction template that matches the first browser application, the first process cannot convert the first request into a first instruction sequence. In this case, the first process can send a response message to the first agent, such as information indicating that instruction sequence conversion cannot be performed because the set of applications does not contain an instruction template that matches the first browser application. The first agent can respond to this response message by dynamically generating a new instruction template for the first browser application and sending the generated instruction template to the first process. The first process then adds the generated instruction template to the set of applications.

[0094] Thus, on the one hand, by leveraging the artificial intelligence capabilities of the first intelligent agent, instruction templates matching unknown browser applications are automatically generated. For the first request, after adding the generated instruction template to the set of applications, the first process can use the generated instruction template matching the first browser application to perform instruction sequence conversion and complete the processing of the first request. On the other hand, for subsequent requests that the first intelligent agent may generate instructing automated operations on the first browser application, instruction templates matching the first browser application in the set of applications can be reused. In the automated operation scenario of the browser application, the autonomous learning and adaptability are improved, and the generalization ability of the automated operation of the browser application is enhanced.

[0095] Figure 4The schematic diagram illustrates a flowchart of a generation instruction template provided in at least one of the scenarios described herein.

[0096] like Figure 4 As shown, in some possible implementations, receiving an instruction template sent by a first intelligent agent that matches the first browser application includes: receiving a second request sent by the first intelligent agent; based on the second request, accessing at least one application page of the first browser application and obtaining page information of at least one application page; sending the page information of at least one application page to the first intelligent agent; and receiving an instruction template sent by the first intelligent agent that matches the first browser application.

[0097] For example, the second request can be used to request access to the first browser application; that is, the second request can be understood as a request sent by the first agent to request exploration of the unknown first browser application.

[0098] Since the first process can communicate with the first browser application, after receiving the second request, the first process can access the first browser application and obtain page information of at least one application page of the first browser application. For example, the page information of at least one application page can be the document object model (DOM) of at least one application page or the page structure information of at least one application page.

[0099] The first process can send page information of at least one application page to the first intelligent agent, so that the first intelligent agent can generate an instruction template that matches the first browser application based on the page information of at least one application page. That is, the instruction template that matches the first browser application can be generated by the first intelligent agent based on the page information of at least one application page.

[0100] For example, the first process can send at least one accessibility tree of an application page to the first agent. Since the accessibility tree of at least one application page is a simplified structure of the application pages of the first browser application, the first agent can use the accessibility tree of at least one application page to efficiently identify the interactive elements and interactive logic of the first browser application, reduce the analysis complexity of the technical implementation of the first browser application, and improve the compatibility with different technical implementations (such as front-end implementations).

[0101] For example, after the first process receives an instruction template that matches the first browser application sent by the first agent, the first process can also verify the instruction template that matches the first browser application to ensure the correctness of the instruction template that matches the first browser application.

[0102] In this way, by using the first process as a communication bridge between the first agent and the first browser application, the first process actually accesses the first browser application and then feeds back the page information of the application page of the first browser application to the first agent. This enables the first agent to analyze the page information of the application page based on the first browser application (for example, to analyze the technical implementation of the first browser application), dynamically generate instruction templates that match the first browser application, and use the artificial intelligence capabilities of the first agent to achieve autonomous adaptation and capability expansion for unknown browser applications.

[0103] In some cases, continue as Figure 3 As shown, after the first process reads at least one browser instance of the first device, it executes a first instruction sequence on the first browser application based on the session state associated with the first browser application, including: using a browser debugging protocol to send the session state associated with the first browser application and the first instruction sequence to the browser kernel, so that the browser kernel executes the first instruction sequence on the first browser application in the session state associated with the first browser application.

[0104] The browser debugging protocol can be understood as a standardized, two-way communication interface. Based on the browser debugging protocol, external programs can control and debug the browser kernel. In other words, the browser debugging protocol can be understood as a remote control interface for the browser kernel.

[0105] The first process can control the browser kernel of the first device based on the browser debugging protocol. The browser debugging protocol sends the session state associated with the first browser application and the first instruction sequence to the browser kernel. The browser kernel can execute the first instruction sequence on the first browser application in the session state associated with the first browser application. For example, when the first browser application involves login processes such as identity authentication, the first instruction sequence can be executed using the session state associated with the first browser application.

[0106] In this way, the first process can automate the operation of the first browser application. The first agent generates the first request representing the upper-level semantic logic, which is then precisely executed by the first process, thereby improving the success rate of the browser application's automated operation.

[0107] In some cases, continue as Figure 3 As shown, obtaining the operation result of the first operation includes: in response to a change in at least one application page of the first browser application, obtaining page data of at least one application page, and generating the operation result of the first operation based on the page data of at least one application page.

[0108] In other words, the first process can detect changes in the application page of the first browser application, collect page data of the application page of the first browser application (such as page change data or page result data), and generate the operation result of the first operation. For example, in addition to the page data of the application page, the operation result of the first operation can also include metadata (such as time consumption, error codes, etc.) during the execution of the first instruction sequence.

[0109] In this way, an observable, debuggable, and repeatable automation loop is formed during the automated operation of browser applications, reducing the probability of failure in the automated operation of browser applications.

[0110] In some cases, the result of the first operation can also be structured to obtain a structured result of the first operation, and then the structured result of the first operation can be sent to the first intelligent agent.

[0111] For example, the first process can format the unstructured result of the first operation into a structured result that can be directly consumed by the first agent, such as a Lightweight Data Interchange Format (JavaScript Object Notation, JSON) object.

[0112] In addition, the first process can also record end-to-end logs and metrics, and send the end-to-end logs and metrics to the first intelligent agent.

[0113] In this way, the first intelligent agent can perform end-to-end detection and measurement of the automated operation execution process of the browser application, providing transparent and reliable data support for performance optimization and error troubleshooting of the automated operation process of the browser application, and improving the traceability of the automated operation of the browser application.

[0114] Based on the browser application operation method provided in at least one aspect of this paper, a browser application operation device is also provided in at least one aspect of this paper. The following will be combined with... Figure 5 A detailed description of the browser application's operating mechanism is provided.

[0115] Figure 5 The diagram illustrates the structure of a browser application operating device that collaborates with an intelligent marketing agent, as provided in at least one of the scenarios described herein.

[0116] like Figure 5As shown, in this scenario, the browser application operating device 500, which collaborates with the intelligent marketing agent, is deployed in the first process of the first device. The first process has read permissions for local files on the first device. The browser application operating device 500, which collaborates with the intelligent marketing agent, includes a communication module 501, a processing module 502, and an execution module 503. For example, the communication module 501, processing module 502, and execution module 503 can be implemented using hardware (e.g., circuit) modules or software modules. The following scenarios are similar and will not be elaborated further. For example, the communication module 501, processing module 502, and execution module 503 can be implemented using a central processing unit (CPU), a general-purpose graphics processing unit (GPGPU), a graphics processing unit (GPU), a tensor processor (TPU), a field-programmable gate array (FPGA), or other processing units with data processing capabilities and / or instruction execution capabilities, along with corresponding computer instructions.

[0117] The communication module 501 is configured to receive a first request sent by a first intelligent agent, wherein the first request is used to request the execution of a first operation on a first browser application, and the first intelligent agent is the intelligent marketing intelligent agent. For example, the communication module 501 can be configured to execute step S201 described above; its specific implementation principle can be found in the relevant description of step S201, and will not be repeated here.

[0118] Processing module 502 is configured to: convert the first request into a first instruction sequence matching the first browser application, and, based on the read permission, read at least one browser instance of the first device from a local file of the first device, wherein the at least one browser instance includes a session state associated with the first browser application. For example, processing module 502 can be configured to execute step S202 described above; its specific implementation principle can be found in the relevant description of step S202, and will not be repeated here.

[0119] The execution module 503 is configured to: execute the first instruction sequence on the first browser application based on the session state associated with the first browser application, and obtain the operation result of the first operation. For example, the execution module 503 can be configured to execute step S203 described above; its specific implementation principle can be found in the relevant description of step S203, and will not be repeated here.

[0120] In at least one embodiment of this document, the processing module 502 is further configured to: match the application information of the first browser application with a set of applications, wherein the set of applications includes instruction templates that match at least one browser application, and each instruction template matches the technical implementation of the corresponding browser application; in response to the first browser application matching the set of applications, convert the first request into the first instruction sequence based on the instruction templates in the set of applications that match the first browser application.

[0121] In at least one of the embodiments described herein, the processing module 502 is further configured to: obtain a first instruction template from the instruction template matched by the first browser application, wherein the first instruction template corresponds to the operation type of the first operation; and convert the first request into the first instruction sequence based on the first instruction template.

[0122] In at least one of the embodiments described herein, the browser application operating device 500 that collaborates with the intelligent marketing agent further includes a construction module configured to: in response to a mismatch between the first browser application and the set of applications, receive an instruction template sent by the first agent that matches the first browser application; and add the instruction template that matches the first browser application to the set of applications.

[0123] In at least one embodiment of this document, the construction module is further configured to: receive a second request sent by the first agent, wherein the second request is for requesting access to the first browser application; based on the second request, access at least one application page of the first browser application and obtain page information of the at least one application page; send the page information of the at least one application page to the first agent; and receive an instruction template sent by the first agent that matches the first browser application, wherein the instruction template that matches the first browser application is generated by the first agent based on the page information of the at least one application page.

[0124] In at least one of the embodiments described herein, the first process provides a first interface, and the communication module 501 is further configured to receive the first request in response to a call operation by the first agent to the first interface.

[0125] In at least one embodiment of this document, the execution module 503 is further configured to: in response to a change in at least one application page of the first browser application, obtain page data of the at least one application page; and generate an operation result of the first operation based on the page data of the at least one application page.

[0126] In at least one embodiment of this document, the execution module 503 is further configured to: perform structured processing on the operation result of the first operation to obtain a structured operation result of the first operation; the communication module 501 is further configured to: send the structured operation result of the first operation to the first intelligent agent.

[0127] In at least one embodiment of this document, the execution module 503 is further configured to: use a browser debugging protocol to send the session state associated with the first browser application and the first instruction sequence to the browser kernel, so that the browser kernel executes the first instruction sequence on the first browser application in the session state associated with the first browser application.

[0128] It should be noted that, for clarity and brevity, this document does not present all the constituent units of the browser application operating device 500 that collaborates with the intelligent marketing agent in at least one scenario. To realize the necessary functions of the browser application operating device 500 that collaborates with the intelligent marketing agent, those skilled in the art can provide or set other constituent units (not shown) according to specific needs, and this document does not limit such possibilities.

[0129] The browser application operation device 500 that collaborates with the intelligent marketing agent provided in at least one aspect of this document and the browser application operation method that collaborates with the intelligent marketing agent provided in at least one aspect of this document are based on the same inventive concept and can achieve the same technical effect and the same technical purpose as the browser application operation method that collaborates with the intelligent marketing agent provided in at least one aspect of this document. For details, please refer to the relevant description above, which will not be repeated here.

[0130] This document also provides, in at least one embodiment, an electronic device including a processing device and a storage device, the storage device including one or more computer program modules; wherein the one or more computer program modules are stored in the storage device and configured to be executed by the processing device, the one or more computer program modules being used to implement the browser application operation method in collaboration with an intelligent marketing agent provided in any embodiment of this document.

[0131] For example, the processing device may be a processor, such as a central processing unit (CPU), digital signal processor (DSP), image processor (GPU), general-purpose graphics processor (GPGPU), or other form of processing unit with data processing capabilities and / or instruction execution capabilities. It may be a general-purpose processor or a dedicated processor and may control other components in the electronic device to perform the desired functions.

[0132] For example, the storage device may be a memory, which may include one or more computer program products. These computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processing device may execute these program instructions to implement the functions described in at least one of the embodiments herein (implemented by the processing device) and / or other desired functions. Various application programs and various data may also be stored on the computer-readable storage medium, which is not limited in the embodiments described herein.

[0133] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing at least one of the embodiments described herein. The terminal device in at least one embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), tablet computers (PADs), portable multimedia players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of at least one of the situations described herein.

[0134] like Figure 6 As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory 602 or a program loaded from storage device 608 into random access memory 603. Random access memory 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, read-only memory 602, and random access memory 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0135] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0136] In particular, according to one or more embodiments herein, the processes described in the above-referenced flowcharts can be implemented as computer software programs. For example, one or more embodiments herein include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via communication device 609, or installed from storage device 608, or installed from read-only memory 602. When the computer program is executed by processing device 601, it performs the functions defined in the methods of at least one embodiment herein.

[0137] The electronic device 600 provided in at least one aspect of this document and the browser application operation method in collaboration with the intelligent marketing agent provided in at least one aspect of this document are based on the same inventive concept. They can achieve the same technical effect and the same technical purpose as the browser application operation method in collaboration with the intelligent marketing agent provided in at least one aspect of this document. For details, please refer to the relevant descriptions above, which will not be repeated here.

[0138] It should be noted that the computer-readable medium described above can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this document, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.

[0139] The computer-readable storage medium provided in at least one aspect of this document and the browser application operation method in collaboration with the intelligent marketing agent provided in at least one aspect of this document are based on the same inventive concept and can achieve the same technical effect and the same technical purpose as the browser application operation method in collaboration with the intelligent marketing agent provided in at least one aspect of this document. For details, please refer to the relevant description above, which will not be repeated here.

[0140] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as the Hypertext Transfer Protocol (HTTP), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0141] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0142] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the aforementioned browser application operation method in collaboration with the intelligent marketing agent.

[0143] Computer program code for performing the operations described herein may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] One or more embodiments of this document also provide a computer program product comprising one or more computer instructions. When these computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in any of these embodiments are generated.

[0145] The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, or data center to another website, computer, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0146] When the computer program product is executed by a computer, the computer executes any of the aforementioned methods of the browser application operation method in collaboration with the intelligent marketing agent. The computer program product can be a software installation package; when any of the aforementioned methods of the browser application operation method in collaboration with the intelligent marketing agent is required, the computer program product can be downloaded and executed on the computer.

[0147] The computer program product provided in at least one of the embodiments described herein and the browser application operation method in collaboration with the intelligent marketing agent provided in at least one of the embodiments described herein are based on the same inventive concept and can achieve the same technical effect and the same technical purpose as the browser application operation method in collaboration with the intelligent marketing agent provided in at least one of the embodiments described herein. For details, please refer to the relevant descriptions above, which will not be repeated here.

[0148] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of the systems, methods, and computer program products according to the various scenarios described herein. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0149] The units or modules described in at least one of the scenarios herein can be implemented in software or hardware. The names of the units or modules do not, in some cases, constitute a limitation on the unit or module itself.

[0150] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0151] In the context of this document, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0152] Based on one or more scenarios described herein, Example 1 provides a browser application operation method for cooperating with an intelligent marketing agent, applied to a first process deployed on a first device, the first process having read permissions for local files on the first device, the method comprising:

[0153] Receive a first request sent by a first intelligent agent, wherein the first request is used to request the execution of a first operation on a first browser application, and the first intelligent agent is the intelligent marketing intelligent agent;

[0154] The first request is converted into a first instruction sequence matching the first browser application, and based on the read permission, at least one browser instance of the first device is read from a local file of the first device, wherein the at least one browser instance includes a session state associated with the first browser application;

[0155] Based on the session state associated with the first browser application, the first instruction sequence is executed on the first browser application to obtain the operation result of the first operation.

[0156] According to one or more scenarios described herein, Example 2 provides the method of converting the first request into a first instruction sequence matching the first browser application, as described in Example 1, including:

[0157] The application information of the first browser application is matched with a set of applications, wherein the set of applications includes instruction templates that match at least one browser application, and each instruction template matches the technical implementation method of the corresponding browser application.

[0158] In response to the first browser application matching the set of applications, the first request is converted into the first instruction sequence based on the instruction template in the set of applications that matches the first browser application.

[0159] According to one or more scenarios described herein, Example 3 provides the instruction template from Example 2 that matches the first browser application within the specified application set, converting the first request into the first instruction sequence, including:

[0160] Obtain a first instruction template from the instruction template matched by the first browser application, wherein the first instruction template corresponds to the operation type of the first operation;

[0161] Based on the first instruction template, the first request is converted into the first instruction sequence.

[0162] Depending on one or more scenarios described in this article, Example 4 provides the method from Example 2, and also includes:

[0163] In response to the first browser application not matching the set of applications, the system receives an instruction template sent by the first intelligent agent that matches the first browser application.

[0164] Add the instruction template that matches the first browser application to the set of applications.

[0165] According to one or more scenarios described herein, Example 5 provides a template for receiving instructions sent by the first agent that match the first browser application, as in Example 4, including:

[0166] Receive a second request sent by the first intelligent agent, wherein the second request is for requesting access to the first browser application;

[0167] Based on the second request, access at least one application page of the first browser application and obtain page information of the at least one application page;

[0168] Send page information of at least one application page to the first intelligent agent;

[0169] The system receives an instruction template sent by the first intelligent agent that matches the first browser application, wherein the instruction template matching the first browser application is generated by the first intelligent agent based on page information of the at least one application page.

[0170] According to one or more scenarios described herein, Example Six provides a first interface provided by a first process in any of Examples One through Five, wherein receiving a first request sent by the first agent includes:

[0171] In response to the first intelligent agent's call operation on the first interface, the first request is received.

[0172] According to one or more scenarios in this document, Example 7 provides the operation result of obtaining the first operation in any of Examples 1 to 5, including:

[0173] In response to a change in at least one application page of the first browser application, page data of the at least one application page is obtained;

[0174] The operation result of the first operation is generated based on the page data of the at least one application page.

[0175] Depending on one or more scenarios described herein, Example 8 provides a method from any of the examples in Examples 1 through 5, and also includes:

[0176] The result of the first operation is processed in a structured manner to obtain the structured result of the first operation;

[0177] The structured operation result of the first operation is sent to the first intelligent agent.

[0178] According to one or more scenarios described herein, Example 9 provides, based on the session state associated with the first browser application, the execution of the first instruction sequence on the first browser application, including:

[0179] Using a browser debugging protocol, the session state associated with the first browser application and the first instruction sequence are sent to the browser kernel, so that the browser kernel executes the first instruction sequence on the first browser application in the session state associated with the first browser application.

[0180] According to one or more scenarios described herein, Example 10 provides a browser application operating device that collaborates with an intelligent marketing agent, a first process deployed in a first device, the first process having read permissions for local files in the first device, the device comprising:

[0181] The communication module is configured to receive a first request sent by a first intelligent agent, wherein the first request is used to request the execution of a first operation on a first browser application, and the first intelligent agent is the intelligent marketing intelligent agent;

[0182] The processing module is configured to: convert the first request into a first instruction sequence matching the first browser application, and, based on the read permission, read at least one browser instance of the first device from a local file of the first device, wherein the at least one browser instance includes a session state associated with the first browser application;

[0183] The execution module is configured to: execute the first instruction sequence on the first browser application based on the session state associated with the first browser application, and obtain the operation result of the first operation.

[0184] According to one or more of the provisions of this document, Example 11 provides an electronic device comprising:

[0185] At least one processor; and

[0186] At least one memory, including one or more computer program instructions;

[0187] The browser application operation method provided herein for collaboration with intelligent marketing agents is executed by the processor when one or more computer program instructions are executed at least once.

[0188] According to one or more of the embodiments described herein, Example Twelve provides a computer-readable storage medium that non-transitory stores computer-readable instructions, wherein when the computer-readable instructions are executed by a processor, a browser application operation method in collaboration with an intelligent marketing agent provided by at least one of the embodiments described herein is implemented.

[0189] According to one or more of the embodiments described herein, Example Thirteen provides a computer program product including a computer program that, when executed by a processor, implements a browser application operation method in collaboration with an intelligent marketing agent provided in at least one of the embodiments described herein.

[0190] The above description is merely a preferred embodiment and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure herein is not limited to technical solutions formed by specific combinations of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed herein that have similar functions.

[0191] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain contexts, multitasking and parallel processing may be advantageous. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this paper. Certain features described in the context of a single case can also be implemented in combination within that single case. Conversely, various features described in the context of a single case can also be implemented individually or in any suitable sub-combination in multiple cases.

[0192] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A browser application operation method in collaboration with an intelligent marketing agent, applied to a first process deployed on a first device, the first process having read permissions for local files on the first device, the method comprising: Receive a first request sent by a first intelligent agent, wherein the first request is used to request the execution of a first operation on a first browser application, and the first intelligent agent is the intelligent marketing intelligent agent; The first request is converted into a first instruction sequence matching the first browser application, and based on the read permission, at least one browser instance of the first device is read from a local file of the first device, wherein the at least one browser instance includes a session state associated with the first browser application; Based on the session state associated with the first browser application, the first instruction sequence is executed on the first browser application to obtain the operation result of the first operation.

2. The method according to claim 1, wherein, The step of converting the first request into a first instruction sequence matching the first browser application includes: The application information of the first browser application is matched with a set of applications, wherein the set of applications includes instruction templates that match at least one browser application, and each instruction template matches the technical implementation method of the corresponding browser application. In response to the first browser application matching the set of applications, the first request is converted into the first instruction sequence based on the instruction template in the set of applications that matches the first browser application.

3. The method according to claim 2, wherein, The step of converting the first request into the first instruction sequence based on the instruction template matching the first browser application in the set of applications includes: Obtain a first instruction template from the instruction template matched by the first browser application, wherein the first instruction template corresponds to the operation type of the first operation; Based on the first instruction template, the first request is converted into the first instruction sequence.

4. The method according to claim 2, further comprising: In response to the first browser application not matching the set of applications, the system receives an instruction template sent by the first intelligent agent that matches the first browser application. Add the instruction template that matches the first browser application to the set of applications.

5. The method according to claim 4, wherein, The receiving of the instruction template sent by the first intelligent agent that matches the first browser application includes: Receive a second request sent by the first intelligent agent, wherein the second request is for requesting access to the first browser application; Based on the second request, access at least one application page of the first browser application and obtain page information of the at least one application page; Send page information of at least one application page to the first intelligent agent; The system receives an instruction template sent by the first intelligent agent that matches the first browser application, wherein the instruction template matching the first browser application is generated by the first intelligent agent based on page information of the at least one application page.

6. The method according to any one of claims 1 to 5, wherein, The first process provides the first interface. Receiving the first request sent by the first intelligent agent includes: In response to the first intelligent agent's call operation on the first interface, the first request is received.

7. The method according to any one of claims 1 to 5, wherein, The step of obtaining the result of the first operation includes: In response to a change in at least one application page of the first browser application, page data of the at least one application page is obtained; The operation result of the first operation is generated based on the page data of the at least one application page.

8. The method according to any one of claims 1 to 5, further comprising: The result of the first operation is processed in a structured manner to obtain the structured result of the first operation; The structured operation result of the first operation is sent to the first intelligent agent.

9. The method according to any one of claims 1 to 5, wherein, The step of executing the first instruction sequence on the first browser application based on the session state associated with the first browser application includes: Using a browser debugging protocol, the session state associated with the first browser application and the first instruction sequence are sent to the browser kernel, so that the browser kernel executes the first instruction sequence on the first browser application in the session state associated with the first browser application.

10. A browser application operating device that collaborates with an intelligent marketing agent, comprising a first process deployed in a first device, the first process having read permissions for local files in the first device, the device comprising: The communication module is configured to receive a first request sent by a first intelligent agent, wherein the first request is used to request the execution of a first operation on a first browser application, and the first intelligent agent is the intelligent marketing intelligent agent; The processing module is configured to: convert the first request into a first instruction sequence matching the first browser application, and, based on the read permission, read at least one browser instance of the first device from a local file of the first device, wherein the at least one browser instance includes a session state associated with the first browser application; The execution module is configured to: execute the first instruction sequence on the first browser application based on the session state associated with the first browser application, and obtain the operation result of the first operation.

11. An electronic device, comprising: At least one processor; as well as At least one memory, including one or more computer program instructions; The one or more computer program instructions are executed by the processor to perform the method according to any one of claims 1 to 9.

12. A computer-readable storage medium for non-transitory storage of computer-readable instructions, wherein, The method of any one of claims 1 to 9 is implemented when the computer-readable instructions are executed by a processor.

13. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1 to 9.

Images