Execution method and device of webpage automation task, electronic equipment and storage medium

Abstract

The invention relates to the field of artificial intelligence, and provides a webpage automation task execution method and device, electronic equipment and a storage medium. The method comprises the following steps: determining a plurality of sub-tasks and a plurality of sub-targets according to a webpage automation task; starting from the first sub-task, sequentially executing the plurality of sub-tasks, in the execution process of the current sub-task, determining and executing an operation sequence of the current sub-task according to the current sub-task, and verifying whether the execution result of the current sub-task accords with the sub-target of the current sub-task; and when the current subtask passes the verification, determining that the execution of the current subtask is finished. And when the verification of the current subtask is not passed, re-executing the operation sequence or re-determining and executing the subtask to realize error correction. According to the method, the execution of the webpage automation task is decomposed into the execution of a plurality of subtasks, the operation sequence is planned by taking the subtasks as units, the reasoning complexity is reduced, the error correction capability is improved, and the execution efficiency of the webpage automation task is improved.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more particularly to a method, apparatus, electronic device, and storage medium for performing web page automation tasks. Background Technology

[0002] Web page automation refers to a user-demand-driven task that automatically plans and executes web page operations based on the web page environment to achieve a goal. Since web page content often involves knowledge from multiple domains, traditional reinforcement learning-based methods cannot provide universal solutions. With the emergence of large language models, using large language models to perform web page automation tasks has become a preferred solution.

[0003] Executing automated tasks on web pages is crucial for the practical application of large language models in web environments and forms the foundation for cutting-edge large language model applications such as intelligent search. However, existing large language models still have shortcomings in planning operation sequences for executing automated tasks on web pages. These sequences are prone to errors and require frequent corrections, resulting in low efficiency. Summary of the Invention

[0004] In view of this, this disclosure proposes a method, apparatus, electronic device, and storage medium for executing web page automation tasks. This method decomposes the execution of web page automation tasks into the execution of multiple sub-tasks, plans the operation sequence on a sub-task basis, reduces reasoning complexity, thereby reducing the probability of errors in the operation sequence and improving the execution efficiency of web page automation tasks.

[0005] According to one aspect of this disclosure, a method for executing a web page automation task is provided. The method includes: determining multiple subtasks and multiple sub-targets based on the web page automation task, each sub-target corresponding to one subtask, the number of sub-targets being less than the number of subtasks; sequentially executing the multiple subtasks starting from the first subtask; during the execution of the current subtask, determining the operation sequence of the current subtask based on the current subtask, executing the operation sequence of the current subtask, and obtaining the execution result of the current subtask; when a sub-target corresponding to the current subtask exists, verifying whether the execution result of the current subtask matches the sub-target of the current subtask; when the verification of the current subtask passes, determining that the execution of the current subtask has ended; when no sub-target corresponding to the current subtask exists, determining that the execution of the current subtask has ended.

[0006] In one possible implementation, the execution result of the current subtask includes at least one of the following: the Uniform Resource Locator (URL) of the webpage, the execution trajectory, and the webpage content. Verifying whether the execution result of the current subtask matches its sub-target includes: selecting at least one of a first verification method, a second verification method, and a third verification method to verify whether the execution result of the current subtask matches its sub-target; if any selected verification method passes, the current subtask passes verification. The first verification method includes verifying the Uniform Resource Locator (URL) of the webpage obtained from the last executed operation and the sub-target corresponding to the current subtask. The first verification method passes when the URL of any webpage matches the subtarget corresponding to the current subtask, based on regular expression matching. The second verification method includes adding at least one of the execution trajectory and the webpage content, along with the verified case, to the verification prompt words of the large language model, and using the large language model to verify the execution trajectory based on the verification prompt words. The third verification method includes adding the webpage content obtained from the last executed operation and the verified case to the verification prompt words of the large language model, and using the large language model to verify the webpage content based on the verification prompt words.

[0007] In one possible implementation, the method further includes: when the current subtask fails verification, determining the first operation in the sequence of operations of all subtasks executed after the previously verified subtask that malfunctioned, re-executing the first malfunctioning operation and the other subsequent operations to obtain a new execution result for the current subtask; and re-verifying whether the execution result of the current subtask matches the sub-target of the current subtask.

[0008] In one possible implementation, the method further includes: when the current subtask fails verification, determining other subtasks to be executed after the previously verified subtask based on the web page automation task and the verified subtasks, and determining at least one sub-target corresponding to the redefined subtask based on the redefined subtasks; and executing the redefined subtasks sequentially starting from the first redefined subtask.

[0009] In one possible implementation, the method further includes: recording the execution trajectory of the current subtask during its execution; removing all recorded execution trajectories when the execution result of the current subtask is verified; and determining and executing the operation sequence of the current subtask based on the current subtask includes: determining and executing the first operation of the current subtask based on the current subtask; and determining and executing the (i+1)th operation of the current subtask based on the current subtask, the execution result of the i-th operation of the current subtask, and the recorded execution trajectories, where i is a positive integer.

[0010] In one possible implementation, determining multiple subtasks and multiple sub-goals based on the web page automation task includes: generating multiple subtasks and multiple sub-goals based on the web page automation task using a large language model, wherein the number of sub-goals is equal to the number of subtasks; determining the operation sequence of the current subtask based on the current subtask includes: generating the operation sequence of the current subtask based on the current subtask using the large language model.

[0011] In one possible implementation, determining multiple subtasks and multiple sub-goals based on the web page automation task includes: generating multiple subtasks using a large language model based on the web page automation task; when the name of any subtask is different from the name of a predefined subtask in a predefined subtask set, generating the corresponding sub-goal using the large language model; determining the operation sequence of the current subtask based on the current subtask includes: when the name of the current subtask is the same as the name of a predefined subtask in a predefined subtask set, determining the operation sequence of the predefined subtask as the operation sequence of the current subtask; when the name of the current subtask is different from the name of a predefined subtask in a predefined subtask set, generating the operation sequence of the current subtask based on the current subtask using the large language model.

[0012] According to another aspect of this disclosure, an execution apparatus for web page automation tasks is provided. The apparatus includes: a first determining module, configured to determine multiple subtasks and multiple sub-targets based on the web page automation task, wherein each sub-target corresponds to one subtask, and the number of sub-targets is less than the number of subtasks; an execution module, configured to execute the multiple subtasks sequentially starting from the first subtask, and during the execution of the current subtask, determine the operation sequence of the current subtask based on the current subtask, execute the operation sequence of the current subtask, and obtain the execution result of the current subtask; a verification module, configured to verify whether the execution result of the current subtask matches the sub-target of the current subtask when a sub-target corresponding to the current subtask exists; a second determining module, configured to determine that the execution of the current subtask has ended when the current subtask verification passes; and a third determining module, configured to determine that the execution of the current subtask has ended when a sub-target corresponding to the current subtask does not exist.

[0013] In one possible implementation, the execution result of the current subtask includes at least one of the following: the Uniform Resource Locator (URL) of the webpage, the execution trajectory, and the webpage content. Verifying whether the execution result of the current subtask matches its sub-target includes: selecting at least one of a first verification method, a second verification method, and a third verification method to verify whether the execution result of the current subtask matches its sub-target; if any selected verification method passes, the current subtask passes verification. The first verification method includes verifying the Uniform Resource Locator (URL) of the webpage obtained from the last executed operation and the sub-target corresponding to the current subtask. The first verification method passes when the URL of any webpage matches the subtarget corresponding to the current subtask, based on regular expression matching. The second verification method includes adding at least one of the execution trajectory and the webpage content, along with the verified case, to the verification prompt words of the large language model, and using the large language model to verify the execution trajectory based on the verification prompt words. The third verification method includes adding the webpage content obtained from the last executed operation and the verified case to the verification prompt words of the large language model, and using the large language model to verify the webpage content based on the verification prompt words.

[0014] In one possible implementation, the device further includes: a judgment module, configured to, when the current subtask fails verification, determine the operation that first erred in the sequence of operations of all subtasks executed after the previously verified subtask, and re-execute the operation that first erred and the other subsequent operations to obtain a new execution result of the current subtask; the verification module is also configured to re-verify whether the execution result of the current subtask matches the sub-target of the current subtask.

[0015] In one possible implementation, the apparatus further includes: a fourth determining module, configured to, when the current subtask fails verification, redetermine other subtasks to be executed after the previously verified subtask based on the web page automation task and the verified subtasks, and determine at least one sub-target corresponding to the redetermined subtasks based on the redetermined subtasks; the execution module is further configured to execute the redetermined subtasks sequentially, starting from the first subtask among the redetermined subtasks.

[0016] In one possible implementation, the device further includes: a recording module for recording the execution trajectory of the current subtask during its execution; and a removal module for removing all recorded execution trajectories when the execution result of the current subtask is verified to be successful. The step of determining and executing the operation sequence of the current subtask includes: determining and executing the first operation of the current subtask based on the current subtask; and determining and executing the (i+1)th operation of the current subtask based on the current subtask, the execution result of the ith operation of the current subtask, and the recorded execution trajectories, where i is a positive integer.

[0017] In one possible implementation, determining multiple subtasks and multiple sub-goals based on the web page automation task includes: generating multiple subtasks and multiple sub-goals based on the web page automation task using a large language model, wherein the number of sub-goals is equal to the number of subtasks; determining the operation sequence of the current subtask based on the current subtask includes: generating the operation sequence of the current subtask based on the current subtask using the large language model.

[0018] In one possible implementation, determining multiple subtasks and multiple sub-goals based on the web page automation task includes: generating multiple subtasks using a large language model based on the web page automation task; when the name of any subtask is different from the name of a predefined subtask in a predefined subtask set, generating the corresponding sub-goal using the large language model; determining the operation sequence of the current subtask based on the current subtask includes: when the name of the current subtask is the same as the name of a predefined subtask in a predefined subtask set, determining the operation sequence of the predefined subtask as the operation sequence of the current subtask; when the name of the current subtask is different from the name of a predefined subtask in a predefined subtask set, generating the operation sequence of the current subtask based on the current subtask using the large language model.

[0019] According to the web page automation task execution method of this disclosure, multiple subtasks and multiple sub-targets are first determined based on the web page automation task, such that each sub-target corresponds to a subtask. Then, starting from the first subtask, multiple subtasks are executed sequentially. During the execution of the current subtask, the operation sequence of the current subtask is determined based on the current subtask, and the operation sequence of the current subtask is executed to obtain the execution result of the current subtask. This decomposes the execution of the entire web page automation task into the execution of multiple subtasks, and the operation sequence is planned on a subtask basis, reducing the complexity of reasoning. When a sub-target corresponding to the current subtask exists, it is verified whether the execution result of the current subtask matches the sub-target of the current subtask. When the execution result of the current subtask passes the verification, the execution of the current subtask ends. Since determining the operation sequence of the subtask is simpler than determining the operation sequence of the web page automation task, the operation sequence of the subtask is also more reasonable than the operation sequence of the web page automation task, which reduces the probability of errors in the execution of the subtask operation sequence and improves the task execution efficiency.

[0020] Furthermore, this method can generate subtasks and operation sequences of subtasks from a large language model without using a predefined set of subtasks, reducing manual costs. It can also be applied to diverse web page automation tasks in different application scenarios, reducing the execution complexity of web page automation tasks.

[0021] Furthermore, when problems arise in the execution results of subtasks, this method can redetermine the operation sequence of the subtasks or redetermine the subtasks themselves, ensuring the accurate execution of web page automation tasks even in the absence of prior environmental knowledge.

[0022] Furthermore, since this method performs verification on a subtask-by-subtask basis, the execution trajectory required for a single verification is greatly reduced. By promptly removing the recorded execution trajectory when any subtask passes verification, the redundancy of the context can be significantly reduced.

[0023] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0024] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0025] Figure 1 This diagram illustrates the execution scheme of a web page automation task in the prior art.

[0026] Figure 2 A schematic diagram illustrating the execution scheme of the web page automation task in the prior art 2 is shown.

[0027] Figure 3 This illustrates an exemplary application scenario of a method for executing web page automation tasks according to embodiments of the present disclosure.

[0028] Figure 4a A schematic diagram illustrating the flow of a method for executing web page automation tasks according to an embodiment of the present disclosure is shown.

[0029] Figure 4b Examples of subtasks and subobjectives determined according to embodiments of this disclosure are shown.

[0030] Figure 4c Examples of subtasks and subobjectives determined according to embodiments of this disclosure are shown.

[0031] Figure 5 A schematic diagram illustrating the flow of a method for executing web page automation tasks according to an embodiment of the present disclosure is shown.

[0032] Figure 6 A schematic diagram illustrating the first erroneous operation determined according to an embodiment of this disclosure.

[0033] Figure 7 A schematic diagram illustrating the flow of a method for executing web page automation tasks according to an embodiment of the present disclosure is shown.

[0034] Figure 8 This diagram illustrates the redefinition of subtasks and subobjectives according to embodiments of the present disclosure.

[0035] Figure 9 A schematic diagram showing the structure of a web page automation device according to an embodiment of the present disclosure is provided.

[0036] Figure 10 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. Detailed Implementation

[0037] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0038] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0039] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0040] The following section introduces two existing technologies for executing web page automation tasks.

[0041] Figure 1 This diagram illustrates the execution scheme of a web page automation task in the prior art.

[0042] like Figure 1 As shown, existing technology 1 proposes using a large language model to directly decompose web page automation tasks into operation sequences composed of web page operations (such as clicks, input, and Enter). However, since large language models often lack prior knowledge of the environment, and web page automation tasks are very complex, many unreasonable operations may appear in the operation sequence, frequently leading to errors during execution and requiring frequent corrections. This results in low task execution efficiency for existing technology 1. Furthermore, determining which operation in the operation sequence caused the error requires analyzing numerous operations during correction, which is time-consuming and further reduces the task execution efficiency of existing technology 1.

[0043] Figure 2 A schematic diagram illustrating the execution scheme of the web page automation task in the prior art 2 is shown.

[0044] like Figure 2As shown, Existing Technology 2 proposes setting a predefined set of subtasks and determining the operation sequence of the subtasks in this set through manual annotation, thus minimizing the error probability during execution. When performing web page automation tasks, matching subtasks are searched from the predefined set and executed. However, Existing Technology 2 requires extensive manual annotation to obtain a suitable set of subtasks. Furthermore, the predefined set of subtasks is limited; for complex web page automation tasks, the set may lack highly matching subtasks. This results in fewer found subtasks, increasing task execution complexity and reducing efficiency. For example, if the requirement of a web page automation task is to find users who have posted the latest threads on a forum, the highly matching subtasks are "find forum," "find threads," and "find users." The absence of any one of these will significantly impact subsequent execution. For instance, if the subtask "find threads" is missing from the set, the subtask "find forum" will be executed immediately after the "find forum" subtask is completed, increasing the difficulty of executing the "find users" subtask. If the missing subtasks are created manually, the labor costs will increase further.

[0045] In view of this, this disclosure proposes a method, apparatus, electronic device, and storage medium for executing web page automation tasks. This method decomposes the execution of web page automation tasks into the execution of multiple sub-tasks, plans the operation sequence on a sub-task basis, reduces reasoning complexity, thereby reducing the probability of operational errors, improving the efficiency of locating erroneous operations, and improving the execution efficiency of web page automation tasks.

[0046] Furthermore, this method can generate subtasks and operation sequences of subtasks from a large language model without using a predefined set of subtasks, reducing manual costs. It can also be applied to diverse web page automation tasks in different application scenarios, reducing the execution complexity of web page automation tasks.

[0047] Furthermore, when problems arise in the execution results of subtasks, this method can redetermine the operation sequence of the subtasks or redetermine the subtasks themselves, ensuring the accurate execution of web page automation tasks even in the absence of prior environmental knowledge.

[0048] Furthermore, since this method performs verification on a subtask-by-subtask basis, the execution trajectory required for a single verification is greatly reduced. By promptly removing the recorded execution trajectory when any subtask passes verification, the redundancy of the context can be significantly reduced.

[0049] Figure 3 This illustrates an exemplary application scenario of a method for executing web page automation tasks according to embodiments of the present disclosure.

[0050] like Figure 3 As shown, the execution method for web page automation tasks can be performed by a first device. The first device can be a server. The second device can be a terminal device used by the user. The first device can communicate with the second device.

[0051] The second device can act as a client for intelligent agents such as intelligent search applications or intelligent task agents, while the first device can act as a server for such agents. The user inputs their requirements on the second device, which then transmits these requirements to the first device. The first device generates automated web page tasks based on the user's requirements. The first device executes these automated tasks using a large language model (not shown) set on it, obtaining the execution results, which are then returned to the second device. The user can view these results on the second device.

[0052] Those skilled in the art will understand that a large language model can also be configured on the second device, and the second device can also generate web page automation tasks according to user needs and execute the web page automation tasks using the large language model. This disclosure does not limit the entity executing the web page automation tasks.

[0053] Those skilled in the art will understand that the first or second device may also use methods other than the large language model to perform web page automation tasks, as long as they can achieve the corresponding functions of the web page automation task execution method of this disclosure. This disclosure does not limit whether the large language model must be used. For clarity, the following description uses the use of the large language model to perform web page automation tasks as an example.

[0054] Figure 4a A schematic diagram illustrating the flow of a method for executing web page automation tasks according to an embodiment of the present disclosure is shown. Figure 4b and Figure 4c Examples of subtasks and subobjectives determined according to embodiments of this disclosure are shown.

[0055] like Figure 4a As shown, in one possible implementation, the method includes:

[0056] Step S41: Based on the web page automation task, determine multiple subtasks and multiple sub-goals. Each sub-goal corresponds to one subtask, and the number of sub-goals is less than the number of subtasks.

[0057] Step S42: Starting from the first subtask, execute multiple subtasks sequentially. During the execution of the current subtask, determine the operation sequence of the current subtask based on the current subtask, execute the operation sequence of the current subtask, and obtain the execution result of the current subtask.

[0058] Step S43: If there is a sub-target corresponding to the current subtask, verify whether the execution result of the current subtask matches the sub-target of the current subtask;

[0059] Step S44: When the current subtask passes verification, the execution of the current subtask ends;

[0060] Step S45: If there is no sub-target corresponding to the current subtask, the execution of the current subtask ends.

[0061] For example, step S41 can be executed first, determining multiple subtasks and multiple sub-goals based on the web page automation task. A large language model can be used to first perform semantic parsing on the web page automation task to determine the user's needs, and then determine multiple subtasks so that, ideally, the execution result after executing multiple subtasks sequentially can meet the user's needs.

[0062] like Figure 4b and Figure 4c As shown, a webpage automation task could be: to obtain the number of comments with more downvotes than upvotes from the comments of users who posted the latest post in section A. The defined subtasks could include: subtask 1 "find section A", subtask 2 "find the latest posted post", ..., subtask N "summarize the final execution results" (N is an integer greater than 2).

[0063] Subtasks may encounter execution errors. To detect these errors promptly, a large language model can be used to further define sub-goals. Each sub-goal corresponds to a subtask, and the execution results of the corresponding subtasks can be verified based on these sub-goals.

[0064] For example, the sub-goal for subtask 1 "Find Section A" could be "The URL of the webpage contains 'Section A'". The sub-goal for subtask 2 "Find the latest posted post" could be "The content of the webpage includes the latest posted post". For any webpage automation task, the last subtask is usually "Summarize the final execution result". This subtask has high accuracy and does not require verification or further sub-goals. Therefore, the number of sub-goals can be less than the number of subtasks.

[0065] Verification can be performed on a subset of the remaining subtasks, further reducing the number of sub-objectives that need to be defined and lowering verification costs. Alternatively, corresponding sub-objectives can be defined for each subtask except the last one, in order to monitor the execution accuracy of more subtasks. This disclosure does not impose any limitation on the specific number of sub-objectives and subtasks.

[0066] After determining the subtasks and sub-goals, step S42 can be executed, starting with the first subtask and executing multiple subtasks sequentially. During the execution of the current subtask, the large language model can be used to determine the operation sequence of the current subtask based on the current subtask, and execute the operation sequence of the current subtask to obtain the execution result of the current subtask.

[0067] The subtask's operation sequence may include at least one operation on the webpage, such as clicking, typing, scrolling, or pressing Enter. Each operation also indicates the specific operation method, for example, in... Figure 4b In the example, the operation sequence for subtask 1, "Finding Section A," can include three actions: the first action can be a click, instructing the user to click "Forum"; the second action can be a scroll, instructing the user to scroll down; and the third action can be a click, instructing the user to click the element "Section A" on the webpage. The operation sequence for subtask 2, "Finding the Newest Posts," can also include three actions: the first action can be a click, instructing the user to click "Posts"; the second action can be a click, instructing the user to click "Sort Options"; and the third action can be a click, instructing the user to click "Sort by Time."

[0068] Those skilled in the art should understand that there can be multiple operation sequences determined based on the same subtask, and this disclosure does not limit the specific operations included in the operation sequence.

[0069] Large language models can either first determine the operation sequence and then execute the operations in the sequence sequentially, or execute each operation as soon as it is determined and determine the next operation based on the result of the previous operation. This disclosure does not impose any restrictions on the temporal relationship between determining and executing the operation sequence.

[0070] Next, depending on whether a sub-goal corresponding to the current subtask exists, either step S43 or step S45 can be executed. If a sub-goal exists, step S43 can be executed to verify whether the execution result of the current subtask matches its sub-goal. If the current subtask passes verification (i.e., the execution result matches its sub-goal), it indicates that the execution of the current subtask has not encountered an error, and step S44 can be executed to determine the end of the current subtask's execution and begin executing the next subtask. If no sub-goal exists for the current subtask, the execution result of the current subtask can be considered not to require verification; step S45 can be executed to determine the end of the current subtask's execution and begin executing the next subtask.

[0071] like Figure 4b As shown, besides subtask N, each of the other subtasks has a corresponding sub-objective. Before executing subtask N, the execution result of each subtask is verified. Figure 4c As shown, subtask 1 does not have a corresponding sub-target, while subtask 2 does have a corresponding sub-target. After executing subtask 1 and subtask 2 in sequence, the execution result of subtask 2 is verified.

[0072] There are various ways to verify whether the execution result of a subtask matches the sub-goal, and this disclosure does not limit this. Exemplary verification methods are given later.

[0073] And so on, each subtask is executed in the same way as above, until the last subtask is completed, at which point the execution result of the web page automation task can be obtained.

[0074] According to the web page automation task execution method of this disclosure, multiple subtasks and multiple sub-targets are first determined based on the web page automation task, such that each sub-target corresponds to a subtask. Then, starting from the first subtask, multiple subtasks are executed sequentially. During the execution of the current subtask, the operation sequence of the current subtask is determined based on the current subtask, and the operation sequence of the current subtask is executed to obtain the execution result of the current subtask. This decomposes the execution of the entire web page automation task into the execution of multiple subtasks, and the operation sequence is planned on a subtask basis, reducing the complexity of reasoning. When a sub-target corresponding to the current subtask exists, it is verified whether the execution result of the current subtask matches the sub-target of the current subtask. When the execution result of the current subtask passes the verification, the execution of the current subtask ends. Since determining the operation sequence of the subtask is simpler than determining the operation sequence of the web page automation task, the operation sequence of the subtask is also more reasonable than the operation sequence of the web page automation task, which reduces the probability of errors in the execution of the subtask operation sequence and improves the task execution efficiency.

[0075] The following describes an exemplary method for verifying the execution results of subtasks.

[0076] In one possible implementation, the execution result of the current subtask includes at least one of the following: the Uniform Resource Locator (URL) of the webpage, the execution trajectory, and the webpage content. Verifying whether the execution result of the current subtask matches the sub-target of the current subtask includes:

[0077] Select at least one of the first verification method, the second verification method, and the third verification method to verify whether the execution result of the current subtask matches the sub-target of the current subtask. If any of the selected verification methods passes the verification, the current subtask passes the verification.

[0078] The first verification method includes performing regular expression matching verification on the Uniform Resource Locator (URL) of the webpage obtained from the last executed operation and the subtarget corresponding to the current subtask. When the URL of any webpage matches the subtarget corresponding to the current subtask, the first verification method passes the verification.

[0079] The second verification method includes adding at least one of the execution trajectory and webpage content, as well as verified cases, to the verification prompt words of the large language model, and using the large language model to verify the execution trajectory based on the verification prompt words;

[0080] The third verification method includes adding the webpage content obtained from the last executed operation and the verified cases to the verification prompts of the large language model, and using the large language model to verify the webpage content based on the verification prompts.

[0081] For example, the execution result verification of the subtasks disclosed herein is a semantic-level verification, thus supporting diverse verification methods. This paper uses the first verification method, the second verification method, and the third verification method as examples.

[0082] The execution result of the current subtask may include at least one of the following: the Uniform Resource Locator (URL) of the webpage, the execution path, and the content of the webpage.

[0083] The URL can include the webpage name. For example, if the webpage name for section A is "Forum_Section A", then the URL of that webpage can include "Forum_Section A". Each operation in the current subtask is an operation on a webpage, therefore each operation can correspond to a separate URL. The execution track records the operations performed during the execution of the current subtask and their execution order. Webpage content refers to the elements included in the webpage, and the execution result of the current subtask can include the webpage content corresponding to each operation.

[0084] When verifying the execution result of the current subtask, an appropriate verification method can be selected based on the data type of the execution result. For example, at least one of the first verification method, the second verification method, and the third verification method can be selected. If any of the selected verification methods passes, the current subtask is verified successfully.

[0085] The first verification method involves performing regular expression matching on the Uniform Resource Locator (URL) of the webpage obtained from at least one of the last executed operations and the subtarget corresponding to the current subtask. The first verification method passes if the URL of any webpage matches the subtarget of the current subtask. If the subtarget is "the URL of the webpage contains 'section A'", then the first verification method can be used directly. Regular expression matching can be implemented using existing technologies and will not be elaborated upon here.

[0086] The second verification method includes adding at least one of the execution trajectory and webpage content, along with verified cases, to the verification prompts of the large language model. The large language model then verifies the execution trajectory based on the verification prompts. A verified case can be a case where the execution trajectory of a subtask has been verified and passed using the second verification method. This case includes the subtask, the corresponding sub-target, the verification prompts, and the verification process record. In this case, the large language model can verify at least one of the execution trajectory and webpage content in the verification prompts using the second verification method, based on the exemplary verification process of the verified cases in the verification prompts. Specific verification methods can be implemented based on existing technologies and will not be elaborated upon here.

[0087] The third verification method involves adding the webpage content obtained from the last executed operation and the verified cases to the verification prompts of the large language model, and then using the large language model to verify the webpage content based on the verification prompts. A verified case can be a case where the webpage content of a subtask has been verified and passed using the third verification method, including the subtask, the corresponding sub-target, the verification prompts, and the verification process record. In this case, the large language model can verify the webpage content in the verification prompts using the third verification method based on the exemplary verification process of the verified cases in the verification prompts. Specific verification methods can be implemented based on existing technologies and will not be elaborated here.

[0088] By setting multiple optional verification methods, the verification of the execution results of subtasks becomes more flexible.

[0089] The following describes how to correct problems in a subtask when the current subtask fails verification. Figure 5 A schematic diagram illustrating the flow of a method for executing web page automation tasks according to an embodiment of the present disclosure is shown.

[0090] like Figure 5 As shown, in one possible implementation, the method further includes:

[0091] Step S46: If the current subtask fails verification, determine the first operation that failed in the sequence of all subtasks executed after the previously verified subtask, and re-execute the first operation that failed and the other operations that followed to obtain a new execution result for the current subtask.

[0092] Re-execute step S43 to verify whether the execution result of the current subtask matches the sub-target of the current subtask.

[0093] For example, if the current subtask fails to pass verification, it means that the current subtask has encountered an error. The reason may be that there is a problem in the execution sequence of the current subtask, or there may be a problem in the execution of other subtasks that were executed after the previously verified subtask and before the current subtask.

[0094] In this case, a partial re-evaluation of the operation sequence can be performed. Specifically, step S46 first identifies the operation that first erred in the sequence of all subtasks executed after the previously validated subtask. Then, the operation that first erred, along with the subsequent operations, is re-executed to obtain a new execution result for the current subtask. When re-executing the operation that first erred, the large language model's analysis of the error cause from the previous execution can be used as an error example. This avoids using the same execution method as before, reducing the probability of further errors during re-execution. Then, step S43 is executed to re-verify whether the execution result of the current subtask matches its sub-objective.

[0095] Figure 6 A schematic diagram illustrating the first erroneous operation determined according to an embodiment of this disclosure.

[0096] like Figure 6 As shown, during step S41, subtasks 11 through 14 were determined. Subtasks 11 through 13 have already been executed, while subtask 14 has not yet been executed. Subtask 13 is the current subtask. Subtask 11 has been verified successfully. Subtask 12 has no corresponding sub-target and therefore has not been verified. Subtask 13 has a corresponding sub-target but has not been verified. Therefore, all subtasks executed after the previously verified subtask include subtasks 12 and 13.

[0097] At this point, we can start by analyzing the first operation (operation 4) in the operation sequence of subtask 12 and then sequentially analyze which operation in the operation sequences of subtask 12 and subtask 13 resulted in an error. The specific analysis of the erroneous operation can be implemented using existing technology and will not be elaborated upon here. In this case, the analyzed operation is the first erroneous operation in the operation sequence of all subtasks executed after the previously verified subtask.

[0098] For example, such as Figure 6 As shown, the operation sequence of subtask 11 includes operations 1-3, the operation sequence of subtask 12 includes operations 4-5, the operation sequence of subtask 13 includes operations 6-8, and the operation sequence of subtask 14 includes operation 9. Assuming that operation 4 is the first operation to fail, operations 4-8 are re-executed. The execution result of operation 8 is the new execution result of subtask 13. Then, the new execution result of subtask 13 is re-verified to ensure it matches the sub-objective of subtask 13.

[0099] If the current subtask still fails verification after re-verification, the problem can be corrected through other means (as described in step S47 below). Alternatively, steps S51 and S43 can be repeated. A first threshold can be preset; if the number of times steps S51 and S43 are repeated during the execution of any subtask reaches the first threshold, then the problem can be corrected through other means. The value of the first threshold can be set according to the application scenario requirements; this disclosure does not limit the specific value of the first threshold.

[0100] In this way, even when there is a lack of prior environmental knowledge, subtasks can be automatically corrected, ensuring the accurate execution of web page automation tasks.

[0101] Figure 7 A schematic diagram illustrating the flow of a method for executing web page automation tasks according to an embodiment of the present disclosure is shown.

[0102] like Figure 7 As shown, in one possible implementation, the method further includes:

[0103] Step S47: When the current subtask fails to pass verification, based on the web automation task and the verified subtasks, redetermine the previously verified subtasks and other subtasks executed after the previously verified subtasks, and based on the redetermined subtasks, determine at least one sub-target corresponding to the redetermined subtasks.

[0104] Re-execute step S42, starting with the first of the newly determined subtasks, and execute the newly determined subtasks sequentially.

[0105] For example, the reason why the current subtask failed verification may be that the current subtask is poorly designed, or that other subtasks executed after the previously verified subtask but before the current subtask are poorly designed.

[0106] In this case, the operation sequence can be globally replanned, i.e., step S47 is executed, which redetermines the other subtasks to be executed after the previously verified subtasks, based on the web page automation task and the verified subtasks. Based on the redetermined subtasks, at least one sub-goal corresponding to the redetermined subtasks can be determined; then, starting from the first of the redetermined subtasks, the redetermined subtasks are executed sequentially.

[0107] Figure 8 This diagram illustrates the redefinition of subtasks and subobjectives according to embodiments of the present disclosure.

[0108] like Figure 8 As shown, during step S41, subtasks 11 through 14 were determined. Subtasks 11 through 13 have already been executed, while subtask 14 has not yet been executed. Subtask 13 is the current subtask. Subtask 11 has been verified successfully. Subtask 12 has no corresponding sub-target and therefore has not been verified. Subtask 13 has a corresponding sub-target but has not been verified. Therefore, the subtask that passed verification previously is subtask 11.

[0109] At this point, based on the web page automation task and the already validated subtask 11, the subtasks to be executed after subtask 11 can be redefined, such as subtasks 15-18. At least one sub-goal corresponding to subtasks 15-18 can be determined, for example, the sub-goal corresponding to subtask 16 and the sub-goal corresponding to subtask 17. Then, starting from subtask 15, subtasks 15-18 are executed sequentially.

[0110] Step S47 can be repeated until any new subtask passes verification.

[0111] This approach enhances the flexibility of automatic error correction for subtasks.

[0112] Those skilled in the art should understand that more subtask correction methods can be selected in practical applications, as long as the subtask problem can be corrected so that the execution result of the subtask passes verification. The embodiments of this disclosure do not limit the subtask correction methods.

[0113] The following describes an exemplary manner in which this disclosure defines multiple subtasks and multiple subobjectives.

[0114] In one possible implementation, based on the web page automation task, multiple subtasks and multiple sub-goals are identified, including:

[0115] The large language model is used to generate multiple sub-tasks and multiple sub-goals based on the web page automation task, and the number of sub-goals is equal to the number of sub-tasks.

[0116] Determine the operation sequence of the current subtask based on the current subtask, including:

[0117] The large language model is used to generate the operation sequence of the current subtask based on the current subtask.

[0118] For example, without using a predefined set of subtasks, a generative task approach can be adopted. A large language model can generate multiple subtasks based on the webpage automation task. In this case, the generated subtasks are more adaptable to the application scenario. Since the execution accuracy of each subtask is unknown, the large language model can simultaneously generate corresponding sub-objectives for each subtask. That is, the number of sub-objectives equals the number of subtasks. Sub-objectives can be generated using a classification task approach, that is, based on the available validation methods, a sub-objective corresponding to at least one validation method is determined. For example, the validation methods mentioned above include validation of webpage content; therefore, the sub-objective could be set to the final webpage being "the user's personal page".

[0119] The generation of subtasks and sub-goals for large language models can be achieved based on existing technologies, which will not be elaborated here.

[0120] Similarly, since a predefined set of subtasks is not used, the operation sequence for each subtask is also unknown. When determining the operation sequence for the current subtask, a large language model can be used to generate the operation sequence for the current subtask based on the current subtask.

[0121] The semantic description of the subtask can be incorporated into the prediction prompts of the large language model. These prompts also include the webpage environment and execution trajectory. The large language model can then generate the operation sequence of the subtask based on the content of the prediction prompts. The specific method for generating the operation sequence can be implemented using existing technologies, and will not be elaborated upon here.

[0122] In this way, the method disclosed herein can avoid using predefined subtask sets, reduce manual costs, and be applicable to diverse web page automation tasks in different application scenarios, thereby reducing the execution complexity of web page automation tasks.

[0123] The following describes another exemplary way of defining multiple subtasks and multiple subobjectives in this disclosure.

[0124] In one possible implementation, based on the web page automation task, multiple subtasks and multiple sub-goals are identified, including:

[0125] Use a large language model to generate multiple subtasks based on web page automation tasks;

[0126] If the name of any subtask is different from the name of a predefined subtask in the predefined subtask set, the large language model is used to generate the subtarget corresponding to that subtask.

[0127] Determine the operation sequence of the current subtask based on the current subtask, including:

[0128] If the name of the current subtask is the same as the name of a predefined subtask in the predefined subtask set, the operation sequence of that predefined subtask is determined as the operation sequence of the current subtask.

[0129] When the name of the current subtask is different from the name of the predefined subtask in the predefined subtask set, the large language model is used to generate the operation sequence of the current subtask based on the current subtask.

[0130] For example, given a predefined set of subtasks, a large language model can first generate multiple subtasks based on the web page automation task, and then determine whether to continue generating sub-targets corresponding to the subtasks based on the predefined set of subtasks.

[0131] In a predefined set of subtasks, the operation sequence of each predefined subtask is already determined, and the execution accuracy of each predefined subtask is known and relatively high. It's understandable that if the name of a subtask generated by the large language model is the same as the name of a predefined subtask, then the execution processes of these two subtasks with the same name can be considered similar, and the operation sequence of the predefined subtask can be directly used as the operation sequence of the subtask generated by the large language model. That is, when the name of the current subtask is the same as the name of a predefined subtask in the predefined set, the operation sequence of that predefined subtask is used as the operation sequence of the current subtask; when the name of the current subtask is different from the name of a predefined subtask in the predefined set, the large language model is used to generate the operation sequence of the current subtask based on the current subtask. An exemplary method for generating the operation sequence of the current subtask using the large language model has been described above and will not be repeated here.

[0132] For subtasks with known and high accuracy of execution results, validation is unnecessary, and therefore, generating corresponding sub-objectives is not required. For subtasks with unknown accuracy of execution results, a large language model can be used to further generate sub-objectives corresponding to the subtasks, and the execution results of the subtasks can then be validated based on these sub-objectives. In other words, if the name of any subtask differs from the name of a predefined subtask in a predefined set of subtasks, a large language model is used to generate the corresponding sub-objective for that subtask.

[0133] By utilizing a predefined set of subtasks to determine subtasks and their operation sequences, and by reducing the number of subtargets that need to be generated, the inference cost of large language models can be reduced.

[0134] The following describes an exemplary execution method for the operation sequence of subtasks.

[0135] In one possible implementation, the method further includes:

[0136] During the execution of the current subtask, record the execution trajectory of the current subtask;

[0137] Once the execution result of the current subtask passes verification, remove all recorded execution traces.

[0138] Determine the operation sequence of the current subtask based on the current subtask, and execute the operation sequence of the current subtask, including:

[0139] Based on the current subtask, determine and execute the first operation of the current subtask;

[0140] Based on the current subtask, the execution result of the i-th operation of the current subtask, and the recorded execution trajectory, determine the (i+1)-th operation of the current subtask and execute it, where i is a positive integer.

[0141] For example, during the execution of the current subtask, the execution trajectory of the current subtask can be recorded and stored for later use.

[0142] Execution trajectories can be used to determine the sequence of operations. For example, a large language model determines the sequence of operations for the current subtask and executes it. This can be done by determining and executing the first operation of the current subtask, or by determining and executing the (i+1)th operation of the current subtask based on the current subtask, the execution result of the i-th operation of the current subtask, and the recorded execution trajectory, where i is a positive integer. The specific methods by which the large language model determines operations can be implemented based on existing technologies and will not be elaborated here.

[0143] If the execution result of the current subtask fails verification, or if the current subtask is a subtask between a previously verified subtask and a subtask that failed verification this time, then the execution trajectory of the current subtask can be used in step S46 to analyze the operation that caused the error and in step S47 to redetermine the subtask and operation sequence. Therefore, the recorded execution trajectory remains valid. When the current subtask passes verification, the recorded execution trajectory becomes invalid and can be removed.

[0144] By promptly removing recorded execution traces when any subtask passes verification, the stored execution traces only include the execution traces of a small number of subtasks. Compared to existing technologies that store the execution traces of the entire web page automation task, the amount of context that needs to be stored is greatly reduced, which can significantly reduce the redundancy of the context.

[0145] This disclosure also proposes an execution device for web page automation tasks. Figure 9 A schematic diagram showing the structure of a web page automation device according to an embodiment of the present disclosure is provided.

[0146] like Figure 9 As shown, in one possible implementation, the device includes:

[0147] The first determining module 91 is used to determine multiple sub-tasks and multiple sub-targets based on the web page automation task, wherein each sub-target corresponds to one sub-task, and the number of sub-targets is less than the number of sub-tasks;

[0148] The execution module 92 is used to execute the multiple subtasks sequentially, starting from the first subtask. During the execution of the current subtask, the operation sequence of the current subtask is determined according to the current subtask, the operation sequence of the current subtask is executed, and the execution result of the current subtask is obtained.

[0149] Verification module 93 is used to verify whether the execution result of the current subtask matches the subtask's subtask when a subtask corresponding to the current subtask exists.

[0150] The second determining module 94 is used to determine the end of the execution of the current subtask when the current subtask passes the verification.

[0151] The third determining module 95 is used to determine the end of the current subtask if there is no corresponding subtarget for the current subtask.

[0152] In one possible implementation, the execution result of the current subtask includes at least one of the following: the Uniform Resource Locator (URL) of the webpage, the execution trajectory, and the webpage content. Verifying whether the execution result of the current subtask matches its sub-target includes: selecting at least one of a first verification method, a second verification method, and a third verification method to verify whether the execution result of the current subtask matches its sub-target; if any selected verification method passes, the current subtask passes verification. The first verification method includes verifying the Uniform Resource Locator (URL) of the webpage obtained from the last executed operation and the sub-target corresponding to the current subtask. The first verification method passes when the URL of any webpage matches the subtarget corresponding to the current subtask, based on regular expression matching. The second verification method includes adding at least one of the execution trajectory and the webpage content, along with the verified case, to the verification prompt words of the large language model, and using the large language model to verify the execution trajectory based on the verification prompt words. The third verification method includes adding the webpage content obtained from the last executed operation and the verified case to the verification prompt words of the large language model, and using the large language model to verify the webpage content based on the verification prompt words.

[0153] In one possible implementation, the device further includes: a judgment module, configured to, when the current subtask fails verification, determine the operation that first erred in the sequence of operations of all subtasks executed after the previously verified subtask, and re-execute the operation that first erred and the other subsequent operations to obtain a new execution result of the current subtask; the verification module is also configured to re-verify whether the execution result of the current subtask matches the sub-target of the current subtask.

[0154] In one possible implementation, the apparatus further includes: a fourth determining module, configured to, when the current subtask fails verification, redetermine other subtasks to be executed after the previously verified subtask based on the web page automation task and the verified subtasks, and determine at least one sub-target corresponding to the redetermined subtasks based on the redetermined subtasks; the execution module is further configured to execute the redetermined subtasks sequentially, starting from the first subtask among the redetermined subtasks.

[0155] In one possible implementation, the device further includes: a recording module for recording the execution trajectory of the current subtask during its execution; and a removal module for removing all recorded execution trajectories when the execution result of the current subtask is verified to be successful. The step of determining and executing the operation sequence of the current subtask includes: determining and executing the first operation of the current subtask based on the current subtask; and determining and executing the (i+1)th operation of the current subtask based on the current subtask, the execution result of the ith operation of the current subtask, and the recorded execution trajectories, where i is a positive integer.

[0156] In one possible implementation, determining multiple subtasks and multiple sub-goals based on the web page automation task includes: generating multiple subtasks and multiple sub-goals based on the web page automation task using a large language model, wherein the number of sub-goals is equal to the number of subtasks; determining the operation sequence of the current subtask based on the current subtask includes: generating the operation sequence of the current subtask based on the current subtask using the large language model.

[0157] In one possible implementation, determining multiple subtasks and multiple sub-goals based on the web page automation task includes: generating multiple subtasks using a large language model based on the web page automation task; when the name of any subtask is different from the name of a predefined subtask in a predefined subtask set, generating the corresponding sub-goal using the large language model; determining the operation sequence of the current subtask based on the current subtask includes: when the name of the current subtask is the same as the name of a predefined subtask in a predefined subtask set, determining the operation sequence of the predefined subtask as the operation sequence of the current subtask; when the name of the current subtask is different from the name of a predefined subtask in a predefined subtask set, generating the operation sequence of the current subtask based on the current subtask using the large language model.

[0158] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0159] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0160] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0161] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0162] Figure 10 A block diagram of an electronic device 1900 according to an embodiment of the present disclosure is shown. For example, the electronic device 1900 may be provided as a server or a terminal device. (Refer to...) Figure 10 The electronic device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0163] Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input / output interface 1958 (I / O interface). Electronic device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0164] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of an electronic device 1900 to perform the above-described method.

[0165] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0166] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0167] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0168] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0169] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0170] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0171] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0172] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0173] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for performing a web automation task, the method comprising: The method comprises: determining a plurality of sub-tasks and a plurality of sub-goals according to the web automation task, each sub-goal corresponding to a sub-task, the number of sub-goals being less than the number of sub-tasks; starting from the first sub-task, sequentially executing the plurality of sub-tasks, in the execution process of the current sub-task, determining an operation sequence of the current sub-task according to the current sub-task, executing the operation sequence of the current sub-task to obtain an execution result of the current sub-task; when there is a sub-goal corresponding to the current sub-task, verifying whether the execution result of the current sub-task is consistent with the sub-goal of the current sub-task; when the current sub-task passes the verification, determining that the execution of the current sub-task ends; when there is no sub-goal corresponding to the current sub-task, determining that the execution of the current sub-task ends.

2. The method of claim 1, wherein, The execution result of the current sub-task comprises at least one of a uniform resource locator (URL) of a web page, an execution track, and web page content, and the verification of whether the execution result of the current sub-task is consistent with the sub-goal of the current sub-task comprises: selecting at least one of a first verification method, a second verification method, and a third verification method to verify whether the execution result of the current sub-task is consistent with the sub-goal of the current sub-task, and when any one of the selected verification methods passes the verification, the current sub-task passes the verification; the first verification method comprises performing regular expression matching verification on the uniform resource locator (URL) of the web page obtained by the last executed operation and the sub-goal corresponding to the current sub-task, and when any one of the uniform resource locators (URLs) of the web pages matches the sub-goal corresponding to the current sub-task, the first verification method passes the verification; the second verification method comprises adding at least one of the execution track and the web page content and a verified case to the verification prompt words of a large language model, and using the large language model to verify the execution track according to the verification prompt words; the third verification method comprises adding the web page content obtained by the last executed operation and a verified case to the verification prompt words of a large language model, and using the large language model to verify the web page content according to the verification prompt words.

3. The method of claim 1, wherein, The method further comprises: when the current sub-task does not pass the verification, determining the first erroneous operation in the operation sequence of all sub-tasks executed after the last sub-task that passed the verification, re-executing the first erroneous operation and other operations thereafter to obtain a new execution result of the current sub-task; re-verifying whether the execution result of the current sub-task is consistent with the sub-goal of the current sub-task.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: when the current sub-task does not pass the verification, re-determining other sub-tasks executed after the last sub-task that passed the verification according to the web automation task and the sub-tasks that have passed the verification, and determining at least one sub-goal corresponding to the re-determined sub-tasks according to the re-determined sub-tasks; starting from the first sub-task in the re-determined sub-tasks, sequentially executing the re-determined sub-tasks.

5. The method of claim 2, wherein, The method further comprises: during the execution of the current sub-task, recording the execution track of the current sub-task; when the execution result of the current sub-task passes the verification, removing all recorded execution tracks. The operation sequence of the current subtask is determined according to the current subtask, and the operation sequence of the current subtask is executed, including: The first operation of the current subtask is determined and executed according to the current subtask; The i+1th operation of the current subtask is determined and executed according to the current subtask, the execution result of the i th operation of the current subtask, and the recorded execution track, i is a positive integer.

6. The method of claim 1, wherein, The plurality of subtasks and the plurality of subgoals are determined according to the web automation task, including: The plurality of subtasks and the plurality of subgoals are generated according to the web automation task using a large language model, and the number of subgoals is equal to the number of subtasks; The operation sequence of the current subtask is determined according to the current subtask, including: The operation sequence of the current subtask is generated according to the current subtask using the large language model.

7. The method of claim 1, wherein, The plurality of subtasks and the plurality of subgoals are determined according to the web automation task, including: The plurality of subtasks are generated according to the web automation task using a large language model; When the name of any one subtask is different from the name of a predefined subtask in a predefined subtask set, the subgoal corresponding to the subtask is generated using the large language model; The operation sequence of the current subtask is determined according to the current subtask, including: When the name of the current subtask is the same as the name of a predefined subtask in a predefined subtask set, the operation sequence of the predefined subtask is determined as the operation sequence of the current subtask; When the name of the current subtask is different from the name of a predefined subtask in a predefined subtask set, the operation sequence of the current subtask is generated according to the current subtask using the large language model.

8. A device for executing web page automation tasks, characterized in that, The device includes: The first determination module is configured to determine a plurality of subtasks and a plurality of subgoals according to the web automation task, each subgoal corresponding to a subtask, and the number of subgoals being less than the number of subtasks; The execution module is configured to sequentially execute the plurality of subtasks from the first subtask, and in the execution process of the current subtask, determine the operation sequence of the current subtask according to the current subtask, execute the operation sequence of the current subtask, and obtain the execution result of the current subtask; The verification module is configured to verify whether the execution result of the current subtask is consistent with the subgoal of the current subtask when the subgoal of the current subtask exists; The second determination module is configured to determine that the execution of the current subtask is completed when the current subtask passes the verification. The third determination module is configured to determine that the execution of the current subtask is completed when the subgoal of the current subtask does not exist.

9. An electronic device, comprising: It includes: A processor; A memory for storing processor-executable instructions; When the processor executes the instructions stored in the memory, the processor is configured to implement the method of any one of claims 1 to 7.

10. A non-transitory computer readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions are executed by the processor to implement the method of any one of claims 1 to 7.

Images