A task processing method, apparatus, equipment, medium, and program product

By generating multiple operation sequences by searching historical operation information sequences and current page structured information, the problem of not being able to accurately and orderly execute multiple interactive operations in existing technologies is solved, thus improving interaction efficiency and accuracy.

CN123068142APending Publication Date: 2026-11-10INDUSTRIAL AND COMMERCIAL BANK OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610957336.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-06-30
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies cannot output multiple interactive operations at once, nor can they execute multiple interactive operations accurately and in an orderly manner.

Method used

By responding to user task commands, the system searches for matching reference operation information sequences from historical operation information sequences, generates prompt words by combining them with the structured information of the current page, inputs a large model to output the current operation information sequence, and determines and executes candidate operations based on the structured information of the page.

Benefits of technology

It enables the generation of multiple operation sequences in a single operation, improving interaction efficiency and ensuring the accuracy and coherence of multiple operations, while avoiding the repeated generation of prompt words and the calling of large models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068142A_ABST
    Figure CN123068142A_ABST
Patent Text Reader

Abstract

This invention discloses a task processing method, apparatus, device, medium, and program product, relating to the field of intelligent interaction technology and applicable to the fintech field. The method includes: responding to a user's task instruction on the current page, searching for a reference operation information sequence matching the task instruction from a historical operation information sequence; generating prompt words based on the reference operation information sequence and the structured information of the current page, inputting a large model, and outputting the current operation information sequence; determining and executing candidate operations matching the current page from the current operation information sequence based on the current operation information sequence and the structured information of the current page; and, upon entering a new page, determining and executing candidate operations matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page, until the current operation information sequence is completed, thus achieving accurate and orderly automated execution of multiple candidate operations.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of intelligent interactive technology, and in particular to a task processing method, apparatus, device, medium, and program product. Background Technology

[0002] When users interact with electronic devices or applications on those devices, they need to actively engage with the devices or applications. To improve the user experience and reduce operational costs, existing technologies have been developed that utilize large models for intelligent interaction.

[0003] However, existing technologies utilize large models to output and execute single interactive operations sequentially, which cannot output multiple operations at once, nor can they accurately and orderly execute multiple interactive operations. Summary of the Invention

[0004] This invention provides a task processing method, apparatus, device, medium, and program product to achieve accurate and orderly execution of multiple interactive operations.

[0005] In a first aspect, embodiments of the present invention provide a task processing method, including:

[0006] In response to a task instruction issued by the user on the current page, a reference operation information sequence matching the task instruction is searched from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position;

[0007] Based on the reference operation information sequence and the structured information of the current page, prompt words are generated, input into the large model, and the current operation information sequence is output; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position;

[0008] Based on the current operation information sequence and the structured information of the current page, a candidate operation matching the current page is determined from the current operation information sequence and executed; and, after entering a new page, based on the current operation information sequence and the structured information of the new page, a candidate operation matching the new page is determined from the current operation information sequence and executed, until the current operation information sequence is completed.

[0009] Secondly, embodiments of the present invention also provide a task processing apparatus, comprising:

[0010] The search module is used to respond to a task instruction issued by the user on the current page and search for a reference operation information sequence that matches the task instruction from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position;

[0011] The output module is used to generate prompt words based on the reference operation information sequence and the structured information of the current page, input a large model, and output the current operation information sequence; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position;

[0012] An execution module is configured to determine and execute a candidate operation matching the current page from the current operation information sequence based on the current operation information sequence and the structured information of the current page; and, after entering a new page, determine and execute a candidate operation matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page, until the current operation information sequence is executed.

[0013] Thirdly, embodiments of the present invention also provide an electronic device, comprising:

[0014] At least one processor; and

[0015] A memory that is communicatively connected to at least one processor; wherein

[0016] The memory stores instructions that can be executed by at least one processor, which enables the at least one processor to perform the task processing method provided in any embodiment of the present invention.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that are used to cause a processor to execute and implement the task processing method of any embodiment of the present invention.

[0018] Fifthly, embodiments of the present invention also provide a computer program product, characterized in that the computer program product includes a computer program, which, when executed by a processor, implements the task processing method of any embodiment of the present invention.

[0019] This invention can respond to user task commands on the current page by searching historical operation information sequences and inputting these sequences, along with the structured information of the current page, as prompt words into a large model. It outputs a current operation information sequence including multiple candidate operations, achieving a single generation of multiple operation sequences. Compared to existing technologies that repeatedly generate prompt words and have the large model output a single step, this avoids repeatedly generating prompt words and calling the large model, thus improving interaction efficiency. Furthermore, based on the structured information of each page, it determines and executes candidate operations matching each page from the current operation information sequence. This allows for accurate selection and execution of matching candidate operations from the current operation information sequence based on the structured information of the real-time page, ensuring the accurate and orderly execution of multiple operations within the current operation information sequence and guaranteeing the accuracy and continuity of multi-operation execution.

[0020] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a task processing method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart of a task processing method provided according to Embodiment 2 of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a task processing device according to Embodiment 3 of the present invention;

[0025] Figure 4 This is a structural diagram of an electronic device that implements a task processing method according to an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first" and "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] The acquisition, storage, and application of historical operation information sequences and other related technologies in the technical solutions of this invention comply with relevant laws and regulations and do not violate public order and good morals.

[0029] Example 1

[0030] Figure 1 This is a flowchart of a task processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to situations where interactive operations are automatically executed in response to user task instructions. The method can be executed by a task processing device, which can be implemented in hardware and / or software and specifically configured in an electronic device.

[0031] See Figure 1 The task processing methods shown include:

[0032] S101. In response to a task instruction issued by the user on the current page, a reference operation information sequence matching the task instruction is searched from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position.

[0033] S102. Generate prompt words based on the reference operation information sequence and the structured information of the current page, input the large model, and output the current operation information sequence; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position.

[0034] S103. Based on the current operation information sequence and the structured information of the current page, determine and execute a candidate operation matching the current page from the current operation information sequence; and, after entering a new page, determine and execute a candidate operation matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page, until the current operation information sequence is executed.

[0035] In this embodiment, the current page can be the real-time page when the user issues a task instruction on an electronic device. The task instruction can be a natural language instruction issued by the user, such as "help me make the minimum repayment." The historical operation information sequence can be an operation information sequence generated based on the complete task operation manually performed by the user before issuing the task instruction. The reference operation information sequence is the historical operation information sequence that matches the task instruction.

[0036] The operation information sequence includes operation attribute information for at least one operation; the operation attribute information for each operation may include, but is not limited to, the operation page semantics, operation action, operation element semantics, and operation element position; wherein, the operation page semantics is the semantics of the page on which the operation is applied; the operation action may be the specific execution action of the operation; the operation element semantics may be the semantics of the page element on which the operation is applied; and the operation element position may be the position of the page element used by the operation. For example, the operation page semantics is "payment account selection page"; the operation action is "click"; the operation element semantics is "default repayment account"; and the operation element position is "(270, 1800)".

[0037] The current operation information sequence is the sequence of operation information used for the task represented by the task instruction. Candidate operations are the operations within the current operation information sequence. The structured information of the current page may include, but is not limited to, page semantics, page element semantics, and page element positions; where page semantics can be the semantics of the current page; page element semantics can be the semantics of at least one page element in the current page; and page element positions can be the positions of at least one page element in the current page.

[0038] Specifically, in response to a task instruction issued by the user on the current page, natural language processing is performed on the task instruction to obtain a task identifier corresponding to the task instruction; for example, if the task instruction is a voice instruction, speech-to-text processing can be performed on the voice instruction to obtain the text corresponding to the task instruction; the text corresponding to the task instruction is input into the natural language processing model to obtain the task identifier corresponding to the task instruction.

[0039] From at least one historical operation information sequence, query the historical operation sequence that matches the task identifier corresponding to the task instruction as a reference operation information sequence; obtain the image of the current page; input the image of the current page into a large model for image element recognition and semantic parsing, and output the structured information of the current page.

[0040] According to the preset prompt word structure framework, the reference operation information sequence and the structured information of the current page are concatenated into prompt words; the prompt words are input into a large model to obtain the current operation information sequence output by the large model; using a certain algorithm, based on the current operation information sequence and the structured information of the current page, candidate operations matching the current page are determined and executed from the current operation information sequence; and, after entering a new page, based on the current operation information sequence and the structured information of the new page, candidate operations matching the new page are determined and executed from the current operation information sequence until the current operation information sequence is executed.

[0041] Optionally, the prompts may also include character setting phrases, task instruction phrases, and output specification phrases to improve the accuracy of the current operation information sequence output by the large model.

[0042] Optionally, the historical operation information sequence is determined as follows: Historical task operation information manually operated by the user is obtained; the historical task operation information includes structured information of at least one historical page manually operated by the user, the trigger time of each historical page, and monitoring attribute information of at least one manual operation; the monitoring attribute information includes the operation action, operation location, and operation time; an association relationship between historical pages and manual operations is established based on the trigger time of each historical page and the operation time of each manual operation; for each manual operation associated with each historical page, operation attribute information for each manual operation is generated based on the structured information of the historical page and the monitoring attribute information of each manual operation; the operation attribute information of each manual operation is concatenated based on the trigger time of each historical page and the operation time of each manual operation to obtain the historical operation information sequence.

[0043] The historical task operation information can be the operation information during the process of the user manually completing a historical task; the historical task is the task manually completed by the user. The trigger time can be the moment when the historical page is triggered to be displayed; the operation location can be the page location where the corresponding manual operation is applied; the operation time is the moment when the corresponding manual operation is executed.

[0044] Specifically, the process involves: acquiring historical task operation information manually performed by the user; establishing an association between historical pages and manual operations based on the trigger time of each historical page and the operation time of each manual operation; wherein the operation time of each manual operation associated with each historical page is later than the trigger time of that historical page but earlier than the trigger time of other historical pages; generating operation attribute information for each manual operation associated with each historical page based on the structured information of the historical page and the monitoring attribute information of each manual operation; concatenating the operation attribute information of each manual operation based on the trigger time of each historical page and the operation time of each manual operation to obtain a historical operation information sequence; determining the task identifier of the historical task and establishing an association between the task identifier of the historical task and the historical operation information sequence.

[0045] Optionally, based on the structured information of the historical page and the monitoring attribute information of each manual operation, operation attribute information for each manual operation is generated, including: for each manual operation, determining the page element operated by the manual operation from the page elements according to the page element positions of each page element in the structured information of the historical page and the operation position of the manual operation, and taking the page element position of the page element as the operation element position of the manual operation, and taking the page element semantics of the page element as the operation element semantics of the manual operation; determining the page semantics of the historical page as the operation page semantics of the manual operation, thereby obtaining the operation attribute information of the manual operation.

[0046] Optionally, obtaining historical task operation information manually performed by the user may include: after obtaining user authorization, during the process of the user manually handling the task, collecting images of at least one historical page, the trigger time of each historical page, and collecting monitoring attribute information of at least one manual operation performed by the user during the process of manually handling the task; for each historical page image, inputting the historical page image into a large model for image element recognition and semantic parsing, and outputting the structured information of the historical page.

[0047] It is understandable that by adopting the above technical solution, historical task operation information manually operated by the user can be collected in advance before determining the reference operation information sequence, and a historical operation information sequence can be generated based on the historical task operation information. Furthermore, an association relationship can be established between the historical operation information sequence and the task identifier of the historical task. This facilitates the rapid search for a reference operation information sequence that matches the task instruction when responding to the user's task instruction, thereby improving the efficiency of determining the reference operation information sequence. By generating the historical operation information sequence from the user's actual manual historical task operation information, the accuracy of the historical operation information sequence is improved.

[0048] This invention can respond to user task commands on the current page by searching historical operation information sequences and inputting these sequences, along with the structured information of the current page, as prompt words into a large model. It outputs a current operation information sequence including multiple candidate operations, achieving a single generation of multiple operation sequences. Compared to existing technologies that repeatedly generate prompt words and have the large model output a single step, this avoids repeatedly generating prompt words and calling the large model, thus improving interaction efficiency. Furthermore, based on the structured information of each page, it determines and executes candidate operations matching each page from the current operation information sequence. This allows for accurate selection and execution of matching candidate operations from the current operation information sequence based on the structured information of the real-time page, ensuring the accurate and orderly execution of multiple operations within the current operation information sequence and guaranteeing the accuracy and continuity of multi-operation execution.

[0049] Example 2

[0050] Figure 2 This is a flowchart of a task processing method provided in Embodiment 2 of the present invention. The present invention is an optimization and improvement based on the technical solution of the above embodiments.

[0051] Furthermore, the phrase "determine and execute a candidate operation matching the current page from the current operation information sequence based on the current operation information sequence and the structured information of the current page" is refined to "determine a first candidate operation matching the current page from the current operation information sequence; calculate the target matching degree between the operation information of the first candidate operation and the structured information of the current page; if the target matching degree is greater than or equal to a first threshold, then execute the first candidate operation on the current page based on the operation information of the first candidate operation," thereby improving the determination and execution of the candidate operation matching the current page.

[0052] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.

[0053] See Figure 2The task processing methods shown include:

[0054] S201. In response to a task instruction issued by the user on the current page, a reference operation information sequence matching the task instruction is searched from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position.

[0055] S202. Generate prompt words based on the reference operation information sequence and the structured information of the current page, input the large model, and output the current operation information sequence; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position.

[0056] S203. In the current operation information sequence, determine the first candidate operation that matches the current page.

[0057] S204. Calculate the target matching degree between the operation information of the first candidate operation and the structured information of the current page.

[0058] S205. If the target matching degree is greater than or equal to the first threshold, then the first candidate operation is executed on the current page according to the operation information of the first candidate operation.

[0059] S206. After entering a new page, based on the current operation information sequence and the structured information of the new page, determine and execute candidate operations that match the new page from the current operation information sequence until the current operation information sequence is executed.

[0060] In this embodiment, the first candidate operation is the candidate operation that matches the current page. It should be noted that the first threshold can be set independently by technicians based on actual needs or practical experience; this invention does not impose specific limitations. As an example only, it could be 95% or 90%, etc.

[0061] Specifically, using a certain algorithm, a first candidate operation matching the current page is determined in the current operation information sequence; using a certain algorithm, the target matching degree between the operation information of the first candidate operation and the structured information of the current page is calculated; if the target matching degree is greater than or equal to a first threshold, the operation action of the first candidate operation is executed at the operation element position on the current page.

[0062] For example, if the operation attribute information of the first candidate operation includes: operation page semantic "payment account selection page", operation action "click", operation element semantic "default repayment account" and operation element position "(270, 1800)", then click at the position (270, 1800) on the current page to select the default repayment account as the payment account.

[0063] Optionally, determining the first candidate operation matching the current page in the current operation information sequence includes: determining the candidate operations in the current operation information sequence that match the operation page semantics with the page semantics of the current page as the first candidate operation matching the current page.

[0064] It is understandable that by adopting the above technical solution, by comparing the operation page semantics of each candidate operation in the current operation information sequence with the page semantics of the current page, the candidate operation whose operation page semantics are consistent with the page semantics of the current page is determined as the first candidate operation, thereby improving the accuracy of determining the first candidate operation.

[0065] Optionally, calculating the target matching degree between the operation information of the first candidate operation and the structured information of the current page includes: calculating a first matching degree between the semantics of the operation element of the first candidate operation and the semantics of the page element of the current page; determining page elements with the first matching degree greater than or equal to a second threshold as auxiliary page elements; calculating a second matching degree between the position of the operation element of the first candidate operation and the position of the page element of each of the auxiliary page elements; and determining the maximum value of the second matching degree as the target matching degree.

[0066] Specifically, the semantics of the operation element are converted into an operation element semantic vector, and the semantics of each page element in the current page are converted into a page element semantic vector; the cosine similarity between the semantic vector of each page element and the semantic vector of the operation element is calculated; page elements with a first matching degree greater than or equal to a second threshold are determined as auxiliary page elements; it should be noted that the second threshold can be set independently by technicians according to actual needs or practical experience, and this invention does not impose specific limitations; as an example only, it can be 95% and 90%, etc.

[0067] Calculate the distance between the position of the operation element of the first candidate operation and the position of the page element of each of the auxiliary page elements; convert each distance into a second matching degree; wherein, the second matching degree obtained by converting larger distances is smaller, and the second matching degree obtained by converting smaller distances is larger. Determine the maximum value of the second matching degree as the target matching degree.

[0068] It is understandable that by adopting the above technical solution, page elements with semantic similarity to the operation element of the first candidate operation can be selected as auxiliary page elements in the current page; then, within the range of semantically similar auxiliary page elements, a second matching degree between the page element position of the auxiliary page element and the operation element position is calculated; and the maximum value of the second matching degree is determined as the target matching degree. This realizes that the target matching degree can characterize the semantic matching degree and position matching degree between the page element and the operation element, thereby ensuring the accuracy of determining whether to execute the first candidate operation based on the target matching degree.

[0069] Optionally, determining and executing a candidate operation matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page includes: determining a second candidate operation matching the new page from the current operation information sequence based on the structured information of the new page; calculating a target matching degree between the operation information of the second candidate operation and the structured information of the new page; and if the target matching degree is greater than or equal to a first threshold, executing the second candidate operation on the new page based on the operation information of the second candidate operation.

[0070] It should be noted that the process of determining the second candidate operation that matches the new page is similar to the process of determining the first candidate operation of the current page, and will not be described again here.

[0071] The process of calculating the target matching degree between the operation information of the second candidate operation and the structured information of the new page is similar to the process of calculating the target matching degree between the operation information of the first candidate operation and the structured information of the current page, and will not be described again here.

[0072] The process of executing the second candidate operation on the new page based on the operation information of the second candidate operation is similar to the process of executing the first candidate operation on the current page based on the operation information of the first candidate operation, and will not be described again here.

[0073] It is understood that by adopting the above technical solution, the second candidate operation matching the new page can be accurately determined in the current operation information sequence, and the target matching degree between the second candidate operation and the new page can be calculated. If the target matching degree is greater than or equal to the first threshold, the second candidate operation is executed on the new page to ensure the correct execution of the second candidate operation.

[0074] The embodiments of the present invention can accurately determine the first candidate operation that matches the current page in the current operation information sequence, and calculate the target matching degree between the first candidate operation and the current page. If the target matching degree is greater than or equal to a first threshold, the first candidate operation is executed on the current page, ensuring that the first candidate operation matches the current page and can be executed correctly.

[0075] Example 3

[0076] Figure 3 This is a schematic diagram of a task processing device according to Embodiment 3 of the present invention. This embodiment of the present invention is applicable to situations where interactive operations are automatically executed in response to user task commands. The device can execute task processing methods and can be implemented in hardware and / or software. The device can be configured in an electronic device.

[0077] See Figure 3 The task processing device shown includes a search module 301, an output module 302, and an execution module 303, wherein...

[0078] The search module 301 is used to respond to a task instruction issued by the user on the current page and search for a reference operation information sequence that matches the task instruction from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position;

[0079] Output module 302 is used to generate prompt words based on the reference operation information sequence and the structured information of the current page, input a large model, and output the current operation information sequence; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position;

[0080] The execution module 303 is configured to determine and execute a candidate operation matching the current page from the current operation information sequence based on the current operation information sequence and the structured information of the current page; and, after entering a new page, determine and execute a candidate operation matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page, until the current operation information sequence is executed.

[0081] This invention can respond to user task commands on the current page by searching historical operation information sequences and inputting these sequences, along with the structured information of the current page, as prompt words into a large model. It outputs a current operation information sequence including multiple candidate operations, achieving a single generation of multiple operation sequences. Compared to existing technologies that repeatedly generate prompt words and have the large model output a single step, this avoids repeatedly generating prompt words and calling the large model, thus improving interaction efficiency. Furthermore, based on the structured information of each page, it determines and executes candidate operations matching each page from the current operation information sequence. This allows for accurate selection and execution of matching candidate operations from the current operation information sequence based on the structured information of the real-time page, ensuring the accurate and orderly execution of multiple operations within the current operation information sequence and guaranteeing the accuracy and consistency of multi-operation execution.

[0082] Optionally, execution module 303 includes:

[0083] The first determining unit is configured to determine a first candidate operation that matches the current page in the current operation information sequence;

[0084] The first calculation unit is used to calculate the target matching degree between the operation information of the first candidate operation and the structured information of the current page;

[0085] The first execution unit is configured to execute the first candidate operation on the current page according to the operation information of the first candidate operation if the target matching degree is greater than or equal to the first threshold.

[0086] Optionally, the first determining unit includes:

[0087] Candidate operations in the current operation information sequence that match the semantics of the operation page with the semantics of the current page are determined as the first candidate operation that matches the current page.

[0088] Optionally, the first computing unit is specifically used for:

[0089] Calculate the first matching degree between the semantics of the operation element of the first candidate operation and the semantics of the page element of the current page;

[0090] Page elements whose first matching degree is greater than or equal to the second threshold are identified as auxiliary page elements;

[0091] Calculate the second matching degree between the operation element position of the first candidate operation and the page element position of each of the auxiliary page elements;

[0092] The maximum value in the second matching degree is determined as the target matching degree.

[0093] Optionally, execution module 303 includes:

[0094] The second determining unit is configured to determine a second candidate operation that matches the new page in the current operation information sequence based on the structured information of the new page.

[0095] The second calculation unit is used to calculate the target matching degree between the operation information of the second candidate operation and the structured information of the new page;

[0096] The second execution unit is configured to execute the second candidate operation on the new page according to the operation information of the second candidate operation if the target matching degree is greater than or equal to the first threshold.

[0097] Optionally, the device further includes:

[0098] The acquisition module is used to acquire historical task operation information manually operated by the user; the historical task operation information includes structured information of at least one historical page manually operated by the user, the trigger time of each historical page, and monitoring attribute information of at least one manual operation; the monitoring attribute information includes operation action, operation location, and operation time;

[0099] The association module is used to establish an association between historical pages and manual operations based on the trigger time of each historical page and the operation time of each manual operation.

[0100] The generation module generates operation attribute information for each manual operation associated with each historical page, based on the structured information of the historical page and the monitoring attribute information of each manual operation.

[0101] The splicing module is used to splice the operation attribute information of each manual operation according to the trigger time of each historical page and the operation time of each manual operation to obtain a sequence of historical operation information.

[0102] The task processing device provided in the embodiments of the present invention can execute the task processing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the task processing method.

[0103] Example 4

[0104] Figure 4A schematic diagram of a task processing device 410 that can be used to implement embodiments of the present invention is shown. The task processing device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The task processing device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0105] like Figure 4 As shown, the task processing device 410 includes at least one processor 411 and a memory, such as a read-only memory (ROM) 412 or a random access memory (RAM) 413, communicatively connected to the at least one processor 411. The memory stores computer programs executable by the at least one processor. The processor 411 can perform various appropriate actions and processes based on the computer program stored in the ROM 412 or loaded from storage unit 418 into the RAM 413. The RAM 413 may also store various programs and data required for the operation of the task processing device 410. The processor 411, ROM 412, and RAM 413 are interconnected via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0106] Multiple components in the task processing device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, optical disk, etc.; and a communication unit 419, such as a network card, modem, wireless transceiver, etc. The communication unit 419 allows the task processing device 410 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0107] Processor 411 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 411 performs the various methods and processes described above, such as task processing methods.

[0108] In some embodiments, the task processing method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 418. In some embodiments, part or all of the computer program may be loaded and / or installed on the task processing device 410 via ROM 412 and / or communication unit 419. When the computer program is loaded into RAM 413 and executed by processor 411, one or more steps of the task processing method described above may be performed. Alternatively, in other embodiments, processor 411 may be configured to perform the task processing method by any other suitable means (e.g., by means of firmware).

[0109] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0110] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable task processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0111] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0112] To provide interaction with a user, the systems and techniques described herein can be implemented on a task processing device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the task processing device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0113] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0114] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0115] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A task processing method, characterized in that, The method includes: In response to a task instruction issued by the user on the current page, a reference operation information sequence matching the task instruction is searched from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position; Based on the reference operation information sequence and the structured information of the current page, prompt words are generated, input into the large model, and the current operation information sequence is output; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position; Based on the current operation information sequence and the structured information of the current page, a candidate operation matching the current page is determined from the current operation information sequence and executed; and, after entering a new page, based on the current operation information sequence and the structured information of the new page, a candidate operation matching the new page is determined from the current operation information sequence and executed, until the current operation information sequence is completed.

2. The method according to claim 1, characterized in that, The step of determining and executing a candidate operation matching the current page from the current operation information sequence based on the current operation information sequence and the structured information of the current page includes: In the current operation information sequence, determine the first candidate operation that matches the current page; Calculate the target matching degree between the operation information of the first candidate operation and the structured information of the current page; If the target matching degree is greater than or equal to the first threshold, then the first candidate operation is executed on the current page according to the operation information of the first candidate operation.

3. The method according to claim 2, characterized in that, The step of determining the first candidate operation that matches the current page in the current operation information sequence includes: Candidate operations in the current operation information sequence that match the semantics of the operation page with the semantics of the current page are determined as the first candidate operation that matches the current page.

4. The method according to claim 2, characterized in that, The calculation of the target matching degree between the operation information of the first candidate operation and the structured information of the current page includes: Calculate the first matching degree between the semantics of the operation element of the first candidate operation and the semantics of the page element of the current page; Page elements whose first matching degree is greater than or equal to the second threshold are identified as auxiliary page elements; Calculate the second matching degree between the operation element position of the first candidate operation and the page element position of each of the auxiliary page elements; The maximum value in the second matching degree is determined as the target matching degree.

5. The method according to claim 1, characterized in that, The step of determining and executing candidate operations matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page includes: Based on the structured information of the new page, a second candidate operation matching the new page is determined in the current operation information sequence; Calculate the target matching degree between the operation information of the second candidate operation and the structured information of the new page; If the target matching degree is greater than or equal to the first threshold, then the second candidate operation is executed on the new page according to the operation information of the second candidate operation.

6. The method according to claim 1, characterized in that, The sequence of historical operation information is determined in the following way: Obtain historical task operation information manually operated by the user; the historical task operation information includes structured information of at least one historical page manually operated by the user, the trigger time of each historical page, and monitoring attribute information of at least one manual operation; the monitoring attribute information includes operation action, operation location, and operation time; Based on the trigger time of each of the historical pages and the operation time of each of the manual operations, establish the association between the historical pages and the manual operations; For each manual operation associated with each historical page, operation attribute information for each manual operation is generated based on the structured information of the historical page and the monitoring attribute information of each manual operation. Based on the trigger time of each historical page and the operation time of each manual operation, the operation attribute information of each manual operation is concatenated to obtain a sequence of historical operation information.

7. A task processing device, characterized in that, The device includes: The search module is used to respond to a task instruction issued by the user on the current page and search for a reference operation information sequence that matches the task instruction from the historical operation information sequence; the operation information sequence includes operation attribute information of at least one operation; the operation attribute information includes operation page semantics, operation action, operation element semantics, and operation element position; The output module is used to generate prompt words based on the reference operation information sequence and the structured information of the current page, input a large model, and output the current operation information sequence; the current operation information sequence includes operation attribute information of at least one candidate operation, and the structured information includes page semantics, page element semantics, and page element position; An execution module is configured to determine and execute a candidate operation matching the current page from the current operation information sequence based on the current operation information sequence and the structured information of the current page; and, after entering a new page, determine and execute a candidate operation matching the new page from the current operation information sequence based on the current operation information sequence and the structured information of the new page, until the current operation information sequence is executed.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the task processing method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the task processing method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the task processing method according to any one of claims 1-6.