Method, apparatus, device, and product for performing tasks

Abstract

The present disclosure relates to methods, apparatuses, devices, and products for performing tasks. The method includes obtaining an operation sequence of a user for a first task on a user interface, the operation sequence including operation steps required to complete the first task. The method includes converting the operation steps into a model context protocol service based on the operation sequence. The method includes obtaining a user input, the user input indicating a second task, the user input including at least one of text input or voice input. The method includes determining a model context protocol service to be executed based on the second task, the model context protocol service to be executed being associated with the second task. The method includes executing the second task based on the task parameters of the second task and the service parameters of the model context protocol service to be executed, the second task being composed of the model context protocol service to be executed associated with the second task, the task parameters of the second task being determined based on the user input and displaying the execution result of the second task.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and more particularly to methods, apparatuses, devices, and products for performing tasks. Background Art

[0002] In today's highly digital and information-based working environment, various software applications and automation tools have improved human work efficiency. From process automation to intelligent decision support, from low-code development to cross-platform collaboration, the innovation of tools is reshaping traditional working modes. For example, robotic process automation (RPA) can complete repetitive tasks in a code-free manner; intelligent document processing tools can achieve efficient parsing of unstructured data through OCR and NLP technologies; while the DevOps automation platform can shorten the software delivery cycle to the minute level. Summary of the Invention

[0003] In a first aspect of an embodiment of the present disclosure, a method for performing a task is provided. The method includes obtaining an operation sequence of a user for a first task on a user interface, where the operation sequence includes operation steps required to complete the first task. The method includes converting the operation steps into a model context protocol service based on the operation sequence. The method includes obtaining a user input, where the user input indicates a second task, and the user input includes at least one of text input or voice input. The method includes determining a model context protocol service to be executed based on the second task, where the model context protocol service to be executed is associated with the second task. The method includes executing the second task based on the task parameters of the second task and the model context protocol service to be executed, where the second task is composed of the model context protocol service to be executed associated with the second task, and the task parameters of the second task are determined based on the user input and the execution result of the second task is displayed.

[0004] In a second aspect of the embodiments of the present disclosure, a device for performing tasks is provided. The device includes an operation sequence acquisition module configured to acquire an operation sequence of the user for a first task on the user interface, where the operation sequence includes operation steps required to complete the first task. The device includes an operation step conversion module configured to convert the operation steps into a model context protocol service based on the operation sequence. The device includes a user input acquisition module configured to acquire a user input, where the user input indicates a second task, and the user input includes at least one of text input or voice input. The device includes a to-be-executed model context protocol service determination module configured to determine a to-be-executed model context protocol service based on the second task, where the to-be-executed model context protocol service is associated with the second task. The device includes a second task execution module configured to execute the second task based on the task parameters of the second task and the to-be-executed model context protocol service, where the second task is composed of the to-be-executed model context protocol service associated with the second task, and the task parameters of the second task are determined based on the user input. The device includes an execution result display module configured to display the execution result of the second task on the user interface.

[0005] In a third aspect of the embodiments of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method of the first aspect.

[0006] In a fourth aspect of the embodiments of the present disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions that, when executed, cause the machine to implement the method of the first aspect.

[0007] The summary of the invention is provided to introduce a selection of concepts in a simplified form, which will be further described in the detailed implementation below. The summary of the invention is not intended to identify the key features or main features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Brief Description of the Drawings

[0008] In conjunction with the accompanying drawings and with reference to the following detailed description, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. In the drawings, the same or similar reference numerals denote the same or similar elements, where:

[0009] Figure 1 A schematic diagram showing an example environment in which multiple embodiments of the present disclosure can be implemented;

[0010] Figure 2A flowchart of a method for performing a task according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram of an example system architecture for performing a task according to some embodiments of the present disclosure is shown;

[0012] Figures 4A - 4D A schematic diagram of an example function of an operation recording module according to some embodiments of the present disclosure is shown;

[0013] Figures 5A - 5D A schematic diagram of an example function of a model context protocol (MCP) service generation according to some embodiments of the present disclosure is shown;

[0014] Figure 6 A schematic diagram of an example function of a service storage and indexing module according to some embodiments of the present disclosure is shown;

[0015] Figures 7A - 7D A schematic diagram of an example function of a large model decision module according to some embodiments of the present disclosure is shown;

[0016] Figures 8A - 8C A schematic diagram of an example function of a tool execution and monitoring module according to some embodiments of the present disclosure is shown;

[0017] Figures 9A - 9C A schematic diagram of an example function of a user interaction module according to some embodiments of the present disclosure is shown;

[0018] Figure 10 A block diagram of a device for performing a task according to some embodiments of the present disclosure is shown; and

[0019] Figure 11 A block diagram of a device capable of implementing multiple embodiments of the present disclosure is shown. Detailed Description of the Embodiments

[0020] It can be understood that all user-related data involved in the present technical solution should be obtained and used after obtaining user authorization. This means that in the present technical solution, if it is necessary to use the user's personal information, then before obtaining this data, the user's explicit consent and authorization are required, otherwise the relevant data collection and use will not be carried out. It should also be understood that when implementing the present technical solution, relevant laws and regulations should be strictly adhered to during the process of data collection, use, and storage, and necessary technical means and measures should be taken to ensure the user's data security and ensure the safe use of the data.

[0021] It is understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0022] For example, when receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be executed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message.

[0023] As an optional but non-limiting implementation manner, the manner of sending a prompt message to the user in response to receiving an active request from the user may be, for example, in the form of a pop-up window, and the prompt message may be presented in text in the pop-up window. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0024] It is understood that the above process of notifying and obtaining the user's authorization is only illustrative and does not limit the implementation manner of the present disclosure, and other manners that meet relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0025] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0026] In the description of the embodiments of the present disclosure, the term "including" and its similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects unless otherwise clearly stated. There may also be other explicit and implicit definitions below.

[0027] As described above, in today's highly digital and information-based working environment, various software applications and automation tools have greatly improved the work efficiency of humans. In related technologies, users need to master programming knowledge to develop automation tools to achieve automated interaction processes. However, this technical threshold limits the participation of non-technical personnel. For users lacking programming foundation, it is too difficult to develop automation tools by themselves, resulting in their inability to convert repetitive tasks into automated processes. In addition, in related technologies, users can also use recording and playback tools to achieve automated execution of preset operations. Although such tools reduce the operation difficulty, this method with limited functions can only perform simple repetitive work and cannot be flexibly adjusted according to the dynamic needs of users, thus unable to meet the requirements of users' complex tasks.

[0028] To this end, the present disclosure proposes a method for executing tasks, which can automatically convert simple operations of users into model online protocol services that can be dynamically combined, so as to achieve task execution. In the solution provided by the present disclosure, first, an operation sequence of the user on the user interface for a first task is obtained, and this operation sequence covers all operation steps necessary to complete the first task. Then, according to the obtained operation sequence, the operation steps therein are converted into model context protocol services. Subsequently, a user input is received, and the user input is used to indicate a second task, and the input includes one of text input or voice input. Next, according to the indicated second task, the to-be-executed model context protocol service associated therewith is determined. Finally, based on the task parameters of the second task and the service parameters of the to-be-executed model context protocol service, the second task composed of the associated to-be-executed model context protocol services is executed, where the task parameters of the second task are determined according to the user input, and the execution result of the second task is displayed on the user interface after execution.

[0029] According to the embodiments of the present disclosure, the operation sequence of the user for the first task can be reused to intelligently match and execute the second task of the user, greatly improving the work efficiency and reducing the manual operation duration and workload. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so as to be able to handle new tasks and new requirements. In addition, by means of natural language interaction, the use threshold is reduced, enabling non-professionals to use it, and comprehensively improving the practicality and scope of application of task execution. That is, through this method, the simple operations of users can be automatically converted into model online protocol services that can be dynamically combined, realizing task execution and improving the user experience.

[0030] Figure 1 A schematic diagram of an example environment 100 in which multiple embodiments of the present disclosure can be implemented is shown. As Figure 1As shown, the example environment 100 includes Task A and Task B. To achieve the automated completion of Task B, the operation steps of the task can be recorded first for analysis, so as to implement the execution of Task B.

[0031] Task A is completed manually by the user. The execution process of Task A can be obtained by recording the operation steps of the user to complete Task A in the user interface. In some embodiments, the recording of the completion process of Task A can be achieved through multi-mode recording techniques. For example, if Task A is executed in a web application, specific scripts can be injected to achieve DOM event capture, so as to record the interaction events of the user during the process of completing Task A. Another example is that the user interface element recognition technology can also be used to identify the UI elements with which the user interacts during the execution of Task A. Another example is that the screen operation recording technology can also be used to capture the interaction operations between the user and the application in the user interface through screen elements during the process of the user completing Task A. Another example is that the application programming interface (API) call capture technology can also be used to record the background API call sequence and parameter information, so as to achieve the recording of the execution steps of Task A. As Figure 1 As shown, the operation steps of Task A can be obtained through recording. The operation steps 110 of Task A include Step 1, Step 2, Step 3, Step 4, Step 5, Step 6, and Step.

[0032] After obtaining the operation steps to complete Task A, the semantic information of the operation steps to complete Task A can be extracted. For example, element semantic analysis can be used to identify the functional attributes in the interaction elements, such as buttons, input boxes, and drop-down menus, etc. Then, operation intention recognition can be used to infer the operation intention. For example, the operation intention of the user during the process of completing Task A can be inferred according to the interaction mode and context. For example, according to Step 1 and Step 2, it can be inferred that Step 3 is an operation such as querying, filtering, or submitting. Then, data flow tracking can be used to record the complete path of data input, transformation, and output. Finally, context state capture can be used to capture the state changes of the system after completing Task A.

[0033] Reference Figure 1, after obtaining the operation steps of Task A, the operation steps (including the extracted voice information) can be converted into the operation sequence 120 of Task A. The operation step sequence of Task A can be as shown in [Step 1, Step 2, Step 3, Step 4, Step 5, Step 6, Step 7]. For example, the operation sequence 120 can include the timing dependencies between each operation step. For example, Step 1 is before Step 2, and Step 2 is before Step 3. For another example, the operation sequence 120 can include the standardized representation of each identified interaction element. For example, a button can be "button"; a text box can be "text". For another example, the operation sequence 120 can include the types of data parameters, which can be divided into variable parameters and immutable parameters. For another example, the label parameter of the button "button" is "Next Step", which is an immutable parameter. And the "text" input parameter of the text box represents the quantity and needs to be determined according to the user's input. For another example, the operation sequence 120 can also include conditional branches and loop structures. For example, Step 1 is a conditional branch, and only when a certain condition is met can one proceed to Step 2.

[0034] In order to achieve the generality and portability of executing other tasks such as Task B in the subsequent process, the specific operation steps can be converted into a model context protocol service (hereinafter also simply referred to as a service). The model context protocol is an open protocol designed to standardize the communication between large language models (LLMs) and external data sources and tools, and solve the problem of standardizing the interaction between models and external resources. Given that the model context protocol service is functionally independent, it is necessary to define which steps in the operation steps of Task A are inseparable and which are consecutive, so as to achieve the identification of the functional boundaries of the model context protocol service. In some embodiments, the functional boundary identification technology of multi-dimensional semantic analysis can be used to determine the functional boundaries of each model context protocol service.

[0035] For example, the functional boundaries can be determined based on the semantic correlation scores between each operation step in the operation sequence. For instance, a graph can be constructed with each step in the operation sequence as a node and the semantic correlation scores as edges, and then a community detection algorithm can be applied to the graph to identify highly cohesive subgraphs. After verifying the functional integrity and independence of each subgraph, the highly cohesive subgraphs can be converted into independent model context protocol services.

[0036] For example, the boundary definition of the model context protocol service can also be achieved by leveraging the semantic information of each operation step in the operation sequence. For instance, segmentation can be performed based on the boundaries of the input and output of the data stream to achieve the definition of the functional boundary. For example, the functional boundary of the model context protocol service can also be determined based on the significant change points in the system state after the task is completed. Alternatively, specific domain knowledge can be combined to assist in identifying the service boundary. For example, the steps of generating a report can be divided into data extraction, data transformation, data analysis, and report generation. Based on the knowledge of data processing, data extraction and data transformation can be defined as an independent service for data extraction and transformation. At this time, the four steps can be defined as three independent functions: data extraction and transformation, data analysis, and report generation.

[0037] As Figure 1 shown, [Step 1, Step 2, Step 3] can be defined as an independent model context protocol service; [Step 4] can be defined as an independent model context protocol service; [Step 5, Step 6] can be defined as an independent model context protocol service; [Step 7] can also be defined as an independent model context protocol service.

[0038] For the automated execution of Task B in the subsequent process, the task parameters of Task B are necessarily required. Parameters are variables declared when defining a function, method, or procedure, and parameter values are the specific data assigned to these parameters when they are called. By passing different parameter values, the same function, method, or procedure can process different data, thereby achieving different functions. For the sake of convenience in explanation, hereinafter, task parameters can be abbreviated as parameters, and parameter values and parameters can be the same expression.

[0039] However, how the task parameters of Task B are applied to the model context protocol service determined from the operation steps of Task A necessarily requires determining which parameter values of the operation steps in Task A are variable and which are immutable. Only in this way can the parameters of Task B be applied to the model context protocol service determined from the completion process of Task A, thereby achieving the automated execution of Task B.

[0040] In some embodiments, an adaptive parameter abstraction technique can be used to identify the variable parts in the operation steps. For example, candidate parameter points for Task A can be identified. The parameters of the candidate parameter points are variable, and the candidate parameter points can be parameter types such as form inputs, query conditions, etc. Then, based on each candidate parameter point, a parameter definition for each parameter point is generated, namely the parameter description, type, and constraint conditions. For example, a certain candidate parameter point is a time range parameter: type = date range, format = YYYY - MM - DD, description = the time range covered by the report. Another example, a certain candidate parameter point is a data type parameter: type = enumeration, optional values = [business, inventory, customer, finance], description = the data type to be analyzed.

[0041] For example, candidate parameter points for Task A can be identified. Then, the feature vector F of each candidate parameter point can be extracted, and with the help of a decision model, the parameterization probability P of each candidate parameter point can be determined. Thus, based on the parameterization probability P, the parameter description, type, and constraint conditions of the parameter points are screened out.

[0042] Alternatively, for example, the pattern features of the input values can also be analyzed. If the input value is in date format or numerical range form, then the input value is variable, thereby identifying the variable parts in the operation steps. For example, the correlation between certain parameter values in Task A and context interaction elements can also be identified, so as to analyze whether the parameter values of the parameter are variable. As described above, the label parameter of the button "button" is "Next Step", which is an immutable parameter. Another example, if the user records the process of completing Task A manually multiple times, the variable and immutable parts of the operation steps can be identified by comparing the difference points in the multiple records. Another example, the domain entity recognition technology can also be used to determine whether the parameter value is an entity in a specific domain, such as to determine whether a certain parameter value is a name or a name field, etc.

[0043] After determining the functional boundaries of the model context protocol service for Task A and the variable parameter parts therein, these model context protocol services need to be encapsulated into independent functional service entities. For example, in the encapsulated model context protocol service, parameter verification rules are also included. The verification rules can be generated according to the parameter definitions of the variable parameters to achieve the accuracy of task execution. For example, a numeric parameter does not allow a string parameter value to be input. When the task parameters of Task B pass the verification, Task B can continue to be executed. For example, in the encapsulated model context protocol service, executable code converted from the operation sequence is also included.

[0044] For example, in the encapsulated model context protocol service, it also includes a standardized service interface definition to enable communication with external data sources and tools. For example, in the encapsulated model context protocol service, it also includes error handling logic to enable automatic addition of exception detection and execution of corrective operations. For instance, when it is detected that data is missing in a certain area, it can be automatically adjusted to only analyze the areas with complete data. For example, in the encapsulated model context protocol service, it also includes service documentation to enable automatic generation of service descriptions and usage instructions.

[0045] In some embodiments, the encapsulated model context protocol service includes executable code converted from an operation sequence, a service identifier, parameter definitions, and verification rules.

[0046] After the analysis of the operation steps for Task A is completed, in order to achieve the goal of automatically completing Task B, it is first necessary to know what Task B is, that is, it is necessary to know the user's intention. And the user's intention can be determined based on the user input 130. In some embodiments, the user input can be in text form or in voice form, but no matter what form it is, ultimately it is in natural language form.

[0047] As Figure 1 shown, Task B 140 and the task parameters of Task B can be determined based on the user input 130. For example, a semantic parsing model such as a decision model can be used to extract the core part of the user input 130. The decision model is an intelligent model that can achieve cross-turn semantic understanding, can also infer the implicit intention of the user based on the context and the user's historical preferences, and can also decompose the user intention into an executable sub-intention sequence.

[0048] The decision model can preprocess the user input 130 to obtain the core intention. For example, if the user input 130 is "Generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", the decision model can analyze and identify that Task B is to generate a business performance report, where the core action: generate a report; key parameters: time range = last quarter, data type = business, region = all, format = PDF, recipient = business team. And the completion process of Task A is a complete process of extracting data from multiple systems (ERP, CRM, and data warehouse) and generating a comprehensive report. It is decomposed into 4 functionally independent model context protocol services: data extraction [Step 1, Step 2, Step 3], data conversion [Step 4], data analysis [Step 5, Step 6], and report generation [Step 7].

[0049] In order to enable Task B to be automatically executed, the most suitable model context service 150 to be executed for Task B can be determined from the 4 functionally independent model context protocol services of Task A. In some embodiments, the semantic relevance between the intent expressed by the user input 130 and the 4 independent services of Task A can be calculated (using a decision model) to determine the most suitable model context service 150 to be executed for Task B. For example, the decision model plans that the set of model context protocol services to be executed for Task B includes: data extraction, data transformation, data analysis, and report generation.

[0050] Then, based on the dependencies between these model context services 150 to be executed, the execution path of service invocation (which can also be called the service invocation sequence) can be determined. In some embodiments, a call dependency graph with these services as nodes can be constructed, and then a path planning algorithm can be used to determine the execution path. For example, the execution path can be: data extraction → data transformation → data analysis → report generation.

[0051] Next, the task parameters of Task B can be mapped to the model context services 150 to be executed to achieve the automatic execution of Task B 160. For example, "last quarter" can be converted into a specific date range (such as "April 1, 2023 to June 30, 2023"); "business team" can be parsed into "specific email list"; necessary default parameters can be supplemented (such as report template = "standard business performance template"). In some embodiments, a parameter mapping table can be generated according to the task parameters of Task B, and the source of the parameters is included in this mapping table. For example, some parameters are obtained from the user input 130, some parameters are inferred default values, and some parameters can be parameters that request the user to supplement again. It can be understood that the model context services 150 to be executed also need service parameters, that is, fixed and immutable parameters.

[0052] Through the above description, Task B can be automatically executed, and the execution result of Task B can also be displayed on the user interface. According to the embodiments of the present disclosure, the operation sequence of the user for Task A can be reused to intelligently match and execute the user's Task B, greatly improving work efficiency and reducing manual operation time and workload. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so as to be able to handle new tasks (such as Task B) and new requirements. In addition, by means of natural language interaction, the usage threshold is reduced, allowing non-professionals to use it, and comprehensively improving the practicability and application scope of task execution. That is, through this method, the simple operations of the user can be automatically converted into model online protocol services that can be dynamically combined to achieve task execution and improve the user experience.

[0053] The following will be combined with Figures 2 to 11 Describe in detail the method according to an embodiment of the present disclosure. For ease of understanding, the specific data mentioned in the following description are exemplary and are not used to limit the protection scope of the present disclosure. It can be understood that the embodiments described below may also include additional actions not shown and / or actions shown may be omitted, and the scope of the present disclosure is not limited in this regard.

[0054] Figure 2 A flowchart of a method 200 for performing a task according to some embodiments of the present disclosure is shown. The method 200 may be executed by a device for performing a task, and the device may be, for example, an independent device or system. The device may be implemented in a software and / or hardware manner. Next, taking the device for performing a task as the execution subject as an example, the method 200 will be schematically described. The method 200 includes block 202, block 204, block 206, block 208, block 210, and block 212.

[0055] As Figure 2 shown, at block 202, obtain an operation sequence of the user on the user interface for a first task, and the operation sequence includes the operation steps required to complete the first task. Refer to Figure 1 , Task A is completed manually by the user, and the operation steps of the user to complete Task A on the user interface can be recorded, so that the execution process of Task A can be obtained. After obtaining the operation steps of Task A, the operation steps can be converted into an operation sequence 120 of Task A, and the operation sequence 120 includes the operation steps of Task A. The operation sequence 120 of Task A is shown as [Step 1, Step 2, Step 3, Step 4, Step 5, Step 6, Step 7].

[0056] At block 204, based on the operation sequence, convert the operation steps into a model context protocol service. Refer to 1, in order to achieve the generality and portability of executing other tasks such as Task B in the subsequent process, the specific operation steps of Task A can be converted into a model context protocol service. As Figure 1 shown, [Step 1, Step 2, Step 3] can be defined as an independent model context protocol service 1; [Step 4] can be defined as an independent model context protocol service 2; [Step 5, Step 6] can be defined as an independent model context protocol service 3; [Step 7] can also be defined as an independent model context protocol service 4.

[0057] At block 206, obtain user input, where the user input indicates a second task, and the user input includes at least one of text input or voice input. Refer to Figure 1, After the analysis of the operation steps for Task A is completed, in order to achieve the goal of automatically completing Task B, it is first necessary to know what Task B is, that is, it is necessary to know the user's intention. And the user's intention can be determined based on the user input 130. In some embodiments, the user input can be in text form or in voice form, but no matter what form it is, it is ultimately in the form of natural language. For example, if the user input 130 is "Generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", then Task B can be "Generate a report".

[0058] In block 208, based on the second task, determine the model context protocol service to be executed, and the model context protocol service to be executed is associated with the second task. Refer to Figure 1 , In order to enable Task B to be automatically executed, the most suitable model context service to be executed 150 for Task B can be determined from the 4 functionally independent model context protocol services of Task A. For example, the model context service to be executed 150 includes: Model Context Protocol Service 1 [Step 1, Step 2, Step 3], Model Context Protocol Service 2 [Step 4], Model Context Protocol Service 3 [Step 5, Step 6], and Model Context Protocol Service 4 [Step 7].

[0059] In block 210, based on the task parameters of the second task and the model context protocol service to be executed, execute the second task. The second task consists of the model context protocol service to be executed associated with the second task, and the task parameters of the second task are determined based on the user input. Refer to Figure 1 , The task parameters of Task B can be mapped to the model context service to be executed 150 to achieve the automatic execution of Task B 160.

[0060] In block 212, display the execution result of the second task on the user interface. Through the above blocks 202 to 210, Task B can be automatically executed, and on the user interface, the execution result of Task B can also be displayed.

[0061] According to the embodiments of the present disclosure, it is possible to reuse the user's operation sequence for Task A to intelligently match and execute the user's Task B, greatly improving work efficiency and reducing manual operation time and workload. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so as to be able to handle new tasks (such as Task B) and new requirements. In addition, by means of natural language interaction, the usage threshold is reduced, allowing non-professionals to use it, and comprehensively improving the practicality and scope of application of task execution. That is, through this method, the user's simple operations can be automatically converted into model online protocol services that can be dynamically combined to achieve task execution and improve the user experience.

[0062] Exemplarily, Figure 3 FIG. 300 is a schematic diagram of an exemplary system architecture for performing tasks according to some embodiments of the present disclosure. The exemplary system architecture 300 includes an operation recording module 310, a model context protocol service generation module 320, a service storage and indexing module 330, a large model decision module 340, a tool execution and monitoring module 350, and a user interaction module 360.

[0063] For ease of explanation, hereinafter, tasks A and B will be used as examples to describe. Task A is a task manually completed by the user, and task B is a task that needs to be automatically executed. Assume that task A is to extract data from multiple systems (ERP, CRM, and data warehouse) and generate a comprehensive report; the user expresses the intention in natural language as "generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format."

[0064] In some embodiments, the operation recording module 310 is used to capture the user operation sequence and extract structured semantic information. For ease of explanation, hereinafter, it will be described in conjunction with Figures 4A - 4D to describe. Exemplarily, Figure 4A FIG. 400A is a schematic diagram of an exemplary function of the operation recording module according to some embodiments of the present disclosure. As Figure 4A shown, the functions of the operation recording module 310 include initializing the recording environment 311.

[0065] For ease of explanation, in conjunction with Figure 4B exemplarily, Figure 4B FIG. 400B is a schematic diagram of an exemplary function of initializing the recording environment according to some embodiments of the present disclosure. In some embodiments, initializing the recording environment 311 refers to initializing the system environment before the user executes task A and before starting to record the operation steps of the task. As Figure 4B shown, initializing the recording environment may include loading a recording engine 3111, and the recording engine is used to record all the operation steps of the user in completing task A.

[0066] Referring to Figure 4B , initializing the recording environment 311 may include injecting an event listener 3112. Injecting an event listener refers to adding an event handling function or object to a specific event source. When the event source triggers a specific event, the associated event listener will be called to execute the corresponding processing logic. The event source may be an HTML element in a web page, a window of an operating system, a control of an application, etc.; the events include mouse clicks, keyboard key presses, timer timeouts, etc., so as to realize the subsequent recording and recording of the process of the user executing task A.

[0067] Referring to Figure 4B, Initializing the recording environment 311 may include initializing the data buffer 3113. To ensure the accuracy of recording and capture, a specific event buffer and filtering mechanism may also be established during the recording process to avoid interference from irrelevant events.

[0068] Return to Figure 4A , The functions of the operation recording module 310 include capturing user operations 312. For the sake of illustration, in conjunction with Figure 4C , Exemplarily, Figure 4C FIG. shows a schematic diagram of an exemplary function 400C for capturing user operations according to some embodiments of the present disclosure. Refer to Figure 4C , Capturing user operation 312 includes listening for DOM events 3121. DOM is a tree structure used to represent HTML or XML documents, where each element (such as 、 <button>Elements such as attributes and text nodes can all be regarded as nodes in a tree. Listening for DOM events means setting an "observer" on a certain node in the DOM tree. When a specific event (such as a user click, mouse movement, keyboard key press, etc.) occurs on that node, the associated code will be triggered and executed. In some embodiments, specific scripts can be injected to capture user interaction events during the execution of Task A in a web application.

[0069] Reference Figure 4C , capturing user operation 312 includes capturing UI interaction events 3122. UI interaction events are a series of actions triggered when a user operates on an application interface, such as clicking a button, swiping the screen, entering text, etc. Capturing these events means enabling the application to perceive these user operations through programming and execute corresponding processing logics according to different events, thereby realizing interaction with the user. In some embodiments, user interface element recognition technology can be used to identify the UI elements with which the user interacts during the execution of Task A. Alternatively, a multi-level selector can also be generated.

[0070] Reference Figure 4C , capturing user operation 312 includes recording a timestamp and context 3123. Recording a timestamp and context means obtaining the time information of an event occurrence (such as when a form submission is triggered during the execution of Task A by the user) and the associated environment and status information at the same time, and saving this information. For example, if the execution steps of Task A include Step 1, Step 2, and Step 3, then the execution times of Step 1, Step 2, and Step 3 and the relevant steps in the context, all this information will be captured.

[0071] By means of capturing user operation 312, interaction events of the user in the web page can be captured, the user interface elements with which the user interacts can be identified in real time, the operations of the user on the user interface application can also be identified, and the parameter information and corresponding context information called by the background application interface can also be recorded.

[0072] Return to Figure 4A , the functions of the operation recording module 310 include operation sequence preprocessing 313. For ease of explanation, in combination with Figure 4D , exemplarily, Figure 4D FIG. shows a schematic diagram of an example function 400D of operation sequence preprocessing according to some embodiments of the present disclosure. As Figure 4D shown, operation sequence preprocessing 313 may include filtering out irrelevant operations 3131. For example, during the capture of the user's execution of Task A, the user will inevitably slide the mouse meaninglessly, and then this meaningless mouse sliding operation will be filtered out.

[0073] As Figure 4D shown, the operation sequence preprocessing 313 may include combining related consecutive operations 3132. Combining Figure 1 , in the process of executing task A, step 1, step 2, and step 3 are consecutive, then step 1, step 2, and step 3 may be pre-combined together.

[0074] As Figure 4D shown, the operation sequence preprocessing 313 may include canonical description 3133. As mentioned above, in order to generate a general model context protocol service subsequently, it is also necessary to define a standardized service interface. Therefore, when preprocessing the operation sequence, some structural descriptions in the operation sequence can be normalized. For example, a button is button, and a text box is text.

[0075] With the aid of Figures 4A - 4D the described exemplary operation recording module, the operation steps of the user executing task A can be accurately captured, providing a basic template for the subsequent automated execution of task B.

[0076] Returning Figure 3 , in some embodiments, the model context protocol service generation module 320 is used to convert the operation sequence into a standard model context protocol service. For ease of illustration, the following will be described in conjunction with Figures 5A - 5D . Exemplarily, Figure 5A shows a schematic diagram of an example function 500A of the model context protocol service generation according to some embodiments of the present disclosure. Referring to Figure 5A , the model context protocol service generation module 320 includes semantic analysis and segmentation 321. For ease of illustration, in conjunction with Figure 5B , exemplarily, Figure 5B shows a schematic diagram of an example function 500B of the semantic analysis and segmentation according to some embodiments of the present disclosure. Referring to Figure 5B , the semantic analysis and segmentation 321 includes analyzing the semantic structure 3211 of the operation sequence. After obtaining the operation steps for completing task A, the semantic information of the operation steps for completing task A can be extracted. For example, element semantic analysis can be used to identify the functional attributes in the interaction elements, such as buttons, input boxes, and drop-down menus, etc. Then, operation intention recognition can be used to infer the operation intention. For example, the operation intention of the user in the process of completing task A can be inferred according to the interaction mode and context. For example, according to step 1 and step 2, it can be inferred that step 3 is an operation such as querying, filtering, or submitting. Then, data flow tracking can be used to record the complete path of the input, transformation, and output of the data. Finally, context state capture can be used to capture the state change of the system after completing task A.

[0077] Referring to Figure 5B , semantic analysis and segmentation 321 includes identifying a functionally complete operation subsequence 3212. Since the operation subsequence is functionally independent, it is necessary to define which steps in the operation steps of task A are inseparable and which are consecutive, so as to realize the identification of the boundary of the functionally complete operation subsequence. For example Figure 1 As shown, [step 1, step 2, step 3] can be a functionally complete subsequence; [step 4] can be a functionally complete subsequence; [step 5, step 6] can be a functionally complete subsequence; [step 7] can also be a functionally complete subsequence.

[0078] Reference Figure 5B , semantic analysis and segmentation 321 includes determining the boundary 3213 of the model context protocol service. In order to achieve the generality and portability of executing other tasks such as task B in the subsequent process, the specific operation subsequence can be converted into a model context protocol service. Since the model context protocol service is functionally independent, that is, it is necessary to realize the identification of the functional boundary of the model context protocol service. The principle is the same as that of the above-mentioned realization of the boundary identification of the operation subsequence. In some embodiments, the functional boundary of each model context protocol service can be determined according to the functional boundary identification technology of multi-dimensional semantic analysis.

[0079] For example, the functional boundary can be determined based on the semantic correlation score between each operation step in the operation sequence. For example, a graph can be constructed with each step in the operation sequence as a node and the semantic correlation score as an edge, and then a design area detection algorithm can be applied to the graph to identify highly cohesive subgraphs. After verifying the functional integrity and independence of each subgraph, the highly cohesive subgraphs can be converted into independent model context protocol services.

[0080] For example, the boundary of the model context protocol service can also be defined by means of the semantic information of each operation step in the operation sequence. For example, segmentation can be performed according to the input and output boundaries of the data stream to achieve the definition of the functional boundary. For example, the functional boundary of the model context protocol service can also be determined according to the significant change points of the system state change after the task is completed.

[0081] For example, the operation sequence of task A can be converted into 4 functionally independent services: data extraction service, data conversion service, data analysis service, and report generation service.

[0082] Alternatively, specific domain knowledge can also be combined to assist in determining the identification of service boundaries. For example, the steps of generating a report can be divided into data extraction, data transformation, data analysis, and report generation. According to the knowledge of data processing, data extraction and data transformation can be defined as an independent service for data extraction and transformation. At this time, the four steps can be defined as three independent functions: data extraction and transformation, data analysis, and report generation. As Figure 1 shown, there are service boundaries between [Step 1, Step 2, Step 3] and [Step 4]; there is a service boundary between [Step 4] and [Step 5, Step 6]; there is also a service boundary between [Step 5, Step 6] and [Step 7].

[0083] Back to Figure 5A , the model context protocol service generation module 320 includes parameter extraction and abstraction 322. In order to apply the task parameters of Task B to the model context protocol service related to Task A, it is also necessary to extract and abstract the parameters involved in Task A. For the sake of illustration, combined with Figure 5C , exemplarily, Figure 5C FIG. shows a schematic diagram of an exemplary function 500C of parameter extraction and abstraction according to some embodiments of the present disclosure. Referring to Figure 5C , parameter extraction and abstraction 322 includes identifying the variable part 3221 in the operation. For the automated execution of Task B in subsequent processes, the task parameters of Task B are necessarily required. However, how the task parameters of Task B are applied to the model context protocol service determined from the operation steps of Task A necessarily requires determining which parameter values of the operation steps in Task A are variable and which are immutable. Only in this way can the parameters of Task B be applied to the model context protocol service determined from the completion process of Task A, thereby realizing the automated execution of Task B.

[0084] Referring to Figure 5C , parameter extraction and abstraction 322 includes determining parameter types and constraint conditions 3222. For example, the candidate parameter points of Task A can be identified. The candidate parameter points are variable. The candidate parameter points can be parameter types such as form input, query conditions, etc. Then, according to each candidate parameter point, the parameter definition of each parameter point is generated, that is, the parameter description, type, and constraint conditions. Similarly, the parameter definitions of the immutable parameter points are also determined based on this. For example, a certain candidate parameter point is a time range parameter: type = date range, format = YYYY-MM-DD, description = time range.

[0085] Referring to Figure 5C , Parameter extraction and abstraction 322 includes generating parameter descriptions and example values 3223. For example, a certain candidate parameter point is a data type parameter: type = enumeration, optional values = [business, inventory, customer, finance], description = the data type to be analyzed. Similarly, the parameter descriptions and example values of immutable parameter points can also be generated. For example, for the label type parameter, type = button, format: none, description = next step.

[0086] For example, through the parameter abstraction technology, identifying the key parameters in task A and generating parameter definitions can be as follows:

[0087] Time range parameter: type = date range, format = YYYY-MM-DD, description = the time range covered by the report;

[0088] Data type parameter: type = enumeration, optional values = [business, inventory, customer, finance], description = the data type to be analyzed;

[0089] Report format parameter: type = enumeration, optional values = [Excel, PDF, Slide], description = the output report format;

[0090] Receiver parameter: type = string array, format = email, description = the list of report receivers.

[0091] Back to Figure 5A , The model context protocol service generation module 320 includes model context protocol service encapsulation 323. After determining the boundaries of the model context protocol services for task A, these model context protocol services can be encapsulated into functionally independent service entities. For ease of explanation, in combination with Figure 5D , Exemplarily, Figure 5D shows a schematic diagram of an example function 500D of model context protocol service encapsulation according to some embodiments of the present disclosure. Referring to Figure 5D , Model context protocol service encapsulation 323 includes generating service interface definitions 3231. For example, in the encapsulated model context protocol service, there are also standardized service interface definitions to enable communication with external data sources and tools.

[0092] Referring to Figure 5D , Model context protocol service encapsulation 323 includes converting the operation sequence into executable code 3232. As mentioned before, after obtaining the functionally independent operation subsequences, these operation subsequences can be converted into executable code, so as to be able to execute task B subsequently.

[0093] Referring to Figure 5D , the model context protocol service encapsulation 323 includes adding verification and error handling logic 3233. In the encapsulated model context protocol service, parameter verification rules are also included, and verification rules can be generated according to the parameter definitions of variable parameters to achieve the accuracy of task execution. For example, if a certain task parameter of task B is numeric, then this numeric parameter is not allowed to be passed into a string. In the encapsulated model context protocol service, error handling logic can also be encapsulated. For example, when it is detected that data in a certain area is missing, it can be automatically adjusted to only analyze the area with complete data.

[0094] Reference Figure 5D , the model context protocol service encapsulation 323 includes generating service documents 3234. In some embodiments, the service documents include descriptions and usage instructions of the model context protocol service, so that it can be convenient for technicians to provide repair support when errors occur later.

[0095] For example, standardized interfaces, execution codes corresponding to each service, parameter verification logic, error handling mechanisms, and service description documents are encapsulated in the data extraction service, data conversion service, data analysis service, and report generation service.

[0096] With the help of Figures 5A - 5D the described exemplary model context protocol generation module, specific operation steps can be converted into standard services, enabling these services to be reused in different systems or environments, improving generality and portability. At the same time, with the help of the model context protocol generation module, the complexity of operations is also reduced, facilitating subsequent management and maintenance.

[0097] Return Figure 3 , in some embodiments, the service storage and indexing module 330 is used to store the model context protocol service. For the sake of illustration, the following will be described in conjunction with Figure 6 to describe. Exemplarily, Figure 6 shows a schematic diagram of an example function 600 of the service storage and indexing module according to some embodiments of the present disclosure. As Figure 6 shown, the functions of the service storage and indexing module 330 include assigning a unique identifier 3301 for the operation service. After converting the operation steps of task A into a functionally independent model context protocol service, a unique identifier needs to be assigned to each service so that it can be accurately called when performing other tasks subsequently. Alternatively, a version number can also be assigned to each service so that in subsequent processes, users can call different versions of the service as needed.

[0098] As Figure 6 As shown, the functions of the service storage and indexing module 330 include adding service metadata and tags 3302. After obtaining the operation steps of Task A, the names, descriptions, and classification tags of each parameter in the operation sequence of Task A can be stored in the service storage and indexing module, thereby ensuring the accuracy and effectiveness of the data.

[0099] As Figure 6 shown, the functions of the service storage and indexing module 330 include storing services in the service library 3303. After converting the operation steps of Task A into function-independent model context protocol services, these function-independent model context protocol services, such as data extraction services, data conversion services, data analysis services, and report generation services, also need to be stored in the service storage and indexing module to enable subsequent accurate invocation.

[0100] As Figure 6 shown, the functions of the service storage and indexing module 330 include establishing service indexes 3304. After converting the operation steps of Task A into function-independent model context protocol services, there are sequential dependencies between the model context protocol services. Therefore, service indexes can be established for these function-independent model context protocol services to provide support for generating subsequent execution paths. For example, data extraction service → data conversion service → data analysis service → report generation service.

[0101] Return Figure 3 , in some embodiments, the large model decision-making module 340 is used to make tool selection and execution decisions based on user intent. For the sake of illustration, the following will be described in conjunction with Figures 7A - 7D to describe. Exemplarily, Figure 7A shows a schematic diagram of an example function 700A of the large model decision-making module according to some embodiments of the present disclosure. As Figure 7A shown, the large model decision-making module 340 includes intent understanding and decomposition 341. For the sake of illustration, in conjunction with Figure 7B , Figure 7B shows a schematic diagram of an example function �00B of intent understanding and decomposition according to some embodiments of the present disclosure.

[0102] Refer to Figure 7B , intention understanding and decomposition 341 includes extracting the core intention and parameters 3411. After the analysis of the operation steps for task A is completed, in order to achieve the goal of automatically completing task B, it is first necessary to know what task B is, that is, it is necessary to know the user's intention. And the user's intention can be determined according to the user's input. In some embodiments, the user input can be in text form or in voice form, but no matter what form, it is ultimately in natural language form. For example, the user's core intention and parameters can be determined according to the user input, that is, task B and the task parameters of task B are determined. For example, a semantic parsing model such as a decision model can be used to extract the core part of the user input.

[0103] For example, if the user input is "Generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", the decision model can analyze and identify that task B is to generate a business performance report, where the core action: generate a report; key parameters: time range = last quarter, data type = business, region = all, format = PDF, recipient = business team.

[0104] Reference Figure 7B , intention understanding and decomposition 341 includes understanding the context 3412 in combination with the conversation history. In some embodiments, a semantic parsing model such as a decision model can also be used to understand the user's true intention by combining the user's conversation history, that is, the context.

[0105] Reference Figure 7B , intention understanding and decomposition 341 includes decomposing complex intentions into subtasks 3413. If the user's input description of task B is too complex, or task B is too complex, then task B can be decomposed into multiple tasks, such as task B1, task B2, and task B3, etc.

[0106] Back to Figure 7A , the large model decision module 340 includes service retrieval and screening 342. For ease of description, in combination with Figure 7C , Figure 7C FIG. shows a schematic diagram of an example function 700C of service retrieval and screening according to some embodiments of the present disclosure. As Figure 7C shown, service retrieval and screening 342 includes retrieving services based on intention 3421. In order to enable task B to be automatically executed, it is possible to retrieve the model context protocol services that are functionally independent of task A to determine the model context service to be executed that is most suitable for task B.

[0107] As Figure 7C As shown, service retrieval and screening 342 includes matching degree 3422 based on computing services and intents, and screening out the most suitable service candidate set 3423. To enable task B to be automatically executed, the function-independent model context protocol services of task A can be retrieved to determine the model context service to be executed that is most suitable for task B. In some embodiments, the semantic relevance between the intent expressed by the user input and the independent services of task A can be calculated, and a decision model can be used to determine the model context service to be executed that is most suitable for task B. For example, the decision model plans that the set of model context protocol services to be executed for task B includes: data extraction service, data transformation service, data analysis service, and report generation service, which are the model context services to be executed that are most suitable for task B.

[0108] Back to Figure 7A , the large model decision module 340 includes execution path planning 343. For ease of description, in combination with Figure 7D , Figure 7D Figure 700D shows a schematic diagram of an example function of execution path planning according to some embodiments of the present disclosure. As Figure 7D shown, execution path planning 343 includes determining the service call order 3431, planning the data flow path 3432, and handling the dependencies between services 3433.

[0109] After the decision model plans the set of model context protocol services to be executed for task B, the execution path of service calls (which can also be called the service call sequence) and the planned path of data flow can be determined based on the dependencies between these model context services to be executed 150 (determining the dependencies with the help of service indexes). In some embodiments, a call dependency graph with these services as nodes can be constructed, and then the execution path can be determined with the help of a path planning algorithm. For example, the execution path and the data flow path can be: data extraction → data transformation → data analysis → report generation.

[0110] With the help of Figures 7A - 7D , it is possible to select the optimal execution path for task B, ensure the correct execution of task B, and at the same time achieve intelligent task allocation.

[0111] Return Figure 3 , in some embodiments, the tool execution and monitoring module 350 is used to execute the model context protocol services and monitor the execution status. For ease of explanation, the following will be described in combination with Figures 8A - 8C to describe. Exemplarily, Figure 8A Figure 800A shows a schematic diagram of an example function of the tool execution and monitoring module according to some embodiments of the present disclosure. As Figure 8A shown, the tool execution and monitoring module 350 includes parameter preparation and verification 351. For ease of explanation, in combination with Figure 8B , Figure 8B A schematic diagram showing an example function 800B of parameter preparation and verification according to some embodiments of the present disclosure is shown. As Figure 8B shown, the parameter preparation and verification 351 includes extracting parameter values 3511 from the user's intention. For example, if the user input is "Generate a comparison report of business performance for all regions in the last quarter", the decision model can analyze and identify the key parameters of task B: time range = last quarter, data type = business, region = all, format = PDF, recipient = business team.

[0112] As Figure 8B shown, the parameter preparation and verification 351 includes filling in missing parameters 3512. For example, for the user input "Generate a comparison report of business performance for all regions in the last quarter and send it to the business team in PDF format", the generated key parameters do not include the parameter of the report template. Then, for the unity of data form, necessary default parameters can be supplemented. For example, report template = standard business performance template.

[0113] As Figure 8B shown, the parameter preparation and verification 351 includes verifying the validity of the parameters 3513. The validity of the parameters can be verified by means of parameter verification rules. In some embodiments, verification rules can be generated according to the parameter definitions (i.e., parameter descriptions, types, and constraint conditions) of variable parameters to achieve the accuracy of task execution. For example, a numeric parameter does not allow a string parameter value to be input. Another example is that a certain parameter is a time range parameter: type = date range, format = YYYY-MM-DD, description = time range covered by the report. Then, the corresponding format cannot be filled with a character long string. If the task parameter verification of task B passes, then task B can be continued to be executed.

[0114] Returning to Figure 8A , the tool execution and monitoring module 350 includes service execution and monitoring 352. For the sake of easy explanation, in combination with Figure 8C , Figure 8C a schematic diagram showing an example function 800C of service execution and monitoring according to some embodiments of the present disclosure is shown. As Figure 8C shown, the service execution and monitoring 352 includes calling services 3521 in the planned order. As mentioned above, the large model can plan the execution path for task B. Therefore, during the execution of task B, relevant services need to be called strictly in the planned order, for example, strictly executed in the order of data extraction service → data conversion service → data analysis service → report generation service.

[0115] As Figure 8C As shown, service execution and monitoring 352 includes monitoring the execution status and intermediate results 3522 and handling abnormal situations 3523. During the execution of task B, it is also necessary to monitor the execution status of task B in real time. If data in a certain area is missing or abnormal, it can be automatically adjusted to only analyze the areas with complete data, or add steps for data verification to the abnormal data areas. In some embodiments, during the execution of task B, key monitoring points can also be determined in advance, and user confirmation can be further obtained at the key monitoring points to ensure the smooth execution of task B. Alternatively, the most suitable visualization chart type can also be dynamically selected according to the characteristics of the data.

[0116] With the Figures 8A - 8C tool execution and monitoring module 350 shown, it is possible to monitor the execution process of task B in real time and react in a timely manner in case of abnormalities, thus ensuring the smooth execution of task B. In some embodiments, when a certain to-be-executed model online protocol service of task B goes wrong, a new to-be-executed model context protocol service can be determined to ensure the smooth execution of task B.

[0117] Return Figure 3 , in some embodiments, the user interaction module 360 is used to receive user intentions and display execution results. For the sake of convenience of description, the following will be described in conjunction with Figures 9A - 9C to describe. Exemplarily, Figure 9A shows a schematic diagram of an exemplary function 900A of the user interaction module according to some embodiments of the present disclosure. As Figure 9A shown, the user interaction module 360 includes receiving user instructions 361. For the sake of convenience of description, in conjunction with Figure 9B , Figure 9B shows a schematic diagram of an exemplary function 900B of receiving user instructions according to some embodiments of the present disclosure. As Figure 9B shown, receiving user instructions includes receiving user input 3611 and preprocessing the input 3612. For example, when the user enters "Generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format" in the user interface, the user input can be received. If the user inputs in the form of voice, then the user input can also be preprocessed, and the voice can be converted into the form of natural language. If the user's input is too complex, then the user's input can also be processed in segments. In some embodiments, if some data needs to be confirmed with the user during the execution of task B, the user input can be received again. In some embodiments, during the execution of task B, key monitoring points can also be determined in advance, and user confirmation can be further obtained at the key monitoring points to ensure the smooth execution of task B.

[0118] As Figure 9A As shown, the user interaction module 360 includes result processing and feedback 362. For the sake of illustration, in conjunction with Figure 9C , Figure 9C FIG. 900C is a schematic diagram showing an example function of result processing and feedback according to some embodiments of the present disclosure. As Figure 9C shown, the result processing and feedback 362 includes integrating execution results 3621. After task B is executed, the execution results of task B can be integrated. For example, an execution summary can be generated, such as "Completed business performance analysis of 6 regions and generated a 15-page PDF report".

[0119] As Figure 9C shown, the result processing and feedback 362 includes generating user-friendly feedback 3622. For example, for another example, highlighted text such as "The business in the eastern region has the fastest growth rate, with an increase of 23%" can also be displayed on the user interface. The result processing and feedback 362 also includes updating the execution history 3623 and collecting user satisfaction feedback 3624. In some embodiments, when task B is executed, the execution history can be updated. In some embodiments, the user can also be interacted with by a language prompt such as "Is this report helpful to you?" to collect the user's feedback. After collecting the user's feedback, a score for the execution of task B can be generated in combination with indicators in multiple dimensions such as the accuracy, efficiency, and stability of task execution, so that the decision-making model can be adjusted based on the user's feedback and the score.

[0120] In some embodiments, according to the user's task execution history, the user often immediately executes task C after executing task B, then task C can be automatically executed for the user. For example, the user often immediately generates an inventory report after generating a business report, then the inventory report can be immediately generated for the user.

[0121] It can be understood that the various modules in the example system architecture 300 can be interconnected through standardized interfaces and form a complete processing flow.

[0122] According to the embodiments of the present disclosure, the operation sequence of the user for the first task can be reused to intelligently match and execute the second task of the user, greatly improving work efficiency and reducing the manual operation time and workload. At the same time, this method of converting the operation steps into a model context protocol service can also effectively improve the scalability and integration of task execution, so as to be able to handle new tasks and new requirements. In addition, by means of natural language interaction, the usage threshold is reduced, enabling non-professional personnel to use it, and comprehensively improving the practicality and scope of application of task execution. That is, through this method, the simple operations of the user can be automatically converted into a model online protocol service that can be dynamically combined to implement task execution, improving the user experience.

[0123] Figure 10 FIG. 1 shows a block diagram of an apparatus 1000 for performing tasks according to some embodiments of the present disclosure. As Figure 10 shown, the apparatus 1000 includes an operation sequence acquisition module 1002 configured to acquire an operation sequence of a user for a first task on a user interface, where the operation sequence includes operation steps required to complete the first task. The apparatus 1000 includes an operation step conversion module 1004 configured to convert the operation steps into a model context protocol service based on the operation sequence. The apparatus 1000 includes a user input acquisition module 1006 configured to acquire a user input, where the user input indicates a second task, and the user input includes at least one of text input or voice input. The apparatus 1000 includes a to-be-executed model context protocol service determination module 1008 configured to determine a to-be-executed model context protocol service based on the second task, where the to-be-executed model context protocol service is associated with the second task. The apparatus 1000 includes a second task execution module 1010 configured to execute the second task based on the task parameters of the second task and the to-be-executed model context protocol service, where the second task is composed of the to-be-executed model context protocol service associated with the second task, and the task parameters of the second task are determined based on the user input. The apparatus 1000 includes an execution result display module 1012 configured to display an execution result for the second task on the user interface.

[0124] In some embodiments, the operation step conversion module 1004 includes: a first determination module configured to determine a semantic relevance score between operation steps. A first conversion module configured to convert the operation steps into a model context protocol service based on the semantic relevance score.

[0125] In some embodiments, the conversion module includes: a first generation module configured to generate a graph with operation steps as nodes and semantic relevance scores as edges; a second determination module configured to determine a highly cohesive subgraph based on the graph, where the highly cohesive subgraph indicates a model context protocol service; a second conversion module configured to convert the operation steps into a model context protocol service in response to the highly cohesive subgraph being independent.

[0126] In some embodiments, the operation sequence includes a data stream associated with the first task, where the conversion module includes: a third determination module configured to determine the boundaries of the input and output of the data stream. A third conversion module configured to convert the operation steps into a model context protocol service based on the boundaries.

[0127] In some embodiments, the conversion module includes: a fourth determination module configured to determine a change point of the system state associated with the first task after the first task is completed. A fifth determination module configured to convert the operation steps into a model context protocol service based on the change point.

[0128] In some embodiments, the operation sequence includes the task parameters of the first task, and further includes: a sixth determination module configured to determine candidate parameter points in the task parameters of the first task, where the candidate parameter points indicate that the task parameters at the candidate parameter points are variable; a second generation module configured to generate a parameter definition based on the candidate parameter points, where the parameter definition includes a parameter description, a parameter type, and a constraint condition.

[0129] In some embodiments, it further includes: a third generation module configured to generate a verification rule based on the parameter definition, where the verification rule is used to verify the model context protocol service.

[0130] In some embodiments, the model context protocol service includes a service identifier, a parameter definition, and a verification rule.

[0131] In some embodiments, the second task execution module 1010 includes: a verification module configured to verify the task parameters of the second task based on the verification rule, where the verification rule is based on the first task; an execution module configured to, in response to successful verification of the task parameters of the second task, execute the to-be-executed model context protocol service associated with the second task based on the task parameters of the second task and the to-be-executed model context protocol service.

[0132] In some embodiments, it further includes: a seventh determination module configured to determine the second task and the task parameters of the second task by a decision model based on user input.

[0133] In some embodiments, the to-be-executed model context protocol service determination module 1008 includes: an eighth determination module configured to determine the semantic relevance between the second task and the model context protocol service; a ninth determination module configured to determine the to-be-executed model context protocol service from the model context protocol services by the decision model based on the semantic relevance.

[0134] In some embodiments, the ninth determination module includes: a tenth determination module configured to determine a set of to-be-executed model context protocol services associated with the second task from the model context protocol services based on the semantic relevance; a fourth generation module configured to generate a graph with the to-be-executed model context protocol services in the set of to-be-executed model context protocol services as nodes; and an eleventh determination module configured to determine an execution path based on the graph, where the execution path indicates the dependency relationship between the to-be-executed model context protocol services, and the to-be-executed model context protocol services are associated with the second task.

[0135] In some embodiments, the task parameters of the second task include first task parameters and second task parameters, where the first task parameters are determined based on user input, and the second task parameters are default values.

[0136] In some embodiments, it further includes: a monitoring module configured to monitor the execution process of the second task; a re-determination module configured to re-determine the model context protocol service to be executed in response to an error occurring in the execution process.

[0137] In some embodiments, it further includes: an acquisition module configured to acquire the user's feedback on the completion of the second task after the execution of the second task is completed; and an eleventh determination module configured to determine the score for the completion of the second task; an adjustment module configured to adjust the decision model based on the feedback and the score.

[0138] In some embodiments, it further includes: a twelfth determination module configured to determine the key monitoring points during the execution process of the second task; a third task parameter acquisition module configured to acquire the third task parameters of the second task input by the user in response to the execution process of the second task reaching the key monitoring points.

[0139] In some embodiments, the operation sequence acquisition module 1002 includes: an initialization module configured to initialize the system state associated with the first task; a recording module configured to record the interaction operations of the user to complete the first task on the user interface; a fourth conversion module configured to convert the interaction operations into an operation sequence.

[0140] In some embodiments, the recording module includes: a first recording module configured to record the functional attributes of the interaction elements corresponding to the interaction operations; record the paths of data stream input, conversion, and output, where the data stream is associated with the first task; and a first recording module configured to record the change points of the system state associated with the first task after the first task is completed.

[0141] In some embodiments, it further includes: a thirteenth determination module configured to determine the third task and the task parameters of the third task in response to the completion of the second task; and a third task execution module configured to execute the third task on the user interface, where the third task is associated with the second task.

[0142] Figure 11 The block diagram of the device 1100 capable of implementing multiple embodiments of the present disclosure is shown. As Figure 11 As shown, device 1100 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 1101, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 1102 or computer program instructions loaded from a storage unit 1108 into a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of device 1100 can also be stored. The CPU / GPU 1101, ROM 1102, and RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104. Although not shown in Figure 11 , device 1100 may also include a coprocessor.

[0143] Multiple components in device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a magnetic disk, an optical disc, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0144] Each of the above-described methods or processes can be executed by the CPU / GPU 1101. For example, in some embodiments, the method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 1100 via the ROM 1102 and / or the communication unit 11'09. When the computer program is loaded into the RAM 1103 and executed by the CPU / GPU 1101, one or more steps or actions of the above-described methods or processes can be executed.

[0145] In some embodiments, the above-described methods and processes can be implemented as a computer program product. The computer program product may include a computer-readable storage medium having thereon computer-readable program instructions for performing various aspects of the present disclosure.

[0146] A computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punched card or raised structures in grooves storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not construed as being an instantaneous signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., an optical pulse through an optical fiber cable), or an electrical signal transmitted through a wire.

[0147] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to respective computing / processing devices, or downloaded to an external computer or an external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include a copper transmission cable, an optical fiber transmission, a wireless transmission, a router, a firewall, a switch, a gateway computer, and / or an edge server. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0148] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state-setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages and conventional procedural programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, alternatively, may be connected to an external computer (e.g., through the Internet using an Internet service provider). In some embodiments, by using the state information of the computer-readable program instructions to customize an electronic circuit, such as a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA), the electronic circuit can execute the computer-readable program instructions to implement various aspects of the present disclosure.

[0149] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is produced that implements the functions / acts specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, the programmable data processing device, and / or other devices to work in a specific manner. Thus, the computer-readable medium storing the instructions includes a manufactured article that includes instructions for implementing various aspects of the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0150] The computer-readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device, such that a series of operation steps are executed on the computer, other programmable data processing device, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing device, or other device to implement the functions / acts specified in one or more blocks of the flowchart and / or block diagram.

[0151] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0152] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the market, or to enable other ordinary skill in the technical field to understand the embodiments disclosed herein.< / button>

Claims

1. A method for performing a task, comprising: Obtaining an operation sequence of a user for a first task on a user interface, the operation sequence including operation steps required to complete the first task; Based on the operation sequence, converting the operation steps into a model context protocol service; Obtaining a user input, the user input indicating a second task, the user input including at least one of text input or voice input; Based on the second task, determining a model context protocol service to be executed, the model context protocol service to be executed being associated with the second task; Based on the task parameters of the second task and the model context protocol service to be executed, performing the second task, the second task being composed of the model context protocol service to be executed associated with the second task, and the task parameters of the second task being determined based on the user input; And Displaying an execution result of the second task on the user interface.

2. The method according to claim 1, wherein converting the operation steps into a model context protocol service based on the operation sequence includes: Determining a semantic relevance score between the operation steps; And Based on the semantic relevance score, converting the operation steps into the model context protocol service.

3. The method according to claim 2, wherein converting the operation steps into the model context protocol service based on the semantic relevance score includes: Generating a graph with the operation steps as nodes and the semantic relevance score as edges; Based on the graph, determining a highly cohesive subgraph, the highly cohesive subgraph indicating the model context protocol service; And In response to the highly cohesive subgraph being independent, converting the operation steps into the model context protocol service.

4. The method according to claim 1, the operation sequence including a data stream associated with the first task, wherein converting the operation steps into a model context protocol service based on the operation sequence includes: Determining boundaries of inputs and outputs of the data stream; And Based on the boundaries, converting the operation steps into the model context protocol service.

5. The method according to claim 1, wherein converting the operation steps into a model context protocol service based on the operation sequence includes: After the first task is completed, determining a change point of the system state associated with the first task; And Based on the change point, converting the operation steps into the model context protocol service.

6. The method according to any one of claims 2-5, the operation sequence including task parameters of the first task, further comprising: Determining candidate parameter points in the task parameters of the first task, the candidate parameter points indicating that the task parameters at the candidate parameter points are variable; And Based on the candidate parameter points, generating a parameter definition, the parameter definition including a parameter description, a parameter type, and constraint conditions.

7. The method according to claim 6, further comprising: Based on the parameter definition, generating a verification rule, the verification rule being used to verify the model context protocol service.

8. The method according to claim 7, wherein the model context protocol service includes a service identifier, the parameter definition, and the verification rule.

9. The method according to claim 7, wherein performing the second task based on the task parameters of the second task and the to-be-executed model context protocol service includes: Verifying the task parameters of the second task based on the verification rule, the verification rule being based on the first task; And In response to successful verification of the task parameters of the second task, performing the to-be-executed model context protocol service associated with the second task based on the task parameters of the second task and the to-be-executed model context protocol service.

10. The method according to claim 1, further comprising: Determining, by a decision model, the second task and the task parameters of the second task based on the user input.

11. The method according to claim 10, wherein determining the to-be-executed model context protocol service based on the second task includes: Determining the semantic relevance between the second task and the model context protocol service; And Based on the semantic relevance, determining, by the decision model, the to-be-executed model context protocol service from the model context protocol services.

12. The method according to claim 11, wherein determining, by the decision model, the to-be-executed model context protocol service from the model context protocol services based on the semantic relevance includes: Based on the semantic relevance, determining a set of to-be-executed model context protocol services associated with the second task from the model context protocol services; And Generating a graph with the to-be-executed model context protocol services in the set of to-be-executed model context protocol services as nodes; And Based on the graph, determining an execution path, the execution path indicating the dependency relationship between the to-be-executed model context protocol services, the to-be-executed model context protocol services being associated with the second task.

13. The method according to claim 10, wherein the task parameters of the second task include first task parameters and second task parameters, the first task parameters being determined based on the user input, and the second task parameters being default values.

14. The method according to claim 1, further comprising: Monitoring the execution process of the second task; And In response to an error in the execution process, re-determining the to-be-executed model context protocol service.

15. The method according to claim 1, further comprising: After the execution of the second task is completed, obtaining the user's feedback on the completion of the second task; And Determining a score for the completion of the second task; Adjusting the decision model based on the feedback and the score.

16. The method according to claim 1, further comprising: Determining key monitoring points during the execution process of the second task; In response to the execution process of the second task being at the key monitoring point, obtaining third task parameters of the second task input by the user.

17. The method according to claim 1, wherein obtaining an operation sequence of a user for a first task on a user interface comprises: Initializing a system state associated with the first task; Recording an interaction operation of the user completing the first task on the user interface; And Converting the interaction operation into the operation sequence.

18. The method according to claim 16, wherein recording an interaction operation of the user completing the first task on the user interface comprises: Recording a functional attribute of an interaction element corresponding to the interaction operation; Recording a path of data stream input, conversion, and output, the data stream being associated with the first task; And Recording a change point of the system state associated with the first task after the first task is completed.

19. The method according to claim 1 further comprises: Determining a third task in response to completion of a second task; And Executing the third task on the user interface, the third task being associated with the second task.

20. An apparatus for performing a task, comprising: An operation sequence obtaining module, configured to obtain an operation sequence of a user for a first task on a user interface, the operation sequence comprising operation steps required to complete the first task; An operation step conversion module, configured to convert the operation steps into a model context protocol service based on the operation sequence; A user input obtaining module, configured to obtain a user input, the user input indicating a second task, the user input comprising at least one of text input or voice input; A model context protocol service to be executed determining module, configured to determine a model context protocol service to be executed based on the second task, the model context protocol service to be executed being associated with the second task; A second task execution module, configured to execute the second task based on task parameters of the second task and the model context protocol service to be executed, the second task being composed of the model context protocol service to be executed associated with the second task, the task parameters of the second task being determined based on the user input; And An execution result display module, configured to display an execution result for the second task on the user interface.

21. An electronic device, comprising: A processor; And A memory coupled to the processor, the memory having instructions stored therein, which when executed by the processor cause the electronic device to execute the method according to any one of claims 1 to 19.

22. A computer program product, the computer program product being tangibly stored on a non-transitory computer-readable medium and comprising machine-executable instructions, which when executed cause a machine to implement the method according to any one of claims 1 to 19.

Images