Method, device, equipment and product for executing task

Abstract

The invention relates to a method, a device, equipment and a product for executing tasks. The method comprises the following steps: acquiring an operation sequence of a user for a first task on a user interface, wherein the operation sequence comprises operation steps required for completing the first task; the method includes converting the operating steps into a model context protocol service based on the sequence of operations. The method includes obtaining a user input, the user input indicating a second task, the user input including at least one of a text input or a voice input. The method includes determining a model context protocol service to be executed based on the second task, the model context protocol service to be executed being associated with the second task. The method includes executing a second task based on task parameters of the second task and service parameters of a to-be-executed model context protocol service, the second task being composed of the to-be-executed model context protocol service associated with the second task, and the task parameters of the second task determining and displaying an execution result of the second task based on user input.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and more particularly to methods, devices, apparatuses, and products for performing tasks. Background Art

[0002] In today's highly digitalized and information-based working environment, various software applications and automation tools have improved human work efficiency. From process automation to intelligent decision support, from low-code development to cross-platform collaboration, tool innovation is reshaping the traditional working model. For example, robotic process automation (RPA) can complete repetitive tasks in a code-free way; intelligent document processing tools can achieve efficient analysis of unstructured data through OCR and NLP technology; and DevOps automation platforms can shorten the software delivery cycle to minutes. Summary of the invention

[0003] In a first aspect of an embodiment of the present disclosure, a method for executing a task is provided. The method includes obtaining an operation sequence of a user for a first task on a user interface, the operation sequence including the operation steps required to complete the first task. The method includes converting the operation steps into a model context protocol service based on the operation sequence. The method includes obtaining user input, the user input indicating a second task, the user input including at least one of text input or voice input. The method includes determining a model context protocol service to be executed based on the second task, the model context protocol service to be executed being associated with the second task. The method includes executing the second task based on the task parameters of the second task and the model context protocol service to be executed, the second task being composed of the model context protocol service to be executed associated with the second task, the task parameters of the second task being determined based on the user input and displaying the execution result of the second task.

[0004] In a second aspect of an embodiment of the present disclosure, a device for executing a task is provided. The device includes an operation sequence acquisition module configured to acquire an operation sequence of a user on a user interface for a first task, the operation sequence including the operation steps required to complete the first task. The device operation step conversion module is configured to convert the operation steps into a model context protocol service based on the operation sequence. The device includes a user input acquisition module configured to acquire user input, the user input indicates a second task, and the user input includes at least one of text input or voice input. The device includes a to-be-executed model context protocol service determination module configured to determine a to-be-executed model context protocol service based on the second task, the to-be-executed model context protocol service being associated with the second task. The device includes a second task execution module configured to execute the second task based on the task parameters of the second task and the to-be-executed model context protocol service, the second task being composed of the to-be-executed model context protocol service associated with the second task, the task parameters of the second task being determined based on the user input. The device includes an execution result display module configured to display the execution result of the second task on the user interface.

[0005] In a third aspect of the embodiments of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method of the first aspect.

[0006] In a fourth aspect of the embodiments of the present disclosure, a computer program product is provided, which is tangibly stored on a non-transitory computer-readable medium and includes machine-executable instructions, which, when executed, cause a machine to implement the method of the first aspect.

[0007] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein: Figure 1 A schematic diagram illustrating an example environment in which various embodiments of the present disclosure may be implemented; Figure 2 A flowchart of a method for performing a task according to some embodiments of the present disclosure is shown; Figure 3 A schematic diagram showing an example system architecture for performing tasks according to some embodiments of the present disclosure; Figures 4A-4D A schematic diagram showing example functions of an operation recording module according to some embodiments of the present disclosure; Figures 5A-5D A schematic diagram illustrating example functionality of Model Context Protocol (MCP) service generation according to some embodiments of the present disclosure; Figure 6 A schematic diagram showing example functions of a service storage and indexing module according to some embodiments of the present disclosure; Figures 7A-7D A schematic diagram showing example functions of a large model decision module according to some embodiments of the present disclosure; Figures 8A-8C A schematic diagram illustrating example functions of a tool execution and monitoring module according to some embodiments of the present disclosure; Figures 9A-9C A schematic diagram showing example functions of a user interaction module according to some embodiments of the present disclosure; Fig.10 A block diagram showing an apparatus for performing a task according to some embodiments of the present disclosure; and Fig.11 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0009] It is understood that all user-related data involved in this technical solution should be obtained and used after the user's authorization. This means that in this technical solution, if the user's personal information needs to be used, the user's explicit consent and authorization are required before obtaining this data, otherwise the relevant data will not be collected and used. It should also be understood that when implementing this technical solution, relevant laws and regulations should be strictly observed during the collection, use and storage of data, and necessary technologies and measures should be taken to protect the user's data security and ensure the safe use of data.

[0010] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0011] For example, when receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0012] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0013] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0014] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0015] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc. may refer to different or the same objects, unless explicitly stated. Other explicit and implicit definitions may also be included below.

[0016] As mentioned above, in today's highly digital and information-based working environment, various software applications and automation tools have greatly improved human work efficiency. In related technologies, users need to master programming knowledge to develop automation tools to achieve automated interactive processes. However, this technical threshold limits the participation of non-technical personnel. For users who lack programming basics, it is too difficult to develop automation tools on their own, making it difficult for them to convert repetitive tasks into automated processes. In addition, in related technologies, users can also use recording and playback tools to automate the execution of preset operations. Although such tools reduce the difficulty of operation, this limited function method can only perform simple repetitive tasks and cannot be flexibly adjusted according to the dynamic needs of users, and thus cannot meet the needs of users for complex tasks.

[0017] To this end, the present disclosure proposes a method for executing tasks, which can automatically convert simple user operations into model online protocol services that can be dynamically combined, thereby realizing the execution of tasks. In the scheme provided by the present disclosure, first, the operation sequence performed by the user on the user interface for the first task is obtained, and this operation sequence covers the various operation steps necessary to complete the first task. Then, according to the obtained operation sequence, the operation steps therein are converted into a model context protocol service. Subsequently, user input is received, and the user input is used to indicate a second task, and the input includes a form of text input or voice input. Next, according to the indicated second task, the model context protocol service to be executed associated therewith is determined. Finally, based on the task parameters of the second task and the service parameters of the model context protocol service to be executed, the second task consisting of the associated model context protocol service to be executed is executed, wherein the task parameters of the second task are determined according to the user input, and the execution result of the second task is displayed on the user interface after the execution is completed.

[0018] According to the embodiments of the present disclosure, the user's operation sequence for the first task can be reused to intelligently match and execute the user's second task, which greatly improves work efficiency and reduces the duration and workload of manual operations. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so as to cope with new tasks and new requirements. In addition, with the help of natural language interaction, the usage threshold is lowered, allowing non-professionals to use it, and the practicality and scope of application of task execution are comprehensively improved. That is, through this method, the user's simple operations can be automatically converted into model online protocol services that can be dynamically combined to achieve task execution and improve user experience.

[0019] Figure 1 1 is a schematic diagram of an example environment 100 in which various embodiments of the present disclosure may be implemented. Figure 1As shown, the example environment 100 includes Task A and Task B. In order to realize the automated completion of Task B, the operation steps of the task may be recorded first for analysis, so as to realize the execution of Task B.

[0020] Task A is completed manually by the user, and the execution process of Task A can be obtained by recording the operation steps of the user to complete Task A in the user interface. In some embodiments, the recording of the completion process of Task A can be achieved through multi-mode recording technology. For example, if Task A is executed in a web application, a specific script can be injected to achieve DOM event capture to record the interaction events of the user in the process of completing Task A. For another example, the user interface element recognition technology can be used to identify the UI elements that the user interacts with in the process of executing Task A. For another example, screen operation recording technology can be used to capture the interaction between the user and the application in the user interface in the process of completing Task A through screen elements. For another example, the application programming architecture (API) call capture technology can be used to record the background API call sequence and parameter information to achieve the recording of the execution steps of Task A. Figure 1 As shown, by recording, the operation steps of task A can be obtained. The operation steps 110 of task A include step 1, step 2, step 3, step 4, step 5, step 6 and step.

[0021] After obtaining the operation steps to complete Task A, the semantic information of the operation steps to complete Task A can be extracted. For example, element semantic analysis can be used to identify functional attributes in interactive elements, such as buttons, input boxes, and drop-down menus. Next, operation intention recognition can be used to infer the operation intention. For example, the user's operation intention in the process of completing Task A can be inferred based on the interaction mode and context. For example, based on steps 1 and 2, it can be inferred that step 3 is an operation such as query, filter, or submit. Next, data flow tracking can be used to record the complete path of input, conversion, and output of the data. Finally, context state capture can be used to capture the state changes of the system after completing Task A.

[0022] refer to Figure 1After obtaining the operation steps of task A, the operation steps (including the extracted voice information) can be converted into an operation sequence 120 of task A. The operation step sequence of task A can be shown as [step 1, step 2, step 3, step 4, step 5, step 6, step 7]. For example, the operation sequence 120 can include the temporal dependency between each operation step. For example, step 1 is before step 2, and step 2 is before step 3. For another example, the operation sequence 120 can include a standardized representation of each identified interactive element, for example, a button can be "button"; a text box can be "text". For another example, the operation sequence 120 can include the type of data parameters, which can be divided into variable parameters and immutable parameters. For another example, the label parameter of the button "button" is "next step", which is an immutable parameter. The "text" input parameter of the text box represents the quantity, which needs to be determined according to the user's input. For another example, the operation sequence 120 can also include conditional branches and loop structures. For example, step 1 is a conditional branch, and step 2 can only be reached if certain conditions are met.

[0023] In order to achieve the versatility and portability of executing other tasks such as Task B in the subsequent process, the specific operation steps can be converted into a model context protocol service (hereinafter referred to as a service). The model context protocol is an open protocol that aims to standardize the communication between large language models (LLMs) and external data sources and tools, and solve the standardization problem of the interaction between models and external resources. Given that the model context protocol service is functionally independent, it is necessary to define which steps in the operation steps of Task A are inseparable and which must be continuous, so as to realize the identification of the functional boundaries of the model context protocol service. In some embodiments, the functional boundaries of each model context protocol service can be determined based on the functional boundary identification technology of multi-dimensional semantic analysis.

[0024] For example, the functional boundary can be determined based on the semantic relevance score between each operation step in the operation sequence. For example, a graph can be constructed with each step in the operation sequence as a node and the semantic relevance score as an edge, and then a community area detection algorithm can be applied to the graph to identify high cohesion subgraphs, so that after verifying the functional integrity and independence of each subgraph, the high cohesion subgraphs can be converted into independent model context protocol services.

[0025] For example, the boundary definition of the model context protocol service can also be achieved with the help of the semantic information of each operation step in the operation sequence. For example, the data flow can be segmented according to the boundaries of input and output to achieve the definition of functional boundaries. For example, the functional boundaries of the model context protocol service can also be determined based on the significant change points of the system state change after the task is completed. Alternatively, specific domain knowledge can be combined to assist in the identification of service boundaries. For example, the steps of generating a report can be divided into data extraction, data conversion, data analysis, and report generation. According to the knowledge of data processing, data extraction and data conversion can be defined as an independent service, namely data extraction and conversion. At this time, the four steps can be defined as three independent functions of data extraction and conversion, data analysis, and report generation. like Figure 1 As shown, [Step 1, Step 2, Step 3] can be defined as an independent model context protocol service; [Step 4] can be defined as an independent model context protocol service; [Step 5, Step 6] can be defined as an independent model context protocol service; [Step 7] can also be defined as an independent model context protocol service.

[0026] In order to automate the execution of Task B in the subsequent process, the task parameters of Task B are necessary. Parameters are variables declared when defining a function, method, or procedure, and parameter values ​​are specific data assigned to these parameters when calling them. By passing in different parameter values, the same function, method, or procedure can process different data and thus achieve different functions. For the sake of convenience, in the following text, task parameters can be referred to as parameters, and parameter values ​​and parameters can be the same expression.

[0027] As for how to apply the task parameters of task B to the model context protocol service determined from the operation steps of task A, it is necessary to determine which operation steps in task A have variable parameter values ​​and which are immutable. Only in this way can the parameters of task B be applied to the model context protocol service determined from the completion process of task A, thereby realizing the automated execution of task B.

[0028] In some embodiments, adaptive parameter abstraction technology can be used to identify variable parts in operation steps. For example, candidate parameter points of task A can be identified. The parameters of candidate parameter points are variable. Candidate parameter points can be parameter types such as form input, query conditions, etc., and then the parameter definition of each parameter point is generated according to each candidate parameter point, that is, parameter description, type and constraint conditions. For example, a candidate parameter point is a time range parameter: type = date range, format = YYYY-MM-DD, description = time range covered by the report. For another example, a candidate parameter point is a data type parameter: type = enumeration, optional values ​​= [business, inventory, customer, finance], description = data type to be analyzed.

[0029] For example, the candidate parameter points of task A can be identified, and then the feature vector F of each candidate parameter point can be extracted. The parameterization probability P of each candidate parameter point can be determined with the help of the decision model, so that the parameter description, type and constraint conditions of the parameter point can be screened out based on the parameterization probability P.

[0030] Alternatively, for example, the pattern characteristics of the input value can also be analyzed. If the input value is in the form of a date format or a numerical range, then the input value is variable, thereby identifying the variable part in the operation step. For example, the correlation between certain parameter values ​​in task A and contextual interaction elements can also be identified, thereby analyzing whether the parameter value of the parameter is variable. For example, as mentioned above, the label parameter of the button "button" is "next step", which is an immutable parameter. For another example, if the user records the process of completing task A multiple times when manually completing task A, the variable and immutable parts of the operation steps can be identified by comparing the differences in the multiple records. For another example, domain entity recognition technology can also be used to determine whether the parameter value is an entity in a specific domain, such as to determine whether a parameter value is a name or name field.

[0031] After determining the functional boundaries of the model context protocol service of task A and the variable parameter parts therein, these model context protocol services need to be encapsulated into independent functional service entities. For example, the encapsulated model context protocol service also includes parameter validation rules, and validation rules can be generated based on the parameter definition of the variable parameters to achieve the accuracy of task execution. For example, numeric parameters are not allowed to be input with string parameter values. When the task parameters of task B are verified, task B can continue to be executed. For example, the encapsulated model context protocol service also includes executable code converted by the operation sequence.

[0032] For example, the encapsulated model context protocol service also includes standardized service interface definitions to enable communication with external data sources and tools. For example, the encapsulated model context protocol service also includes error handling logic to automatically add anomaly detection and perform corrective operations. For example, when missing data in a certain area is detected, it can be automatically adjusted to only analyze areas with complete data. For example, the encapsulated model context protocol service also includes service documentation to automatically generate service descriptions and instructions for use.

[0033] In some embodiments, the encapsulated model context protocol service includes executable code converted by the sequence of operations, a service identifier, parameter definitions, and validation rules.

[0034] After the analysis of the operation steps of task A is completed, in order to achieve the goal of automatically completing task B, it is first necessary to know what task B is, that is, it is necessary to know the user's intention. The user's intention can be determined based on the user's input 130. In some embodiments, the user input can be in text form or in voice form, but no matter what form it is, it is ultimately in natural language form.

[0035] like Figure 1 As shown, task B 140 and task parameters of task B can be determined according to user input 130. For example, a semantic parsing model such as a decision model can be used to extract the core part of the user input 130. The decision model is an intelligent model that can achieve cross-round semantic understanding, infer the user's implicit intention based on the context and user's historical preferences, and decompose the user's intention into executable sub-intention sequences.

[0036] The decision model can pre-process the user input 130 to obtain the core intent. For example, if the user input 130 is "generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", the decision model can analyze and identify that task B is to generate a business performance report, where the core action is: generate report; key parameters: time range = last quarter, data type = business, region = all, format = PDF, recipient = business team. The completion process of task A is the complete process of extracting data from multiple systems (ERP, CRM and data warehouse) and generating a comprehensive report. It is decomposed into 4 functionally independent model context protocol services: data extraction [step 1, step 2, step 3], data conversion [step 4], data analysis [step 5, step 6] and report generation [step 7].

[0037] In order to enable Task B to be automatically executed, the most suitable model context service 150 to be executed for Task B can be determined from the four functionally independent model context protocol services of Task A. In some embodiments, the semantic relevance between the intention expressed by the user input 130 and the four independent services of Task A can be calculated (using a decision model) to determine the most suitable model context service 150 to be executed for Task B. For example, the decision model plans that the set of model context protocol services to be executed for Task B includes: data extraction, data conversion, data analysis, and report generation.

[0038] Then, the execution path of the service call (which can also be called a service call sequence) is determined based on the dependency relationship between these to-be-executed model context services 150. In some embodiments, a call dependency graph with these services as nodes can be constructed, and then the execution path can be determined with the help of a path planning algorithm. For example, the execution path can be: data extraction → data conversion → data analysis → report generation.

[0039] Next, the task parameters of task B can be mapped to the to-be-executed model context service 150 to implement the automated execution of task B 160. For example, "last quarter" can be converted into a specific date range (such as "2023-04-01 to 2023-06-30"); "business team" can be parsed as "specific mailing list"; and necessary default parameters can be supplemented (such as report template = "standard business performance template"). In some embodiments, a parameter mapping table can be generated based on the task parameters of task B, and the mapping table includes the source of the parameters. For example, some parameters are obtained from the deaf user input 130, some parameters are inferred default values, and some parameters can be requested from the user again. It can be understood that the to-be-executed model context service 150 itself also needs to have service parameters, that is, fixed and immutable parameters.

[0040] Through the above description, task B can be automatically executed, and the execution result of task B can also be displayed on the user interface. According to the embodiment of the present disclosure, the user's operation sequence for task A can be reused to intelligently match and execute the user's task B, which greatly improves work efficiency and reduces the duration and workload of manual operations. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so as to cope with new tasks (such as task B) and new requirements. In addition, with the help of natural language interaction, the usage threshold is lowered, so that non-professionals can also use it, and the practicality and scope of application of task execution are comprehensively improved. That is, through this method, the user's simple operations can be automatically converted into model online protocol services that can be dynamically combined to realize the execution of tasks and improve the user experience.

[0041] The following will combine Figures 2 to 11 The method according to the embodiment of the present disclosure is described in detail. For ease of understanding, the specific data mentioned in the following description are exemplary and are not intended to limit the scope of protection of the present disclosure. It is understood that the embodiments described below may also include additional actions not shown and / or the actions shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0042] Figure 2 A flow chart of a method 200 for performing a task according to some embodiments of the present disclosure is shown. The method 200 may be performed by a device for performing a task, which may be, for example, an independent device or system. The device may be implemented in software and / or hardware. Next, the method 200 is schematically described by taking the device for performing a task as an execution subject as an example. The method 200 includes a frame 202, a frame 204, a frame 206, a frame 208, a frame 210, and a frame 212.

[0043] like Figure 2 As shown, in block 202, an operation sequence of a user on a user interface for a first task is obtained, where the operation sequence includes operation steps required to complete the first task. Figure 1 , Task A is completed manually by the user, and the operation steps of the user in the user interface to complete Task A can be recorded, so that the execution process of Task A can be obtained. After obtaining the operation steps of Task A, the operation steps can be converted into an operation sequence 120 of Task A, and the operation sequence 120 includes the operation steps of Task A. The operation sequence 120 of Task A is shown as [Step 1, Step 2, Step 3, Step 4, Step 5, Step 6, Step 7].

[0044] In block 204, based on the operation sequence, the operation steps are converted into model context protocol services. Referring to 1, in order to achieve universality and portability in executing other tasks such as task B in the subsequent process, the specific operation steps of task A can be converted into model context protocol services. Figure 1 As shown, [Step 1, Step 2, Step 3] can be defined as an independent model context protocol service 1; [Step 4] can be defined as an independent model context protocol service 2; [Step 5, Step 6] can be defined as an independent model context protocol service 3; and [Step 7] can also be defined as an independent model context protocol service 4.

[0045] In block 206, a user input is obtained, where the user input indicates the second task, and the user input includes at least one of a text input or a voice input. Figure 1After the analysis of the operation steps of task A is completed, in order to achieve the goal of automatically completing task B, it is first necessary to know what task B is, that is, it is necessary to know the user's intention. The user's intention can be determined based on the user's input 130. In some embodiments, the user input can be in text form or in voice form, but no matter what form it is, it is ultimately in natural language form. For example, if the user input 130 is "generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", then task B can be "generate a report".

[0046] At block 208, based on the second task, a model context protocol service to be executed is determined, the model context protocol service to be executed being associated with the second task. Figure 1 In order to enable Task B to be automatically executed, the most suitable model context service 150 to be executed for Task B can be determined from the four functionally independent model context protocol services of Task A. For example, the model context service 150 to be executed includes: Model context protocol service 1 [Step 1, Step 2, Step 3], Model context protocol service 2 [Step 4], Model context protocol service 3 [Step 5, Step 6] and Model context protocol service 4 [Step 7].

[0047] At block 210, the second task is executed based on the task parameters of the second task and the model context protocol service to be executed, the second task being composed of the model context protocol service to be executed associated with the second task, the task parameters of the second task being determined based on the user input. Figure 1 , the task parameters of task B can be mapped to the to-be-executed model context service 150 to implement the automated execution of task B 160 .

[0048] In block 212, the execution result of the second task is displayed on the user interface. Through the above blocks 202 to 210, task B can be automatically executed, and the execution result of task B can also be displayed on the user interface.

[0049] According to the embodiments of the present disclosure, the user's operation sequence for task A can be reused to intelligently match and execute the user's task B, which greatly improves work efficiency and reduces the duration and workload of manual operations. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so that new tasks (such as task B) and new requirements can be responded to. In addition, with the help of natural language interaction, the usage threshold is lowered, allowing non-professionals to use it, and the practicality and scope of application of task execution are comprehensively improved. That is, through this method, the user's simple operations can be automatically converted into model online protocol services that can be dynamically combined to achieve task execution and improve user experience.

[0050] For example, Figure 3 A schematic diagram of an example system architecture 300 for performing tasks according to some embodiments of the present disclosure is shown. The example system architecture 300 includes an operation recording module 310, a model context protocol service generation module 320, a service storage and indexing module 330, a large model decision module 340, a tool execution and monitoring module 350, and a user interaction module 360.

[0051] For ease of explanation, let's continue to use Task A and Task B as examples, where Task A is a task that users complete manually and Task B is a task that needs to be automatically executed. Assume that Task A is to extract data from multiple systems (ERP, CRM, and data warehouse) and generate a comprehensive report; the user expresses the intention through natural language as "generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format."

[0052] In some embodiments, the operation recording module 310 is used to capture the user operation sequence and extract structured semantic information. Figures 4A-4D For example, Figure 4A FIG. 4 is a schematic diagram showing an example function 400A of an operation recording module according to some embodiments of the present disclosure. Figure 4A As shown, the function of operating the recording module 310 includes initializing the recording environment 311 .

[0053] For the sake of explanation, Figure 4B , illustratively, Figure 4B Schematic diagram of an example function 400B of initializing a recording environment according to some embodiments of the present disclosure is shown. In some embodiments, initializing the recording environment 311 refers to initializing the system environment before the user performs task A and before starting to record the operation steps of the task. Figure 4B As shown, initializing the recording environment may include loading a recording engine 3111, where the recording engine is used to record all operation steps of the user in completing Task A.

[0054] refer to Figure 4B , initializing the recording environment 311 may include injecting an event listener 3112. Injecting an event listener refers to adding an event processing function or object to a specific event source. When the event source triggers a specific event, the event listener associated with it will be called to execute the corresponding processing logic. The event source can be an HTML element in a web page, a window of an operating system, a control of an application, etc.; events include mouse clicks, keyboard keys, timer timeouts, etc., so that the subsequent recording and recording of the user's execution of task A can be realized.

[0055] refer to Figure 4BInitializing the recording environment 311 may include initializing the data buffer 3113. In order to ensure the accuracy of recording and recording, a specific event buffer and filtering mechanism may be established during the recording process to avoid interference from irrelevant events.

[0056] Back to Figure 4A , the function of the operation recording module 310 includes capturing user operations 312. For ease of explanation, Figure 4C , illustratively, Figure 4C FIG. 4 is a schematic diagram showing an example function 400C for capturing user operations according to some embodiments of the present disclosure. Figure 4C , capturing user operations 312 includes monitoring DOM events 3121. DOM is a tree structure for representing HTML or XML documents, in which each element (such as 、 <button>), attribute and text nodes can all be considered as a node in the tree. Listening to DOM events is to set an "observer" on a node in the DOM tree. When a specific event (such as user click, mouse movement, keyboard key, etc.) occurs on the node, the code associated with it will be triggered and executed. In some embodiments, a specific script can be injected to capture user interaction events in the process of executing task A in the web application.

[0057] refer to Figure 4C , capturing user operations 312 includes capturing UI interaction events 3122. UI interaction events are a series of actions triggered when a user operates with the application interface, such as clicking a button, sliding the screen, entering text, etc. Capturing these events is to enable the application to perceive these user operations through programming, and execute corresponding processing logic according to different events, thereby realizing interaction with the user. In some embodiments, user interface element recognition technology can be used to identify the UI elements that the user interacts with during the execution of task A. Alternatively, a multi-level selector can also be generated.

[0058] refer to Figure 4C , capturing user operations 312 includes recording timestamps and contexts 3123. Recording timestamps and contexts means that when an event occurs (for example, a user triggers form submission during the execution of task A), the time information of the event and the related environment and state information are obtained and saved. For example, if the execution steps of task A include step 1, step 2, and step 3, the execution time of step 1, step 2, and step 3 and the related steps will be captured.

[0059] By capturing user operations 312, user interaction events in the web page can be captured, user interface elements with which the user interacts can be identified in real time, user operations on user interface applications can be identified, and parameter information called by the background application interface and corresponding context information can be recorded.

[0060] Back to Figure 4A The function of the operation recording module 310 includes operation sequence preprocessing 313. For ease of explanation, Figure 4D , illustratively, Figure 4D FIG. 4 is a schematic diagram showing an example function 400D of operating sequence preprocessing according to some embodiments of the present disclosure. Figure 4D As shown, the operation sequence preprocessing 313 may include filtering irrelevant operations 3131. For example, in the process of capturing the user performing task A, the user will inevitably slide the mouse meaninglessly, and such meaningless mouse sliding operations will be filtered out.

[0061] like Figure 4D As shown, operation sequence preprocessing 313 may include merging related consecutive operations 3132. Figure 1 , in the process of executing task A, step 1, step 2 and step 3 are continuous, then step 1, step 2 and step 3 can be pre-merged together.

[0062] like Figure 4D As shown, the operation sequence preprocessing 313 may include a specification description 3133. As mentioned above, in order to subsequently generate a universal model context protocol service, a standardized service interface needs to be defined. Therefore, when preprocessing the operation sequence, some structural descriptions in the operation sequence may be standardized, such as a button as a button and a text box as a text.

[0063] With the help of Figures 4A-4D The exemplary operation recording module described can accurately capture the operation steps of the user in performing task A, and provide a basic template for the automated execution of subsequent task B.

[0064] return Figure 3 In some embodiments, the model context protocol service generation module 320 is used to convert the operation sequence into a standard model context protocol service. Figures 5A-5D For example, Figure 5A FIG. 5 is a schematic diagram showing an example function 500A generated by a model context protocol service according to some embodiments of the present disclosure. Figure 5A , the model context protocol service generation module 320 includes semantic analysis and segmentation 321. For ease of explanation, Figure 5B , illustratively, Figure 5B FIG. 5 is a schematic diagram showing an example function 500B of semantic analysis and segmentation according to some embodiments of the present disclosure. Figure 5B , semantic analysis and segmentation 321 includes analyzing the semantic structure 3211 of the operation sequence. After obtaining the operation steps to complete task A, the semantic information of the operation steps to complete task A can be extracted. For example, element semantic analysis can be used to identify functional attributes in interactive elements, such as buttons, input boxes, and drop-down menus. Then, operation intention recognition can be used to infer the operation intention. For example, the user's operation intention in the process of completing task A can be inferred based on the interaction mode and context. For example, based on steps 1 and 2, it can be inferred that step 3 is an operation such as query, filter, or submit. Then, data flow tracking can be used to record the complete path of input, conversion, and output of the additional data. Finally, context state capture can be used to capture the state changes of the system after completing task A.

[0065] refer to Figure 5B , semantic analysis and segmentation 321 includes identifying functionally complete operation subsequences 3212. Since the operation subsequences are functionally independent, it is necessary to define which steps in the operation steps of task A are inseparable and which must be continuous, so as to achieve the identification of the functionally complete boundaries of the operation subsequence. Figure 1 As shown, [step 1, step 2, step 3] can be a functionally complete subsequence; [step 4] can be a functionally complete subsequence; [step 5, step 6] can be a functionally complete subsequence; and [step 7] can also be a functionally complete subsequence.

[0066] refer to Figure 5B , semantic analysis and segmentation 321 includes determining the boundary 3213 of the model context protocol service. In order to achieve the versatility and portability of executing other tasks such as task B in the subsequent process, the specific operation subsequence can be converted into a model context protocol service. In view of the fact that the model context protocol service is functionally independent, it is necessary to realize the identification of the functional boundaries of the model context protocol service. This is consistent with the above-mentioned principle of realizing the boundary identification of the operation subsequence. In some embodiments, the functional boundary of each model context protocol service can be determined based on the functional boundary identification technology of multi-dimensional semantic analysis.

[0067] For example, the functional boundary can be determined based on the semantic relevance score between each operation step in the operation sequence. For example, a graph can be constructed with each step in the operation sequence as a node and the semantic relevance score as an edge, and then the design area detection algorithm can be applied to the graph to identify high cohesion subgraphs, so that after verifying the functional integrity and independence of each subgraph, the high cohesion subgraphs can be converted into independent model context protocol services.

[0068] For example, the boundary of the model context protocol service can be defined by using the semantic information of each operation step in the operation sequence. For example, the data flow can be segmented according to the input and output boundaries to define the functional boundary. For example, the functional boundary of the model context protocol service can be determined according to the significant change points of the system state after completing the task.

[0069] For example, the operation sequence of task A can be converted into four functionally independent services: data extraction service, data conversion service, data analysis service, and report generation service.

[0070] Alternatively, specific domain knowledge can be combined to assist in identifying service boundaries. For example, the steps of generating a report can be divided into data extraction, data conversion, data analysis, and report generation. Based on the knowledge of data processing, data extraction and data conversion can be defined as an independent service, data extraction and conversion. In this case, the four steps can be defined as three independent functions: data extraction and conversion, data analysis, and report generation. Figure 1 As shown, there is a service boundary between [Step 1, Step 2, Step 3] and [Step 4]; there is a service boundary between [Step 4] and [Step 5, Step 6]; and there is also a service boundary between [Step 5, Step 6] and [Step 7].

[0071] Back to Figure 5A , the model context protocol service generation module 320 includes parameter extraction and abstraction 322. In order to apply the task parameters of task B to the model context protocol service related to task A, it is also necessary to extract and abstract the parameters involved in task A. Figure 5C , illustratively, Figure 5C FIG. 5 is a schematic diagram showing an example function 500C of parameter extraction and abstraction according to some embodiments of the present disclosure. Figure 5C , parameter extraction and abstraction 322 includes identifying the variable part in the operation 3221. In order to automatically execute task B in the subsequent process, the task parameters of task B are necessarily required. And how the task parameters of task B are applied to the model context protocol service determined from the operation steps of task A, it is necessary to determine which operation steps in task A have variable parameter values ​​and which are immutable. Only in this way can the parameters of task B be applied to the model context protocol service determined from the completion process of task A, thereby realizing the automated execution of task B.

[0072] refer to Figure 5C , parameter extraction and abstraction 322 includes determining parameter types and constraints 3222. For example, candidate parameter points of task A can be identified, and candidate parameter points are variable. Candidate parameter points can be parameter types such as form input, query conditions, etc., and then the parameter definition of each parameter point is generated based on each candidate parameter point, that is, parameter description, type and constraint conditions. Similarly, the parameter definition of immutable parameter points is also based on being determined. For example, a candidate parameter point is a time range parameter: type = date range, format = YYYY-MM-DD, description = time range.

[0073] refer to Figure 5C , Parameter extraction and abstraction 322 includes generating parameter descriptions and sample values ​​3223. For example, a candidate parameter point is a data type parameter: type = enumeration, optional value = [business, inventory, customer, financial], description = data type to be analyzed. Similarly, parameter descriptions and sample values ​​of immutable parameter points can also be generated. For example, a label type parameter, type = button, format: none, description = next step.

[0074] For example, through parameter abstraction technology, identifying key parameters in task A and generating parameter definitions can be as follows: Time range parameters: Type = Date range, Format = YYYY-MM-DD, Description = Time range covered by the report; Data type parameter: type = enumeration, optional values ​​= [business, inventory, customer, financial], description = the type of data to be analyzed; Report format parameters: type = enumeration, optional values ​​= [Excel, PDF, Slide], description = output report format; Recipients Parameters: Type = String Array, Format = Email, Description = List of report recipients.

[0075] Back to Figure 5A The model context protocol service generation module 320 includes a model context protocol service encapsulation 323. After determining the boundaries of the model context protocol service of task A, these model context protocol services can be encapsulated into functionally independent service entities. Figure 5D , illustratively, Figure 5D FIG. 5 is a schematic diagram showing an example functionality 500D of a model context protocol service package according to some embodiments of the present disclosure. Figure 5D The model context protocol service encapsulation 323 includes generating a service interface definition 3231. For example, the encapsulated model context protocol service also includes a standardized service interface definition to achieve communication with external data sources and tools.

[0076] refer to Figure 5D , the model context protocol service encapsulation 323 includes converting the operation sequence into execution code 3232. As mentioned above, after obtaining the functionally independent operation subsequences, these operation subsequences can be converted into executable codes, so as to realize the subsequent execution of task B.

[0077] refer to Figure 5D , the model context protocol service encapsulation 323 includes adding validation and error handling logic 3233. In the encapsulated model context protocol service, parameter validation rules are also included. Validation rules can be generated according to the parameter definition of variable parameters to achieve the accuracy of task execution. For example, if a task parameter of task B is numeric, then the numeric parameter is not allowed to be passed into a string. In the encapsulated model context protocol service, error handling logic can also be encapsulated. For example, when missing data in a certain area is detected, it can be automatically adjusted to only analyze areas with complete data.

[0078] refer to Figure 5D The model context protocol service encapsulation 323 includes generating a service document 3234. In some embodiments, the service document includes a description of the model context protocol service and instructions for use, so as to facilitate subsequent repair support by technical personnel when errors occur.

[0079] For example, data extraction services, data conversion services, data analysis services, and report generation services all encapsulate standardized interfaces, execution codes corresponding to each service, parameter validation logic, error handling mechanisms, and service description documents.

[0080] With the help of Figures 5A-5D The exemplary model context protocol generation module described can convert specific operation steps into standard services, so that these services can be used in different systems or environments, improving versatility and portability. At the same time, with the help of the model context protocol generation module, the complexity of the operation is also reduced, which facilitates subsequent management and maintenance.

[0081] return Figure 3 In some embodiments, the service storage and indexing module 330 is used to store model context protocol services. Figure 6 For example, Figure 6 FIG. 6 is a schematic diagram showing an example function 600 of a service storage and indexing module according to some embodiments of the present disclosure. Figure 6 As shown, the function of the service storage and indexing module 330 includes allocating an operation service unique identifier 3301. After converting the operation steps of task A into functionally independent model context protocol services, it is necessary to allocate a unique identifier to each service so that it can be accurately called when performing other tasks later. Alternatively, a version number can also be allocated to each service so that in the subsequent process, the user can call the required different versions of the service.

[0082] like Figure 6 As shown, the function of the service storage and index module 330 includes adding service metadata and tags 3302. After obtaining the operation steps of task A, the name, description and classification tag of each parameter in the operation sequence of task A can be stored in the service storage and index module, thereby ensuring the accuracy and validity of the data.

[0083] like Figure 6 As shown, the function of the service storage and indexing module 330 includes storing services in the service library 3303. After converting the operation steps of task A into functionally independent model context protocol services, it is also necessary to store these functionally independent model context protocol services, such as data extraction services, data conversion services, data analysis services, and report generation services, in the service storage and indexing module to achieve subsequent accurate calls.

[0084] like Figure 6 As shown, the function of the service storage and index module 330 includes establishing a service index 3304. After the operation steps of task A are converted into functionally independent model context protocol services, the model context protocol services have a sequential dependency relationship. Therefore, a service index can be established for these functionally independent model context protocol services to provide support for the subsequent generation of execution paths. For example, data extraction service → data conversion service → data analysis service → report generation service.

[0085] return Figure 3 In some embodiments, the large model decision module 340 is used to make tool selection and execution decisions based on user intent. Figures 7A-7D For example, Fig. 7A FIG. 7 is a schematic diagram showing an example function 700A of a large model decision module according to some embodiments of the present disclosure. Fig. 7A As shown, the large model decision module 340 includes intention understanding and decomposition 341. For ease of explanation, combined Figure 7B , Figure 7B A schematic diagram of an example functionality 700B is shown for purposes of understanding and decomposition according to some embodiments of the present disclosure.

[0086] refer to Figure 7B , intent understanding and decomposition 341 includes extracting core intent and parameters 3411. After the analysis of the operating steps of task A is completed, in order to achieve the goal of automating the completion of task B, it is first necessary to know what task B is, that is, it is necessary to know the user's intent. The user's intent can be determined based on the user's input. In some embodiments, the user input can be in text form or in voice form, but no matter what form it is, it is ultimately in natural language form. For example, the user's core intent and parameters can be determined based on the user input, that is, task B and the task parameters of task B can be determined. For example, a semantic parsing model such as a decision model can be used to extract the core part of the user input.

[0087] For example, if the user input is "Generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", the decision model can analyze and identify that Task B is to generate a business performance report, where the core action is: generate report; key parameters: time range = last quarter, data type = business, region = all, format = PDF, recipient = business team.

[0088] refer to Figure 7B , intention understanding and decomposition 341 includes understanding context in combination with conversation history 3412. In some embodiments, a semantic parsing model such as a decision model can also be used to understand the user's true intention in combination with the user's conversation history, that is, context.

[0089] refer to Figure 7B , intention understanding and decomposition 341 includes decomposing complex intentions into subtasks 3413. If the user's input description of task B is too complex, or task B is too complex, task B can be decomposed into multiple tasks, such as task B1, task B2, and task B3.

[0090] Back to Fig. 7A , the large model decision module 340 includes service retrieval and screening 342. For ease of description, combined Figure 7C , Figure 7C FIG. 7 is a schematic diagram showing an example function 700C of service retrieval and screening according to some embodiments of the present disclosure. Figure 7C As shown, service retrieval and screening 342 includes intent-based retrieval services 3421. In order to enable task B to be automatically executed, the functionally independent model context protocol services of task A can be retrieved to determine the most suitable model context service to be executed for task B.

[0091] like Figure 7C As shown, service retrieval and screening 342 includes calculating the matching degree between the service and the intent 3422 and screening out the most suitable service candidate set 3423. In order to enable task B to be executed automatically, the functionally independent model context protocol service of task A can be retrieved to determine the model context service to be executed that is most suitable for task B. In some embodiments, the semantic relevance between the intent expressed by the user input and the independent service of task A can be calculated, and the decision model can be used to determine the model context service to be executed that is most suitable for task B. For example, the decision model plans out the model context protocol service set to be executed for task B, including: data extraction service, data conversion service, data analysis service and report generation service, which is the most suitable substitute for the model context service for task B.

[0092] Back to Fig. 7A , the large model decision module 340 includes an execution path planning 343. For ease of description, combined Fig.7D , Fig.7D FIG. 7 is a schematic diagram showing an example function 700D for performing path planning according to some embodiments of the present disclosure. Fig.7D As shown, execution path planning 343 includes determining the service call sequence 3431, planning the data flow path 3432, and processing the dependencies between services 3433.

[0093] After the decision model plans the set of model context protocol services to be executed for task B, the execution path of the service call (which can also be called a service call sequence) and the planned path of the data flow can be determined based on the dependencies between these model context services 150 to be executed (the dependencies are determined with the help of service indexes). In some embodiments, a call dependency graph with these services as nodes can be constructed, and then the execution path can be determined with the help of a path planning algorithm. For example, the execution path and the data flow path can be: data extraction → data conversion → data analysis → report generation.

[0094] With the help of Figures 7A-7D , which can select the optimal execution path for task B, ensure the correct execution of task B, and realize intelligent task allocation.

[0095] return Figure 3 In some embodiments, the tool execution and monitoring module 350 is used to execute the model context protocol service and monitor the execution status. Figures 8A-8C For example, Fig. 8A FIG. 8 is a schematic diagram showing an example function 800A of a tool execution and monitoring module according to some embodiments of the present disclosure. Fig. 8A As shown, the tool execution and monitoring module 350 includes parameter preparation and verification 351. Figure 8B , Figure 8B FIG. 8 is a schematic diagram showing an example function 800B of parameter preparation and verification according to some embodiments of the present disclosure. Figure 8B As shown, parameter preparation and verification 351 includes extracting parameter values ​​from user intent 3511. For example, if the user input is "generate a business performance comparison report for all regions in the last quarter", the decision model can analyze and identify the key parameters of task B: time range = last quarter, data type = business, region = all, format = PDF, and recipient = business team.

[0096] like Figure 8B As shown, parameter preparation and verification 351 includes completing missing parameters 3512. For example, for the user input "generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format", the generated key parameters do not have report template parameters. In order to unify the data format, necessary default parameters can be added. For example, report template = standard business performance template.

[0097] like Figure 8B As shown, parameter preparation and verification 351 includes verifying the validity of parameters 3513. The validity of parameters can be verified with the help of parameter verification rules. In some embodiments, verification rules can be generated based on the parameter definition of variable parameters (i.e., parameter description, type and constraints) to achieve accuracy in task execution. For example, numeric parameters are not allowed to be entered with string parameter values. For another example, a parameter is a time range parameter: type = date range, format = YYYY-MM-DD, description = time range covered by the report. Then, the corresponding format cannot be filled with a long string of characters. If the task parameter verification of task B is successful, then task B can continue to be executed.

[0098] Back to Fig. 8A , the tool execution and monitoring module 350 includes a service execution and monitoring 352. For ease of description, in conjunction with Figure 8C , Figure 8C FIG. 8 is a schematic diagram showing an example function 800C of service execution and monitoring according to some embodiments of the present disclosure. Figure 8C As shown, service execution and monitoring 352 includes calling services 3521 in a planned order. As mentioned above, the large model can plan the execution path for task B. Therefore, during the execution of task B, the related services need to be called strictly in the planned order, for example, strictly in the order of data extraction service → data conversion service → data analysis service → report generation service.

[0099] like Figure 8C As shown, service execution and monitoring 352 includes monitoring execution status and intermediate results 3522 and handling abnormal situations 3523. In the process of executing task B, it is also necessary to monitor the execution status of task B in real time. If the data in a certain area is missing or abnormal, it can be automatically adjusted to only analyze the area with complete data, or add data verification steps for the abnormal data area. In some embodiments, during the execution of task B, key monitoring points can also be pre-determined, and user confirmation can be further obtained at the key monitoring points to ensure the smooth execution of task B. Alternatively, the most suitable visualization chart type can be dynamically selected according to the characteristics of the data.

[0100] With the help of Figures 8A-8C The tool execution and monitoring module 350 shown can monitor the execution process of task B in real time and respond promptly when an exception occurs, thereby ensuring the smooth execution of task B. In some embodiments, when a certain model to be executed online protocol service of task B fails, a model to be executed context protocol service can be re-determined to ensure the smooth execution of task B.

[0101] return Figure 3 In some embodiments, the user interaction module 360 ​​is used to receive the user's intention and display the execution result. Figures 9A-9C For example, Fig.9A FIG. 9 is a schematic diagram showing an example function 900A of a user interaction module according to some embodiments of the present disclosure. Fig.9A As shown, the user interaction module 360 ​​includes receiving user instructions 361. For ease of explanation, Fig. 9B , Fig. 9B FIG. 9 is a schematic diagram showing an example function 900B for receiving user instructions according to some embodiments of the present disclosure. Fig. 9B As shown, receiving user instructions includes receiving user input 3611 and pre-processing input 3612. For example, when the user inputs "generate a business performance comparison report for all regions in the last quarter and send it to the business team in PDF format" in the user interface, the user input can be received. If the user inputs in the form of voice, the user input can also be pre-processed, and the voice can be converted into a natural language. If the user input is too complicated, the user input can also be processed in segments. In some embodiments, if during the execution of task B, it is found that some data needs to be confirmed with the user, the user input can be received again. In some embodiments, during the execution of task B, key monitoring points can also be pre-determined, and the user's confirmation can be further obtained at the key monitoring points to ensure the smooth execution of task B.

[0102] like Fig.9A As shown, the user interaction module 360 ​​includes result processing and feedback 362. Fig. 9C , Fig. 9C FIG. 9 is a schematic diagram showing an example function 900C of result processing and feedback according to some embodiments of the present disclosure. Fig. 9C As shown, result processing and feedback 362 includes integrated execution results 3621. When task B is completed, the execution results of task B can be integrated. For example, an executive summary can be generated, such as "6 regional business performance analysis has been completed, and a 15-page PDF report has been generated."

[0103] like Fig. 9C As shown, result processing and feedback 362 includes generating user-friendly feedback 3622. For example, as another example, the highlighted text "Eastern District business grew fastest, with an increase of 23%" can also be displayed on the user interface. Result processing and feedback 362 also includes updating execution history 3623 and collecting user satisfaction feedback 3624. In some embodiments, when task B is completed, the execution history can be updated. In some embodiments, it is also possible to interact with the user through a language prompt "Is this report helpful to you?" to collect user feedback. After collecting user feedback, a score for the execution of task B can be generated by combining indicators of multiple dimensions such as accuracy, efficiency, and stability of task execution, so that the decision model can be adjusted based on user feedback and the score.

[0104] In some embodiments, based on the user's task execution history, if the user often executes task C immediately after executing task B, task C can be automatically executed for the user. For example, if the user often generates an inventory report immediately after generating a business report, the inventory report can be generated immediately for the user.

[0105] It is understandable that the various modules in the exemplary system architecture 300 can be interconnected through standardized interfaces to form a complete processing flow.

[0106] According to the embodiments of the present disclosure, the user's operation sequence for the first task can be reused to intelligently match and execute the user's second task, which greatly improves work efficiency and reduces the duration and workload of manual operations. At the same time, this method of converting operation steps into model context protocol services can also effectively improve the scalability and integration of task execution, so as to cope with new tasks and new requirements. In addition, with the help of natural language interaction, the usage threshold is lowered, allowing non-professionals to use it, and the practicality and scope of application of task execution are comprehensively improved. That is, through this method, the user's simple operations can be automatically converted into model online protocol services that can be dynamically combined to achieve task execution and improve user experience.

[0107] Fig.10 1 shows a block diagram of an apparatus 1000 for performing a task according to some embodiments of the present disclosure. Fig.10 As shown, the device 1000 includes an operation sequence acquisition module 1002, which is configured to acquire the operation sequence of the user for the first task on the user interface, and the operation sequence includes the operation steps required to complete the first task. The device 1000 includes an operation step conversion module 1004, which is configured to convert the operation steps into a model context protocol service based on the operation sequence. The device 1000 includes a user input acquisition module 1006, which is configured to acquire user input, the user input indicates the second task, and the user input includes at least one of text input or voice input. The device 1000 includes a to-be-executed model context protocol service determination module 1008, which is configured to determine the to-be-executed model context protocol service based on the second task, and the to-be-executed model context protocol service is associated with the second task. The device 1000 includes a second task execution module 1010, which is configured to execute the second task based on the task parameters of the second task and the to-be-executed model context protocol service, the second task is composed of the to-be-executed model context protocol service associated with the second task, and the task parameters of the second task are determined based on the user input. The apparatus 1000 includes an execution result display module 1012 configured to display the execution result of the second task on the user interface.

[0108] In some embodiments, the operation step conversion module 1004 includes: a first determination module configured to determine the semantic relevance score between the operation steps; and a first conversion module configured to convert the operation steps into a model context protocol service based on the semantic relevance score.

[0109] In some embodiments, the conversion module includes: a first generation module, configured to generate a graph with operation steps as nodes and semantic relevance scores as edges; a second determination module, configured to determine a high cohesion subgraph based on the graph, wherein the high cohesion subgraph indicates a model context protocol service; and a second conversion module, configured to convert the operation steps into a model context protocol service in response to the high cohesion subgraph being independent.

[0110] In some embodiments, the operation sequence includes a data flow associated with the first task, wherein the conversion module includes: a third determination module configured to determine the boundaries of the input and output of the data flow. The third conversion module is configured to convert the operation steps into a model context protocol service based on the boundaries.

[0111] In some embodiments, the conversion module includes: a fourth determination module configured to determine a change point of a system state associated with the first task after the first task is completed; and a fifth determination module configured to convert the operation steps into a model context protocol service based on the change point.

[0112] In some embodiments, the operation sequence includes task parameters of a first task, and also includes: a sixth determination module, configured to determine candidate parameter points in the task parameters of the first task, the candidate parameter points indicating that the task parameters at the candidate parameter points are variable; a second generation module, configured to generate parameter definitions based on the candidate parameter points, the parameter definitions including parameter descriptions, parameter types, and constraints.

[0113] In some embodiments, it also includes: a third generation module configured to generate verification rules based on parameter definitions, and the verification rules are used to verify the model context protocol service.

[0114] In some embodiments, a model context protocol service includes a service identifier, parameter definitions, and validation rules.

[0115] In some embodiments, the second task execution module 1010 includes: a verification module, configured to verify the task parameters of the second task based on a verification rule, wherein the verification rule is based on the first task; an execution module, configured to execute the model context protocol service to be executed associated with the second task based on the task parameters of the second task and the model context protocol service to be executed in response to successful verification of the task parameters of the second task.

[0116] In some embodiments, the system further includes: a seventh determination module configured to determine the second task and task parameters of the second task based on user input by a decision model.

[0117] In some embodiments, the module 1008 for determining the model context protocol service to be executed includes: an eighth determination module, configured to determine the semantic relevance between the second task and the model context protocol service; and a ninth determination module, configured to determine the model context protocol service to be executed from the model context protocol services based on the semantic relevance by the decision model.

[0118] In some embodiments, the ninth determination module includes: a tenth determination module, configured to determine, based on semantic relevance, a set of model context protocol services to be executed that are associated with the second task from the model context protocol services; a fourth generation module, configured to generate a graph with the model context protocol services to be executed in the set of model context protocol services to be executed as nodes; and a tenth determination module, configured to determine an execution path based on the graph, the execution path indicating dependencies between the model context protocol services to be executed, and the model context protocol services to be executed are associated with the second task.

[0119] In some embodiments, the task parameters of the second task include a first task parameter and a second task parameter, the first task parameter is determined based on user input, and the second task parameter is a default value.

[0120] In some embodiments, it also includes: a monitoring module configured to monitor the execution process of the second task; and a redetermining module configured to redetermine the model context protocol service to be executed in response to an error in the execution process.

[0121] In some embodiments, it also includes: an acquisition module, configured to obtain user feedback on the completion of the second task after the execution of the second task is completed; and an eleventh determination module, configured to determine the score for the completion of the second task; and an adjustment module, configured to adjust the decision model based on the feedback and the score.

[0122] In some embodiments, it also includes: a twelfth determination module, configured to determine the key monitoring points in the execution process of the second task; a third task parameter acquisition module, configured to obtain the third task parameters of the second task input by the user in response to the key monitoring points in the execution process of the second task.

[0123] In some embodiments, the operation sequence acquisition module 1002 includes: an initialization module, configured to initialize the system state associated with the first task; a recording module, configured to record the user's interactive operations on the user interface to complete the first task; and a fourth conversion module, configured to convert the interactive operations into an operation sequence.

[0124] In some embodiments, the recording module includes: a first recording module, configured to record the functional attributes of the interactive elements corresponding to the interactive operation; record the path of data flow input, conversion and output, the data flow is associated with the first task; and a first recording module, configured to record the change point of the system state associated with the first task after the first task is completed.

[0125] In some embodiments, it also includes: a thirteenth determination module, configured to determine a third task and task parameters of the third task in response to the completion of the second task; and a third task execution module, configured to execute the third task on the user interface, the third task being associated with the second task.

[0126] Fig.11 1 shows a block diagram of a device 1100 capable of implementing various embodiments of the present disclosure. Fig.11 As shown, the device 1100 includes a central processing unit (CPU) and / or a graphics processing unit (GPU) 1101, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 1102 or loaded from a storage unit 1108 to a random access memory (RAM) 1103. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. The CPU / GPU 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104. Although not shown in FIG. Fig.11 As shown in FIG. 1 , device 1100 may further include a coprocessor.

[0127] A number of components in the device 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The various methods or processes described above may be executed by the CPU / GPU 1101. For example, in some embodiments, the methods may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the CPU / GPU 1101, one or more steps or actions in the methods or processes described above may be performed.

[0129] In some embodiments, the methods and processes described above may be implemented as a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0130] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0131] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0132] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, and conventional procedural programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a separate software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby implementing various aspects of the present disclosure.

[0133] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0134] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0135] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the equipment, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the frame can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous frames can actually be executed substantially in parallel, and they can also be executed in the opposite order sometimes, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart, and the combination of frames in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0136] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.< / button>

Claims

1. A method for performing a task, comprising: Acquire an operation sequence of a user for a first task on a user interface, where the operation sequence includes operation steps required to complete the first task; Based on the operation sequence, converting the operation steps into model context protocol services; Obtaining user input, where the user input indicates a second task, and the user input includes at least one of text input or voice input; Based on the second task, determining a model context protocol service to be executed, wherein the model context protocol service to be executed is associated with the second task; executing the second task based on task parameters of the second task and the to-be-executed model context protocol service, the second task consisting of the to-be-executed model context protocol service associated with the second task, the task parameters of the second task being determined based on the user input; as well as The execution result of the second task is displayed on the user interface.

2. The method according to claim 1, wherein based on the operation sequence, converting the operation steps into a model context protocol service comprises: Determining semantic relevance scores between the operation steps; as well as Based on the semantic relevance score, the operation steps are converted into the model context protocol service.

3. The method according to claim 2, wherein converting the operation steps into the model context protocol service based on the semantic relevance score comprises: Generate a graph with the operation steps as nodes and the semantic relevance scores as edges; Based on the graph, determining a high cohesion subgraph, the high cohesion subgraph indicating the model context protocol service; as well as In response to the high cohesion subgraph being independent, converting the operation steps into the model context protocol service.

4. The method according to claim 1, wherein the operation sequence comprises a data flow associated with the first task, wherein based on the operation sequence, converting the operation steps into a model context protocol service comprises: Determining the input and output boundaries of the data stream; as well as Based on the boundary, the operation steps are converted into the model context protocol services.

5. The method according to claim 1, wherein based on the operation sequence, converting the operation steps into a model context protocol service comprises: After the first task is completed, determining a change point of a system state associated with the first task; as well as Based on the change points, the operation steps are converted into the model context protocol services.

6. The method according to any one of claims 2 to 5, wherein the operation sequence includes task parameters of the first task, and further includes: Determining a candidate parameter point in the task parameters of the first task, the candidate parameter point indicating that the task parameter at the candidate parameter point is variable; as well as Based on the candidate parameter points, a parameter definition is generated, wherein the parameter definition includes the parameter description, parameter type, and constraint conditions.

7. The method according to claim 6, further comprising: Based on the parameter definition, a validation rule is generated, where the validation rule is used to validate the model context protocol service.

8. The method of claim 7, the model context protocol service comprising a service identifier, the parameter definition, and the validation rule.

9. The method according to claim 7, wherein based on the task parameters of the second task and the to-be-executed model context protocol service, executing the second task comprises: Verifying a task parameter of the second task based on the verification rule, wherein the verification rule is based on the first task; as well as In response to successful verification of the task parameters of the second task, the model context protocol service to be executed associated with the second task is executed based on the task parameters of the second task and the model context protocol service to be executed.

10. The method according to claim 1, further comprising: The second task and task parameters of the second task are determined by a decision model based on the user input.

11. The method according to claim 10, wherein based on the second task, determining the model context protocol service to be executed comprises: determining a semantic relevance between the second task and the model context protocol service; as well as Based on the semantic relevance, the decision model determines the model context protocol service to be executed from the model context protocol services.

12. The method according to claim 11, wherein based on the semantic relevance, determining, by the decision model, the model context protocol service to be executed from the model context protocol services comprises: Based on the semantic relevance, determining a to-be-executed model context protocol service set associated with the second task from the model context protocol services; as well as Generate a graph with the model context protocol services to be executed in the set of model context protocol services to be executed as nodes; as well as Based on the graph, an execution path is determined, the execution path indicating dependencies between the model context protocol services to be executed, and the model context protocol services to be executed are associated with the second task. 13 . The method according to claim 10 , wherein the task parameters of the second task include a first task parameter and a second task parameter, the first task parameter is determined based on the user input, and the second task parameter is a default value.

14. The method according to claim 1, further comprising: Monitoring the execution process of the second task; as well as In response to an error in the execution process, the model context protocol service to be executed is re-determined.

15. The method according to claim 1, further comprising: After the execution of the second task is completed, obtaining feedback from the user on the completion of the second task; and determining a score for completion of the second task; Based on the feedback and the scores, the decision model is adjusted.

16. The method according to claim 1, further comprising: Determining key monitoring points during the execution of the second task; In response to the execution process of the second task at the key monitoring point, a third task parameter of the second task input by a user is obtained.

17. The method according to claim 1, wherein obtaining the operation sequence of the user on the user interface for the first task comprises: Initializing a system state associated with the first task; Recording the interactive operation of the user to complete the first task on the user interface; as well as The interactive operation is converted into the operation sequence.

18. The method according to claim 16, wherein recording the interactive operation of the user to complete the first task on the user interface comprises: Recording the functional attributes of the interactive elements corresponding to the interactive operation; Recording a path of data flow input, conversion, and output, the data flow being associated with the first task; as well as A change point of a system state associated with the first task after the first task is completed is recorded.

19. The method according to claim 1, further comprising: In response to completion of the second task, determining a third task; as well as The third task is performed on the user interface, where the third task is associated with the second task.

20. An apparatus for performing a task, comprising: An operation sequence acquisition module, configured to acquire an operation sequence of a user for a first task on a user interface, wherein the operation sequence includes operation steps required to complete the first task; An operation step conversion module, configured to convert the operation steps into a model context protocol service based on the operation sequence; A user input acquisition module, configured to acquire user input, where the user input indicates a second task, and the user input includes at least one of text input or voice input; a to-be-executed model context protocol service determination module, configured to determine, based on the second task, a to-be-executed model context protocol service, wherein the to-be-executed model context protocol service is associated with the second task; a second task execution module configured to execute the second task based on task parameters of the second task and the model context protocol service to be executed, the second task consisting of the model context protocol service to be executed associated with the second task, the task parameters of the second task being determined based on the user input; as well as The execution result display module is configured to display the execution result of the second task on the user interface.

21. An electronic device comprising: processor; as well as A memory coupled to the processor, the memory having instructions stored therein, wherein when the instructions are executed by the processor, the electronic device executes the method according to any one of claims 1 to 19.

22. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine executable instructions which, when executed, cause a machine to implement the method according to any one of claims 1 to 19.

Images