A data processing method and device, computer equipment and storage medium
By constructing a directed acyclic graph and calling plugins in the plugin collection in logical order, the problems of cumbersome operations and unstable execution during plugin invocation are solved, thus achieving automated execution of complex tasks and accurate results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING QIYI CENTURY SCI & TECH CO LTD
- Filing Date
- 2026-02-03
- Publication Date
- 2026-05-29
AI Technical Summary
In existing technologies, the plugin invocation process is cumbersome and inefficient, leading to unstable execution and uncontrollable results.
By determining the plugin set based on the tasks to be processed, a directed acyclic graph is constructed, and plugins in the plugin set are called in the order of execution logic to achieve consistency in the order of plugins and fine-grained identification of tasks, ensuring that each subtask has a dedicated plugin for execution.
It enables end-to-end automated execution of complex tasks, improving operational efficiency and execution stability, and avoiding inaccurate results caused by chaotic execution logic.
Smart Images
Figure CN122111597A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer equipment, and storage medium. Background Technology
[0002] As a general-purpose semantic engine, the large model is embedded in scenarios such as search, customer service, code, and office work, undertaking the tasks of understanding, generation, and reasoning. It significantly reduces R&D costs, improves the naturalness of interaction and the efficiency of content production, and has become a new generation of digital infrastructure.
[0003] While large models are powerful, they have some limitations. These limitations are typically addressed through plugins, which expand their functionality and application scenarios. Current technology only supports single-plugin calls.
[0004] However, the following problems may occur during the plugin call process: user requirements are often complex tasks with multiple steps, which require manual breakdown, making the operation cumbersome and inefficient; plugin collaboration relies on the large model to "guess" the process, resulting in unstable execution and uncontrollable results, which seriously affects the application performance of the large model in complex scenarios. Summary of the Invention
[0005] This application provides a data processing method, apparatus, computer equipment, and storage medium to solve the problems of cumbersome and inefficient operation during plug-in calling in the prior art, which leads to unstable execution and uncontrollable results.
[0006] In a first aspect, this application provides a data processing method, including: A set of plugins is determined based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A directed acyclic graph is constructed using the execution logic order of each plugin in the plugin set; According to the execution logic order in the directed acyclic graph, each plugin in the plugin set is called sequentially to process each subtask in the task to be processed. When the last plugin in the directed acyclic graph finishes executing the subtask, the task to be processed is determined to be completed.
[0007] In one possible implementation, the step of calling each plugin in the plugin set sequentially to process each subtask in the task to be processed according to the execution logic order of the directed acyclic graph includes: The first plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, and the first plugin is called to execute the corresponding subtask in the task to be processed, so as to obtain the first processing result. The second plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, the second plugin is called, and the first processing result is used as the input of the second plugin to execute the corresponding subtask in the task to be processed, and the second processing result is obtained. Similarly, following the execution logic order in the directed acyclic graph, each plugin in the plugin set is sequentially called to execute the corresponding subtask in the task to be processed, thus obtaining the final processing result.
[0008] In one possible implementation, the method further includes: Determine whether the plugin has successfully executed the corresponding subtask in the pending task. When the current plugin successfully executes the corresponding subtask in the pending task, the next plugin in the directed acyclic graph is called to execute the corresponding subtask in the pending task. If the plugin fails to execute the corresponding subtask in the pending task, it returns an error message and terminates the execution of the pending task.
[0009] In one possible implementation, before determining the set of plugins based on the task to be processed, the method further includes: Obtain the task to be processed, and decompose the task to be processed to obtain one or more sub-tasks.
[0010] In one possible implementation, determining the plugin set based on the task to be processed includes: For each subtask in the task to be processed, match the corresponding plugin from the preset plugin library; Based on the plugins corresponding to each subtask, a set of plugins corresponding to the task to be processed is obtained.
[0011] In one possible implementation, constructing a directed acyclic graph using the execution logic order of the various plugins in the plugin set includes: Analyze the dependencies between the plugins in the plugin set to determine the execution logic order of the plugins in the plugin set; A directed acyclic graph is constructed based on the execution logic order of each plugin in the plugin set, wherein each node of the directed acyclic graph contains plugin information of the corresponding plugin.
[0012] In one possible implementation, the method further includes: The execution engine of a workflow sequentially controls the plugins corresponding to each node of the directed acyclic graph to execute the corresponding subtasks, and stores the execution results of each plugin executing the corresponding subtasks into a variable pool; The execution results in the variable pool serve as the input for the corresponding subtask to be executed by the plugin of the next node.
[0013] Secondly, this application provides a data processing apparatus, comprising: The first determining module is used to determine a set of plugins based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A construction module is used to construct a directed acyclic graph using the execution logic order of each plugin in the plugin set; The execution module is used to call each of the plugins in the plugin set in the execution logic order of the directed acyclic graph to process each subtask in the task to be processed in turn. The second determining module is used to determine that the task to be processed has been completed when the last plugin in the directed acyclic graph has finished executing the subtask.
[0014] Thirdly, this application provides a computer device, including: a processor and a memory, wherein the processor is configured to execute a data processing program stored in the memory to implement the data processing method described in any one of the first aspects.
[0015] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the data processing method described in any one aspect.
[0016] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application determines a set of plugins based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A directed acyclic graph (DAG) is constructed using the execution logic order of each plugin in the DAG. Each plugin in the set is called sequentially to process each subtask in the task to be processed according to the execution logic order in the DAG. When the last plugin in the DAG finishes executing the subtask, the task to be processed is determined to be completed. By decomposing the task to be processed into subtasks according to the step attributes of the task, the task to be processed is finely identified, ensuring the accuracy of task execution. The corresponding plugins for executing subtasks are matched according to the subtasks, and each subtask has a dedicated plugin for processing, ensuring the integrity of the task to be processed. The DAG ensures the consistency of the plugin processing order, avoiding the problem of inaccurate results due to chaotic execution logic. Thus, end-to-end automated execution of complex tasks is achieved, improving operational efficiency and execution stability. Attached Figure Description
[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0019] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.
[0020] Figure 1 A flowchart illustrating an embodiment of a data processing method provided in this application; Figure 2 A flowchart illustrating an embodiment of another data processing method provided in this application; Figure 3 A flowchart illustrating an embodiment of a real-time monitoring method for subtask processing provided in this application; Figure 4 A block diagram illustrating an embodiment of a data processing apparatus provided in this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0022] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.
[0023] To address the technical problems of cumbersome and inefficient operation during plugin invocation in existing technologies, which leads to unstable execution and uncontrollable results, this application provides a data processing method that enables end-to-end automated execution of complex tasks by intelligently decomposing tasks, automatically arranging the plugin invocation order, and managing parameter transmission, thereby improving operational efficiency and execution stability.
[0024] Figure 1 This is a flowchart illustrating an embodiment of a data processing method provided in this application. Figure 1 As shown, the method includes the following steps: S101. Determine the plugin set based on the task to be processed. The plugin set contains one or more plugins, which are used to execute subtasks in the task to be processed.
[0025] This application is primarily used in scenarios involving the extension of large models through plugins. By automatically combining plugins and workflows with large models, it enables large models to automatically plan and execute complex multi-step tasks, ensuring the sequential execution of plugins.
[0026] In this embodiment, the system first obtains the user-submitted task to be processed and analyzes it to obtain each subtask within the task. Each subtask can be a series of logically connected and independently executable tasks. Then, based on the requirements of the subtasks and the functions of the plugins, a plugin corresponding to each subtask is matched from a preset plugin library. Based on the matched plugins for each subtask, a plugin set corresponding to the task to be processed is obtained.
[0027] S102. Construct a directed acyclic graph using the execution logic order of each plugin in the plugin set.
[0028] In this embodiment, after obtaining the plugin set, the system further analyzes the dependencies between the plugins in the plugin set to determine the execution logic order of each plugin based on these dependencies. These dependencies can be conditional, such as the execution of one plugin requiring the output of another; or they can be parallel, meaning multiple plugins can execute independently simultaneously. Then, the system constructs a directed acyclic graph (DAG) representing the entire workflow topology based on the execution logic order of the plugins in the plugin set. This DAG contains multiple nodes, each corresponding to plugin information. The execution order of the plugins can be reproduced by following the order of the nodes in the DAG.
[0029] S103. In accordance with the execution logic order in the directed acyclic graph, call each plugin in the plugin set to process each subtask in the task to be processed in turn.
[0030] In this embodiment, after obtaining the directed acyclic graph (DAG), the system calls each plugin in the plugin set according to the execution logic order of the DAG, processing each subtask in the pending task sequentially. That is, once the workflow is built and started, the system will automatically schedule and execute tasks according to the DAG. Specifically, the execution engine starts with the initial plugin and gradually advances to subsequent nodes, automatically triggering tasks, monitoring status, and handling exceptions. During process execution, the system uses the result of the previous plugin's execution as an input parameter, accurately transmitting it to the next dependent plugin to ensure the continuity and consistency of the data flow and task flow, until all tasks required by all plugins have been executed.
[0031] S104. When the last plugin in the directed acyclic graph finishes executing its subtask, determine that the pending task has been completed.
[0032] In this embodiment, when the system reaches the last node of the directed acyclic graph, i.e., when the last plugin completes its assigned subtask processing, the system determines that the entire processing flow of the pending task has been completed. At this time, the system can output the final execution result, or output feedback indicating that the task processing is complete, and clear the intermediate state, thereby indicating that the pending task has been successfully executed.
[0033] In an optional embodiment of this application, when an exception occurs during the execution of a subtask of the above-mentioned task to be processed, the system will immediately execute the exception handling mechanism and terminate the execution process of the subtask.
[0034] The technical solution provided in this application determines a plugin set based on the task to be processed. This plugin set contains one or more plugins used to execute subtasks within the task. A directed acyclic graph (DAG) is constructed using the execution logic order of the plugins in the DAG. Each plugin in the DAG is called sequentially to process each subtask of the task according to the execution logic order in the DAG. The task is considered complete when the last plugin in the DAG finishes executing its subtask. By breaking down the task into subtasks according to its step attributes, refined identification of the task is achieved, ensuring the accuracy of task execution. Plugins are matched to each subtask, ensuring the integrity of the task. The DAG ensures consistency in the plugin processing order, preventing inaccurate results due to chaotic execution logic. Therefore, end-to-end automated execution of complex tasks is achieved, improving operational efficiency and execution stability.
[0035] Figure 2 A flowchart illustrating an embodiment of another data processing method provided in this application. Figure 2 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: S201. Obtain the task to be processed, and decompose the task to be processed to obtain one or more sub-tasks.
[0036] This application is primarily used in scenarios involving plugin extensions for large models. By automatically combining plugins and workflows for large models, it enables large models to automatically plan and execute complex multi-step tasks, ensuring the sequential execution of plugins.
[0037] In this embodiment, the user inputs a natural language task, which can be a natural language instruction or a specific task objective, detailing the execution content and order of the task. The large model receives the user-input natural language task and treats it as a task to be processed. Then, the system performs a deep understanding of the context, target elements, and logical dependencies of the task to be processed, decomposes the task into one or more subtasks, and returns the execution order of these subtasks. This task decomposition process may involve step division, conditional judgment, and subtask extraction. Furthermore, the resulting one or more subtasks can be output in a structured data format, such as JSON.
[0038] In one example, a user inputs a natural language task: you need to query last month's sales data, create a graph based on the retrieved data, and send the graph to your boss. These tasks must be performed in sequence. The main model receives this user-input natural language task as a pending task. It then breaks down this task into multiple subtasks: querying sales data, creating a graph based on the sales data, and sending an email to the boss. These subtasks are executed in sequence and output in a structured JSON format. This structured JSON data also includes the plugin parameters for any required plugins, such as the plugin name and the arguments required by the plugin.
[0039] S202. For each subtask in the task to be processed, match the corresponding plugin from the preset plugin library.
[0040] S203. Based on the plugins corresponding to each subtask, obtain the set of plugins corresponding to the task to be processed.
[0041] The following is a unified discussion of S202-S203.
[0042] In this embodiment, after the large model is decomposed into one or more subtasks corresponding to the tasks to be processed, the system matches the plugins corresponding to each subtask from a preset plugin library. Specifically, the system obtains the expected output target based on the requirements of each subtask. Simultaneously, the system maintains a preset plugin library storing multiple plugins, which details the name, capabilities, application scenarios, and functions of each plugin. The system can match the requirements of the subtasks with the functions of the plugins to obtain the plugins corresponding to each subtask. Finally, based on the plugins corresponding to each matched subtask, a plugin set corresponding to the task to be processed is obtained to facilitate subsequent automated scheduling and execution.
[0043] Continuing from the previous example, after the large model outputs the above three sub-tasks, it matches the requirements of the sub-tasks with the functions of the plugins to obtain the corresponding plugins for each sub-task: data query plugin, plotting plugin, and email sending plugin. This results in the set of plugins corresponding to the task to be processed, which facilitates subsequent automated scheduling and execution.
[0044] S204. Analyze the dependencies between the plugins in the plugin set and determine the execution logic order of the plugins in the plugin set.
[0045] In this embodiment, after obtaining the plugin set, the system analyzes the dependencies between each plugin in the plugin set to determine the final execution logic order of each plugin. Specifically, the system first parses the dependencies between each plugin. These dependencies can be sequential relationships of subtask execution, such as executing subtask 1 first and then subtask 2; conditional relationships, such as the execution of one plugin requiring the output of another plugin; or parallel relationships, where multiple plugins can execute independently simultaneously. Based on the obtained dependencies, the execution logic order of each plugin in the plugin set is determined.
[0046] S205. Construct a directed acyclic graph based on the execution logic order of each plugin in the plugin set, wherein each node of the directed acyclic graph contains the plugin information of the corresponding plugin.
[0047] In this embodiment, the system constructs a directed acyclic graph (DAG) representing the entire workflow topology, using each plugin as a node and the data flow and dependencies between plugins as directed edges, based on the execution logic order of each plugin in the plugin set. This clearly describes the execution paths and sequential constraints of all tasks in the workflow. The DAG contains multiple nodes, each corresponding to plugin information, as well as its node ID and parameters. Following the order of the nodes in the DAG allows for the reproduction of the plugin execution order, ensuring consistency in the plugin processing order and preventing inaccurate results due to disordered execution logic.
[0048] Continuing the previous example, using the execution logic sequence of the various plugins obtained above, the structured JSON data of the multiple subtasks is transformed to obtain a directed acyclic graph: query sales data -> draw a graph based on sales data -> send an email to the boss. Each node corresponds to information and parameters of its respective plugin.
[0049] S206. Determine the first plugin from the plugin set according to the execution logic order in the directed acyclic graph, call the first plugin to execute the corresponding subtask in the task to be processed, and obtain the first processing result.
[0050] In this embodiment, after constructing a directed acyclic graph (DAG) representing the entire workflow topology, the system will sequentially call each plugin in the plugin set according to the execution logic order of the DAG, processing each subtask in the task to be processed in turn. Specifically, the system first locates the starting node in the DAG, calls the plugin corresponding to that node (i.e., the first plugin), executes the subtask corresponding to the first plugin in the task to be processed, and obtains the first processing result obtained after the subtask is processed.
[0051] In an optional embodiment of this application, a workflow execution engine sequentially controls the plugins corresponding to each node in the directed acyclic graph to execute their respective subtasks, and stores the execution results of each plugin's corresponding subtasks in a variable pool. Specifically, a variable pool is pre-set in the system. After the execution engine controls the plugins corresponding to each node in the directed acyclic graph to complete their corresponding subtasks and obtain the execution results, it stores the execution results in the aforementioned variable pool so that the next node can retrieve the execution results from the variable pool as input data for the plugins corresponding to the next node.
[0052] Continuing from the previous example, based on the execution logic sequence of the directed acyclic graph (DAG), the system locates the starting node in the DAG as the one for querying sales data. It then calls the corresponding plugin for that node: the data query plugin, to retrieve the sales data from the previous month. The retrieved sales data is then stored in a variable pool for use by the next node.
[0053] S207. Determine the second plugin from the plugin set according to the execution logic order in the directed acyclic graph, call the second plugin, and use the first processing result as the input of the second plugin to execute the corresponding subtask in the task to be processed, and obtain the second processing result.
[0054] In this embodiment, after the system calls the first plugin and executes the corresponding subtask in the pending task, it then determines the subtask corresponding to the second node according to the execution logic order in the directed acyclic graph, and identifies the second plugin from the plugin set. The system retrieves the first processing result from the variable pool as the input data for the second plugin, calls the second plugin, executes the corresponding subtask in the pending task, and obtains the second processing result. After obtaining the second processing result, it stores it in the aforementioned variable pool so that the next node can retrieve the second processing result from the variable pool as the input data for the plugin corresponding to the next node.
[0055] Continuing the previous example, the system, following the execution logic order in the directed acyclic graph, determines the subtask corresponding to the second node: drawing a chart based on sales data. It then selects the second plugin from the plugin set: the charting plugin. The system retrieves the first processing result from the variable pool as input data for the charting plugin, calls the charting plugin, executes the corresponding subtask in the pending task, and obtains the second processing result. This second processing result can be a chart generated based on the sales data. This second processing result is stored in the variable pool so that the next node can retrieve it from the variable pool as input data for its corresponding plugin.
[0056] S208. Similarly, following the execution logic order in the directed acyclic graph, each plugin in the plugin set is called sequentially to execute the corresponding subtask in the task to be processed, and the final processing result is obtained.
[0057] In this embodiment, the system sequentially calls each plugin in the plugin set according to the execution logic order in the directed acyclic graph. After being called, each plugin retrieves the execution result stored in the previous node from the variable pool and uses it as input to execute the corresponding subtask in the task to be processed, obtains the processing result, and stores the processing result in the variable pool for use by subsequent nodes. This process continues until the final processing result is obtained. Each plugin is called, executed, and transmits data sequentially according to the execution logic order in the directed acyclic graph, ensuring the accuracy of data flow between each stage and the controllability of the execution process.
[0058] Continuing the previous example, after executing the subtask corresponding to the second plugin, the system sequentially calls each plugin in the plugin set according to the execution logic order in the directed acyclic graph to execute the subtasks corresponding to subsequent nodes. In this example, the system determines the task corresponding to the third child node and selects the third plugin from the plugin set: the email sending plugin. The system retrieves the second processing result from the variable pool as the input data for the email sending plugin, calls the email sending plugin, executes the corresponding subtask in the pending task, and sends an email to the boss. This is the last node. After completing this subtask, the system will determine that the entire processing flow of the pending task is complete and output feedback indicating that the task has been completed.
[0059] In an optional embodiment of this application, when an exception occurs during the execution of a subtask of the aforementioned task to be processed, the system will immediately execute the exception handling mechanism and terminate the execution flow of that subtask. The specific process of exception handling can be referred to... Figure 3 The relevant descriptions in the text will not be detailed here.
[0060] Figure 2 The illustrated process provides a data processing method that breaks down the task to be processed into subtasks according to the task's step attributes, achieving refined identification of the task and ensuring the accuracy of task execution. It also matches corresponding plugins to each subtask, with each subtask having its own dedicated plugin for processing, ensuring the integrity of the task to be processed. Furthermore, a directed acyclic graph ensures the consistency of the plugin processing order, avoiding inaccurate results due to chaotic execution logic. Thus, it achieves end-to-end automated execution of complex tasks, improving operational efficiency and execution stability.
[0061] Figure 3This document provides a flowchart of an embodiment of a real-time monitoring method for subtask processing. In an optional embodiment of the invention, real-time monitoring of subtask processing is performed to ensure the orderly execution of the entire process of the task to be processed, specifically including the following: S301. During the processing of each plugin subtask, determine whether the current plugin has successfully executed the corresponding subtask in the pending task.
[0062] In this embodiment, the system performs real-time monitoring during the execution of the task to be processed. Whenever a plugin is called to execute its corresponding subtask, the system monitors its running status in real time to determine whether the subtask was successfully executed. By monitoring the execution process, the system determines whether the current plugin has successfully executed the corresponding subtask in the task to be processed.
[0063] S302. When the current plugin successfully executes the corresponding subtask in the pending task, the next plugin in the directed acyclic graph is called to execute the corresponding subtask in the pending task.
[0064] In this embodiment, when the system monitors the current plugin's execution of the corresponding subtask within the pending task and no abnormalities occur, it indicates that the subtask has been successfully executed, and the processing result obtained after the task is completed is stored in the variable pool. Subsequently, the system executes the plugin corresponding to the next node in the directed acyclic graph based on the plugin's execution, thereby retrieving the corresponding subtask from the pending task.
[0065] S303. When the corresponding subtask in the pending task fails to be executed by the current plugin, return an error message and terminate the execution of the pending task.
[0066] In this embodiment, when the system detects any abnormal situation during the execution of a subtask corresponding to a pending task by the current plugin, indicating that the subtask has failed, the system will immediately execute the exception handling mechanism, return an error message, and terminate the execution of the pending task. Users can troubleshoot based on the error message to facilitate the subsequent continuation of the corresponding subtask within the pending task.
[0067] S304. When the last plugin in the directed acyclic graph finishes executing its subtask, the pending task is confirmed to be completed.
[0068] In this embodiment, the task to be processed is determined to be completed when the last plugin in the directed acyclic graph finishes executing its subtask. Specifically, when converting structured JSON data into nodes of the directed acyclic graph, the start node and end node are marked. When the system successfully executes a subtask and that subtask node is the end node, the task to be processed is determined to be completed.
[0069] Figure 3 The illustrated process provides a real-time monitoring method for subtask processing. During the processing of each plugin's subtask, it determines whether the current plugin has successfully executed the corresponding subtask within the pending task. If the current plugin successfully executes the corresponding subtask, it calls the next plugin in the directed acyclic graph (DAG) to execute the corresponding subtask. If the current plugin fails to execute the corresponding subtask, it returns an error message and terminates the execution of the pending task. The pending task is considered complete when the last plugin in the DAG has finished executing its subtask. This method captures and responds to various errors that may occur during task execution in real time, ensuring the orderly execution of the entire process and improving operational efficiency and execution stability.
[0070] Figure 4 This is a block diagram illustrating an embodiment of a data processing apparatus provided in this application. Figure 4 As shown, the device includes: The first determining module 401 is used to determine a set of plugins based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. Construction module 402 is used to construct a directed acyclic graph using the execution logic order of each plugin in the plugin set; Execution module 403 is used to call each of the plugins in the plugin set to process each subtask in the task to be processed in the order of execution logic in the directed acyclic graph. The second determining module 404 is used to determine that the task to be processed has been completed when the last plugin in the directed acyclic graph has finished executing the subtask.
[0071] In one possible implementation, the execution module 403 is specifically configured to: determine the first plugin from the plugin set according to the execution logic order in the directed acyclic graph; call the first plugin to execute the corresponding subtask in the task to be processed, and obtain a first processing result; determine the second plugin from the plugin set according to the execution logic order in the directed acyclic graph; call the second plugin and use the first processing result as input to the second plugin to execute the corresponding subtask in the task to be processed, and obtain a second processing result; and so on, sequentially calling each plugin in the plugin set according to the execution logic order in the directed acyclic graph to execute the corresponding subtask in the task to be processed, and obtain a final processing result.
[0072] In one possible implementation, the device further includes: The judgment module 405 is specifically used to determine whether the current plugin has successfully executed the corresponding subtask in the pending task; when the current plugin has successfully executed the corresponding subtask in the pending task, it calls the next plugin in the directed acyclic graph to execute the corresponding subtask in the pending task; when the current plugin fails to execute the corresponding subtask in the pending task, it returns an error message and terminates the execution of the pending task.
[0073] In one possible implementation, the device further includes: The acquisition module 406 is specifically used to acquire the task to be processed, and to perform task decomposition processing on the task to be processed to obtain one or more sub-tasks.
[0074] In one possible implementation, the first determining module 401 is specifically configured to match the plugin corresponding to each subtask in the task to be processed from a preset plugin library; and obtain the plugin set corresponding to the task to be processed based on the plugin corresponding to each subtask.
[0075] In one possible implementation, the construction module 402 is specifically used to analyze the dependencies of each plugin in the plugin set, determine the execution logic order of each plugin in the plugin set, and construct a directed acyclic graph according to the execution logic order of each plugin in the plugin set, wherein each node of the directed acyclic graph contains plugin information of the corresponding plugin.
[0076] In one possible implementation, the device further includes: The storage module 407 is specifically used to sequentially control the plugins corresponding to each node of the directed acyclic graph to execute the corresponding subtasks through a workflow execution engine, and to store the execution results of each plugin executing the corresponding subtasks into a variable pool; wherein, the execution results in the variable pool serve as the input for the next node to execute the corresponding subtask.
[0077] like Figure 5 As shown in the figure, this application provides a computer device including a processor 511, a communication interface 512, a memory 513, and a communication bus 514, wherein the processor 511, the communication interface 512, and the memory 513 communicate with each other through the communication bus 514. Memory 513 is used to store computer programs; In one embodiment of this application, when the processor 511 executes a program stored in the memory 513, it implements the data processing method provided in any of the foregoing method embodiments, including: A set of plugins is determined based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A directed acyclic graph is constructed using the execution logic order of each plugin in the set of plugins. Each plugin in the set of plugins is called in turn to process each subtask in the task to be processed according to the execution logic order in the directed acyclic graph. When the last plugin in the directed acyclic graph finishes executing the subtask, the task to be processed is determined to be completed.
[0078] In one possible implementation, the first plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, and the first plugin is invoked to execute the corresponding subtask in the task to be processed, resulting in a first processing result; the second plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, and the second plugin is invoked, using the first processing result as input to the second plugin, to execute the corresponding subtask in the task to be processed, resulting in a second processing result; and so on, in which each plugin in the plugin set is invoked sequentially according to the execution logic order in the directed acyclic graph to execute the corresponding subtask in the task to be processed, resulting in a final processing result.
[0079] In one possible implementation, it is determined whether the current plugin successfully executes the corresponding subtask in the pending task; if the current plugin successfully executes the corresponding subtask in the pending task, the next plugin in the directed acyclic graph is called to execute the corresponding subtask in the pending task; if the current plugin fails to execute the corresponding subtask in the pending task, an error message is returned and the execution of the pending task is terminated.
[0080] In one possible implementation, a task to be processed is obtained, and the task to be processed is decomposed to obtain one or more sub-tasks.
[0081] In one possible implementation, for each subtask in the task to be processed, a plugin corresponding to the subtask is matched from a preset plugin library; based on the plugin corresponding to each subtask, a plugin set corresponding to the task to be processed is obtained.
[0082] In one possible implementation, the dependencies between the plugins in the plugin set are analyzed to determine the execution logic order of the plugins in the plugin set; a directed acyclic graph is constructed according to the execution logic order of the plugins in the plugin set, wherein each node of the directed acyclic graph contains plugin information of the corresponding plugin.
[0083] In one possible implementation, a workflow execution engine sequentially controls the plugins corresponding to each node of the directed acyclic graph to execute the corresponding subtasks, and stores the execution results of each plugin executing the corresponding subtasks in a variable pool; wherein, the execution results in the variable pool serve as inputs for the next node to execute the corresponding subtask.
[0084] This application embodiment also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; the memory may also include combinations of the above types of memory. When one or more programs in the storage medium can be executed by one or more processors to implement the data processing method executed on the data processing device side described above. The processor is used to execute the data processing program stored in the memory to implement the following steps of executing the data processing method on the data processing device side: A set of plugins is determined based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A directed acyclic graph is constructed using the execution logic order of each plugin in the set of plugins. Each plugin in the set of plugins is called in turn to process each subtask in the task to be processed according to the execution logic order in the directed acyclic graph. When the last plugin in the directed acyclic graph finishes executing the subtask, the task to be processed is determined to be completed.
[0085] In one possible implementation, the first plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, and the first plugin is invoked to execute the corresponding subtask in the task to be processed, resulting in a first processing result; the second plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, and the second plugin is invoked, using the first processing result as input to the second plugin, to execute the corresponding subtask in the task to be processed, resulting in a second processing result; and so on, in which each plugin in the plugin set is invoked sequentially according to the execution logic order in the directed acyclic graph to execute the corresponding subtask in the task to be processed, resulting in a final processing result.
[0086] In one possible implementation, it is determined whether the current plugin successfully executes the corresponding subtask in the pending task; if the current plugin successfully executes the corresponding subtask in the pending task, the next plugin in the directed acyclic graph is called to execute the corresponding subtask in the pending task; if the current plugin fails to execute the corresponding subtask in the pending task, an error message is returned and the execution of the pending task is terminated.
[0087] In one possible implementation, a task to be processed is obtained, and the task to be processed is decomposed to obtain one or more sub-tasks.
[0088] In one possible implementation, for each subtask in the task to be processed, a plugin corresponding to the subtask is matched from a preset plugin library; based on the plugin corresponding to each subtask, a plugin set corresponding to the task to be processed is obtained.
[0089] In one possible implementation, the dependencies between the plugins in the plugin set are analyzed to determine the execution logic order of the plugins in the plugin set; a directed acyclic graph is constructed according to the execution logic order of the plugins in the plugin set, wherein each node of the directed acyclic graph contains plugin information of the corresponding plugin.
[0090] In one possible implementation, a workflow execution engine sequentially controls the plugins corresponding to each node of the directed acyclic graph to execute the corresponding subtasks, and stores the execution results of each plugin executing the corresponding subtasks in a variable pool; wherein, the execution results in the variable pool serve as inputs for the next node to execute the corresponding subtask.
[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0093] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.
[0094] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A data processing method, characterized in that, The method includes: A set of plugins is determined based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A directed acyclic graph is constructed using the execution logic order of each plugin in the plugin set; According to the execution logic order in the directed acyclic graph, each plugin in the plugin set is called sequentially to process each subtask in the task to be processed. When the last plugin in the directed acyclic graph finishes executing the subtask, the task to be processed is determined to be completed.
2. The method according to claim 1, characterized in that, The step of calling each plugin in the plugin set sequentially to process each subtask in the task to be processed according to the execution logic order of the directed acyclic graph includes: The first plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, and the first plugin is called to execute the corresponding subtask in the task to be processed, so as to obtain the first processing result. The second plugin is determined from the plugin set according to the execution logic order in the directed acyclic graph, the second plugin is called, and the first processing result is used as the input of the second plugin to execute the corresponding subtask in the task to be processed, so as to obtain the second processing result. Similarly, following the execution logic order in the directed acyclic graph, each plugin in the plugin set is sequentially called to execute the corresponding subtask in the task to be processed, thus obtaining the final processing result.
3. The method according to claim 1, characterized in that, The method further includes: Determine whether the plugin has successfully executed the corresponding subtask in the pending task. When the current plugin successfully executes the corresponding subtask in the pending task, the next plugin in the directed acyclic graph is called to execute the corresponding subtask in the pending task. If the plugin fails to execute the corresponding subtask in the pending task, it returns an error message and terminates the execution of the pending task.
4. The method according to claim 1, characterized in that, Before determining the plugin set based on the tasks to be processed, the method further includes: Obtain the task to be processed, and decompose the task to be processed to obtain one or more sub-tasks.
5. The method according to claim 4, characterized in that, The process of determining the plugin set based on the tasks to be processed includes: For each subtask in the task to be processed, match the corresponding plugin from the preset plugin library; Based on the plugins corresponding to each subtask, a set of plugins corresponding to the task to be processed is obtained.
6. The method according to claim 1, characterized in that, The construction of a directed acyclic graph using the execution logic order of each plugin in the plugin set includes: Analyze the dependencies between the plugins in the plugin set to determine the execution logic order of the plugins in the plugin set; A directed acyclic graph is constructed based on the execution logic order of each plugin in the plugin set, wherein each node of the directed acyclic graph contains plugin information of the corresponding plugin.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: The execution engine of a workflow sequentially controls the plugins corresponding to each node of the directed acyclic graph to execute the corresponding subtasks, and stores the execution results of each plugin executing the corresponding subtasks into a variable pool; The execution results in the variable pool serve as the input for the corresponding subtask to be executed by the plugin of the next node.
8. A data processing apparatus, characterized in that, The device includes: The first determining module is used to determine a set of plugins based on the task to be processed. The set of plugins contains one or more plugins, which are used to execute subtasks in the task to be processed. A construction module is used to construct a directed acyclic graph using the execution logic order of each plugin in the plugin set; The execution module is used to call each of the plugins in the plugin set in the execution logic order of the directed acyclic graph to process each subtask in the task to be processed in turn. The second determining module is used to determine that the task to be processed has been completed when the last plugin in the directed acyclic graph has finished executing the subtask.
9. A computer device, characterized in that, include: A processor and a memory, the processor being configured to execute a data processing program stored in the memory to implement the data processing method according to any one of claims 1-7.
10. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the data processing method according to any one of claims 1-7.