Task processing method and related device

Abstract

The invention discloses a task processing method and a related device, which not only realizes unified modeling of tasks, but also fuses resource information required by a task processing unit, can quickly identify task processing requirements of various structural data, provides a solid data foundation for task processing of the various structural data, and improves the task processing efficiency. By planning the execution path of the to-be-executed task, the target execution path meeting the task requirement of the to-be-executed task can be determined, so that compared with a mode of performing task scheduling by constructing a simple and fixed knowledge graph in the prior art, diversified requirements of task processing in a complex application scene are met, and the task scheduling efficiency is improved. And the task processing flexibility, efficiency and accuracy are effectively improved.

Description

Technical Field

[0001] This application relates to the field of task processing technology, specifically to a task processing method and related apparatus. Background Technology

[0002] In current computational processing models, the entire computation process is usually divided into several sequentially executed steps. Each step corresponds to a processing unit with a specific function. The steps are connected in series through explicit input-output dependencies to form an execution flow with a relatively fixed path.

[0003] In related technologies, by combining domain-specific knowledge, static knowledge graphs can be constructed to depict the relationships between tasks, thereby enabling automated scheduling of tasks within the domain.

[0004] However, the above methods are insufficient to address the diverse needs of complex applications. Summary of the Invention

[0005] In view of this, this application provides a task processing method and related apparatus, which not only meets the diverse task requirements in different scenarios, but also effectively improves the accuracy, efficiency and flexibility of task processing.

[0006] To solve the above problems, the technical solution provided in this application is as follows:

[0007] On the one hand, this application provides a task processing method, the method comprising:

[0008] The system acquires tasks to be executed, multi-source heterogeneous data, and a pre-built basic knowledge graph. The nodes of the basic knowledge graph are used to identify the task processing units, the attributes of the nodes are used to identify the attribute information corresponding to the task processing units, and the edges of the basic knowledge graph are used to identify the dependency relationships. The multi-source heterogeneous data includes historical task execution data.

[0009] Based on the multi-source heterogeneous data and the basic knowledge graph, a fused knowledge graph is obtained, wherein the attributes of the edges in the fused knowledge graph include the resource information required by the task processing unit.

[0010] The execution path of the task to be executed is planned based on the fused knowledge graph, and the target execution path that meets the task requirements of the task to be executed is determined.

[0011] In another aspect, this application provides a task processing apparatus, which includes an acquisition unit, a fusion unit, and a determination unit:

[0012] The acquisition unit is used to acquire the task to be executed, multi-source heterogeneous data, and a pre-built basic knowledge graph. The nodes of the basic knowledge graph are used to identify the task processing unit, the attributes of the nodes are used to identify the attribute information corresponding to the task processing unit, the edges of the basic knowledge graph are used to identify the dependency relationship, and the multi-source heterogeneous data includes historical task execution data.

[0013] The fusion unit is used to obtain a fused knowledge graph based on the multi-source heterogeneous data and the basic knowledge graph. The attributes of the edges in the fused knowledge graph include the resource information required by the task processing unit.

[0014] The determining unit is used to plan the execution path of the task to be executed based on the fused knowledge graph, and determine the target execution path that meets the task requirements of the task to be executed.

[0015] In another aspect, this application provides a computer device, which includes a processor and a memory:

[0016] The memory is used to store computer programs;

[0017] The processor is configured to execute the method described in any of the above-described embodiments according to the computer program.

[0018] In another aspect, this application provides a computer-readable storage medium for storing a computer program that, when executed by a computer device, implements the method described in any of the above-mentioned embodiments.

[0019] As can be seen from the above technical solution, this solution first acquires the task to be executed, multi-source heterogeneous data, and a basic knowledge graph. The nodes of the basic knowledge graph are used to identify task processing units, the attributes of the nodes are used to identify the attribute information of the task processing units, and the edges are used to identify the dependencies between task processing units. The multi-source heterogeneous data includes historical task execution data. Based on the multi-source heterogeneous data and the basic knowledge graph, a fused knowledge graph is obtained. This not only achieves unified modeling of tasks but also contains the resource information required by the task processing units. The optimized fused knowledge graph can quickly identify tasks with various data structures and provides a solid data foundation for selecting the execution path of tasks with various data structures. Furthermore, the fused knowledge graph is used to plan the execution path of the task to be executed and determine the target execution path that meets the task requirements. This can meet the diverse needs of task processing in complex application scenarios. Compared with related technologies that use simple, fixed knowledge graphs for task scheduling, this solution effectively improves the flexibility, efficiency, and accuracy of task processing. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 One of the flowcharts of a task processing method provided in this application embodiment;

[0022] Figure 2 A schematic diagram of a basic knowledge graph provided in an embodiment of this application;

[0023] Figure 3 A second schematic flowchart illustrating a task processing method provided in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram of a task processing device provided in an embodiment of this application. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0026] As described in the background section, current hybrid computing models typically divide the entire computational process into several sequentially executed steps, each corresponding to a specific functional processing unit, such as data preprocessing, quantum circuit execution, and result parsing. These steps are linked together through explicit input-output dependencies, forming a relatively fixed execution flow. For example, with the rapid development of quantum computing technology, the hybrid use of quantum and classical computing resources has become an important direction for solving complex problems. Currently, some research has attempted to use independent knowledge graph construction methods to implement a dependency-based task scheduling framework to coordinate the collaborative execution between quantum and classical computing resources.

[0027] However, while this approach achieves automated task scheduling to some extent, its knowledge graph structure is relatively simple and fixed, making it unable to flexibly adapt to task requirements in different scenarios. For example, it cannot prioritize low-time-consuming paths in resource-constrained environments, or attempt multiple processing schemes in parallel and select the best one when high-precision results are required. Furthermore, if a critical task fails, the entire process often needs to be restarted, lacking an effective path switching mechanism to ensure the overall task success rate. More importantly, existing systems lack methods for handling different forms of knowledge representation and knowledge graphs in task modeling, making it difficult to support semantic reasoning and intelligent decision-making across systems and tasks. Moreover, these knowledge graphs are usually statically constructed and non-scalable, lacking awareness of runtime states and failing to provide path evaluation and optimization mechanisms based on historical execution data. This makes it difficult to cope with the diverse performance, accuracy, and fault tolerance requirements of multi-data processing models in complex applications, resulting in insufficient flexibility, accuracy, and efficiency in task processing.

[0028] To overcome the challenges of identifying and processing different task entities, relationships, and attribute descriptions in related technologies, and to address issues such as a relatively simple task scheduling structure, lack of multi-path exploration capabilities, absence of dynamic decision-making mechanisms, and insufficient fault tolerance, this application provides a task processing method. This method not only achieves unified task modeling but also incorporates the resource information required by the task processing unit. The optimized fused knowledge graph can quickly identify tasks with various data structures and provide a solid data foundation for selecting execution paths for tasks with diverse data structures. Furthermore, by planning the execution path of the task to be executed through the fused knowledge graph, the target execution path that meets the task requirements can be determined. This method can meet the diverse needs of task processing in complex application scenarios. Compared to related technologies that use simple, fixed knowledge graphs for task scheduling, this method effectively improves the flexibility, efficiency, and accuracy of task processing.

[0029] The solutions provided in this application relate to the field of task processing technology, and are specifically illustrated through the following embodiments.

[0030] See Figure 1 The diagram shown is one of the flowcharts of a task processing method provided in this application embodiment. In this embodiment, it can be executed by a computer device.

[0031] S101: Obtain tasks to be executed, multi-source heterogeneous data, and pre-built basic knowledge graphs.

[0032] In this application, nodes in the basic knowledge graph are used to identify task processing units, and task processing units are used to identify the smallest execution unit of a task. This application does not impose specific limitations on this. For example, for an image detection task, it may include multiple task processing units such as image data acquisition, data cleaning, format conversion, data detection, and detection result output.

[0033] The attributes of a node are used to identify the attribute information corresponding to the task processing unit. This attribute information is related to the task processing unit, and this application does not impose specific restrictions on it. For example, it may include task identity document (ID), task type, task description, input information, output information, resource requirements, etc.

[0034] The edges of the basic knowledge graph are used to identify the dependencies between task processing units. The edges of the basic knowledge graph are directed edges, pointing to the next task processing unit that can be executed after the current task processing unit is completed. A task processing unit may have multiple outputs, and this application does not impose specific restrictions on this.

[0035] This application does not impose specific limitations on the construction method of the basic knowledge graph. For example, it can be constructed by parsing the data of the task to be executed and the code of the executed task based on pre-built parsing code, obtaining multiple task processing units, and then identifying the internal logical relationships of the multiple task processing units through a large language model. Alternatively, the basic knowledge graph can be constructed by directly identifying the relevant data of the task through a large language model.

[0036] In one possible implementation, the basic knowledge graph is constructed as follows:

[0037] A1: Obtain task information.

[0038] Task information refers to data information related to a task. This application does not impose specific restrictions on this information. For example, it may include task code, log information, performance indicators, and resource data of executed tasks, as well as task code and resource configuration information of unexecuted tasks.

[0039] A2: Process the task information to obtain multiple task processing units.

[0040] This application does not impose specific restrictions on the processing method of task information. For example, it can be identified and encapsulated through programming languages, or parsed through large language models, etc.

[0041] As an example, user-defined task code can be identified and encapsulated using custom decorators (such as `@ct.electron` and `@wraps` for Python code). Each task can be defined with information such as its type, input, output, and resource requirements. Each encapsulated task can be abstracted as a "thinking unit," and each thinking unit corresponds to a task processing unit, including corresponding attribute information such as task type, description, input fields, output fields, and resource requirements. For example, a data preprocessing task can be a data processing unit that receives the raw data path as input, outputs the cleaned data path, and specifies CPU resources for execution; a quantum circuit execution task can be a task processing unit that takes a feature file as input, outputs a measurement result file, and specifies QPU resources for execution.

[0042] Furthermore, the task definition is encapsulated using decorators to obtain multiple task processing units, which helps users or the system to expand and maintain them later.

[0043] A3: Determine the dependencies between multiple task processing units based on the attribute information corresponding to each task processing unit in the multiple task processing units.

[0044] The attribute information includes the output information and output details of the task processing unit.

[0045] By comparing the input and output information of multiple task processing units, the dependency relationship between task processing units is determined. For example, if the input information of task A1 exists in the output information of task B1, it means that there is a dependency relationship between A1 and B2.

[0046] A4: Construct a basic knowledge graph based on multiple task processing units, attribute information, and dependencies.

[0047] First, nodes and node attributes are generated based on task processing units and attribute information: a corresponding task ID is generated for each task processing unit as a node in the basic knowledge graph. The node's attributes are the task processing unit's attribute information, such as task type (e.g., data preprocessing, quantum circuit execution), whether it is a critical path node, input information, output information, resource requirements, etc. The task ID and attribute information can be automatically generated by the system or manually specified by the user.

[0048] Then, directed edges of the knowledge graph are constructed based on the dependencies, specifically including the following steps:

[0049] a. Collect the input and output information corresponding to each task processing unit;

[0050] b. Traverse all task processing unit pairs and compare the output information with the input information to see if they match;

[0051] c. If there is field overlap between the output information of the first task processing unit and the input information of the second task processing unit, then establish an edge from the first task processing unit to the second task processing unit.

[0052] d. Ultimately, a complete basic knowledge graph is formed.

[0053] See Figure 2 The diagram shown is a schematic of a basic knowledge graph provided in an embodiment of this application. The node is the task ID corresponding to the task processing unit. The solid box indicates that the node is a critical path node, and the dashed box indicates that the node is a non-critical path node.

[0054] Therefore, by processing task information and constructing a basic implementation map, a unified model of the task is achieved, which helps with later expansion and maintenance.

[0055] Multi-source heterogeneous data refers to heterogeneous data from multiple sources, including at least historical task execution data, and may also include multidimensional data from other open-source tasks or knowledge graph information, etc. This application does not impose specific limitations on this. Among them, historical task execution data can provide the basic knowledge graph with real resource information required for processing tasks, and thus the basic knowledge graph can be optimized through multi-source heterogeneous data.

[0056] Heterogeneity can be manifested at the level of the execution carrier during task execution, such as the data heterogeneity caused by the differences in instruction sets, parallel mechanisms and memory models of different architecture processors such as CPU, GPU or QPU.

[0057] The task to be executed refers to the task specified by the user that needs to be processed. In this embodiment of the application, the content of the task to be executed is not specifically limited. For example, the task to be executed can be a hybrid computing task, such as a quantum electronic hybrid computing task with high uncertainty, dynamism and heterogeneity.

[0058] S102: Based on multi-source heterogeneous data and basic knowledge graphs, a fused knowledge graph is obtained.

[0059] Among them, the attributes of the edges in the integrated knowledge graph include the resource information required by the task processing unit.

[0060] This application does not impose specific limitations on the implementation method of obtaining a fused knowledge graph based on multi-source heterogeneous data and a basic knowledge graph. For example, a large language model can be used to analyze the semantic information used to identify task processing units in multi-source heterogeneous data and the basic knowledge graph, thereby achieving semantic alignment, mapping the diverse heterogeneous data to the basic knowledge graph, and realizing attribute supplementation and dependency optimization of nodes in the basic knowledge graph.

[0061] Therefore, by integrating knowledge graphs, we can not only quickly identify the task processing requirements of various data structures, but also provide a solid data foundation for selecting execution paths for tasks with diverse data structures. For example, knowledge graphs can enable more efficient processing of quantum electronic hybrid computing tasks that are highly uncertain, dynamic, and heterogeneous.

[0062] S103: Based on the fusion knowledge graph, plan the execution path of the task to be executed and determine the target execution path that meets the task requirements of the task to be executed.

[0063] The appointment requirement refers to the requirements that must be met to perform the task to be performed. This application does not impose specific restrictions on this. In one possible implementation, the task requirements include one or more of the following: time requirements, accuracy requirements, and reliability requirements.

[0064] For example, time requirements could be the need to determine the fastest execution path or one with a shorter execution time than a preset time; accuracy requirements could be the need to determine the most accurate execution path or one with a higher accuracy than a preset threshold; resource requirements could be resource configuration requirements, resource consumption constraints, etc.; and reliability requirements could be the need to provide multiple execution paths. This application does not impose specific limitations on these requirements.

[0065] Therefore, by providing target guidance for path planning of the execution path through a variety of task requirements, it is possible to quickly determine the target execution path that meets the task requirements, thereby effectively improving the flexibility, accuracy, reliability and efficiency of task processing.

[0066] There can be one or more target execution paths. This application does not impose specific restrictions on this. For example, when the task requirement is reliability, a primary execution path and an alternative execution path can be set. When the task to be executed is processed through the primary execution path, if the execution fails, the alternative execution path is selected for execution.

[0067] When performing path planning, a large model can be used to analyze the first and last steps of the task to be executed, and determine the corresponding start and end nodes. Then, the paths between the start and end nodes are explored to obtain multiple execution paths for the task. Then, the resource information required for the multiple execution paths is analyzed (for example, based on historical execution data, the historical average time of each task processing unit is calculated, and the total time of each execution path is calculated by combining the knowledge graph). The target execution path that meets the task requirements is determined (e.g., if speed is the requirement, the path with the shortest total time is selected; if performance is the goal, multiple paths are executed in parallel and the best result is selected based on the quality score; if reliability is the goal, primary and backup paths are set, and the backup path is automatically switched when the primary path fails). Alternatively, during the path exploration process, paths that do not meet the task requirements are continuously eliminated based on the resource information of the explored paths, thus obtaining the target execution path.

[0068] Therefore, the technical solution of this application first obtains the task to be executed, multi-source heterogeneous data, and a basic knowledge graph. The nodes of the basic knowledge graph are used to identify task processing units, the attributes of the nodes are used to identify the attribute information of the task processing units, and the edges are used to identify the dependencies of the task processing units. The multi-source heterogeneous data includes historical task execution data. Then, based on the multi-source heterogeneous data and the basic knowledge graph, a fused knowledge graph is obtained. This not only realizes unified modeling of tasks but also contains the resource information required by the task processing units. This allows the optimized fused knowledge graph to quickly identify tasks with various data structures and provides a solid data foundation for selecting the execution path of tasks with various data structures. Then, the execution path of the task to be executed is planned through the fused knowledge graph, and the target execution path that meets the task requirements of the task to be executed is determined. This can meet the diverse needs of task processing in complex application scenarios. Compared with related technologies that use simple and fixed knowledge graphs for task scheduling, this effectively improves the flexibility, efficiency, and accuracy of task processing.

[0069] In one possible implementation, S103 includes:

[0070] B1: Supplement the basic knowledge graph based on multi-source heterogeneous data.

[0071] By combining keyword recognition and semantic-based context enhancement analysis methods, we can deeply mine the deep semantic information, contextual information, and logical constraints of task entities and dependencies in multi-source heterogeneous data. Then, by combining the semantic similarity between task entities and nodes in the basic knowledge graph, we can improve the alignment accuracy with the basic knowledge graph, supplement the knowledge graph, and improve its robustness.

[0072] For example, when the semantic similarity between a task entity and a third task processing unit in the basic knowledge graph reaches a preset threshold, the attribute information of the task entity identified from multi-source heterogeneous data is mapped to the attribute information of the third task processing unit. When a task entity has a dependency relationship with a fourth task processing unit in the basic knowledge graph, but there is no task processing unit corresponding to the task entity, a new node is added to the basic knowledge graph based on the task entity, and the attribute information of the node is supplemented.

[0073] B2: Perform logical checks on multi-source heterogeneous data and optimize the basic knowledge graph based on the check results.

[0074] Identify conflicts and complementarities of the same task entity in data from different sources, and optimize the dependencies in the basic knowledge graph by combining human intervention and machine learning methods. For example, when it is determined that there are conflicts or complementarities based on the attribute information of the same task entity in different sources, task processing units that can be executed in parallel are generated.

[0075] B3: Reason and fill in the missing parameters in the basic knowledge graph.

[0076] After supplementing and optimizing the basic knowledge graph based on multi-source heterogeneous data, the missing attribute information and relational structure are inferred to fill in the missing information.

[0077] For example, missing dependencies between nodes can be inferred and filled in based on multi-source heterogeneous data (e.g., inferring that task processing unit A must be executed after task processing unit B is completed) or node attributes (e.g., inferring that a certain task processing unit C requires 8GB of memory to execute).

[0078] B4: Generate a fusion knowledge graph.

[0079] Based on the basic knowledge graph with missing parameters filled in, a fused knowledge graph is generated. Furthermore, it can acquire multi-source heterogeneous data in real time, and when new task information is available, it records information such as the execution time, status, input / output paths, resource consumption, and performance metrics of each task, storing this information in local files or a database as required, thus enabling dynamic updates to the fused knowledge graph.

[0080] This constructs a dynamic knowledge graph with complete attribute information, a clear structure, and explicit semantics, providing a more accurate and comprehensive data foundation for subsequent task processing. For example, processing and analyzing different historical codes and graph information of quantum electronics hybrid computing tasks / task flows can provide solid data support for quantum electronics hybrid computing tasks.

[0081] In one possible implementation, during the processing of the task to be executed based on the target execution path, the method further includes:

[0082] If the task corresponding to the first node in the integrated knowledge graph fails to execute, it will fall back to the second node before the first node, and the execution path of the task to be executed will be replanned starting from the second node.

[0083] or,

[0084] If the task corresponding to the first node fails to execute, the task corresponding to the first node is re-executed. When the number of executions reaches the preset number, the process rolls back to the second node, and the execution path of the task to be executed is re-planned starting from the second node.

[0085] The second node is the node that precedes the execution of the task corresponding to the first node during the process of processing the task to be executed. It represents the task execution node that has been executed. This application embodiment does not impose specific restrictions on it. For example, it can be the previous task execution node, the second task execution node before that, or the previous key node, etc.

[0086] This application provides an exception capture and retry mechanism, as well as a path backtracking mechanism, to capture task execution exceptions during the execution of the task to be executed. If the task corresponding to the first node fails to execute, it can backtrack to the second node before the first node to select another path to continue execution, or when a task execution exception occurs, it can attempt to retry according to the configuration. When the preset maximum number of attempts is reached, it can backtrack to the second node to try other paths to continue execution.

[0087] Therefore, by using exception handling and retry mechanisms, as well as path backtracking mechanisms, the integrity of the process can be guaranteed, and the reliability of task processing can be further improved.

[0088] The aforementioned exception capture and retry mechanism, as well as the path backtracking mechanism, can also be used in the path planning process. If it is found that the task corresponding to a node may not be able to be executed normally during the path planning process, the process can be rolled back to the previous task processing unit to re-plan and continue to explore the path that is expected to be executed normally.

[0089] In one possible implementation, S103 includes:

[0090] D1: Based on the task to be executed, determine the starting node and the ending node by integrating the knowledge graph.

[0091] D2: Based on the resource information corresponding to each of the multiple pending paths, determine the evaluation result for each path.

[0092] D3: Based on the evaluation results, determine the target execution path that meets the task requirements of the task to be executed.

[0093] The start node identifies the first task processing unit corresponding to the task to be executed, and the end node identifies the last task processing unit corresponding to the task to be executed. This application does not impose specific limitations on the method for determining the start and end nodes. For example, techniques such as semantic similarity or keyword matching can be used to match the task and expected goal calculated in the fused knowledge graph based on the first step of the task to be executed.

[0094] Multiple undetermined paths refer to multiple paths from the starting node to the ending node.

[0095] For example, after determining the start and end nodes, multi-path exploration is performed in the fused knowledge graph to obtain multiple undetermined paths from the start node to the end node. Then, based on the resource information corresponding to each path, analysis can be performed based on different task requirements to determine the evaluation result of each path under different requirements (e.g., performance score, time evaluation result, etc.) or the evaluation result corresponding to each path can be determined according to the task requirements to be executed. Then, based on the evaluation results, the target execution path that meets the task requirements is quickly determined.

[0096] Therefore, by analyzing the resource information of multiple paths to be executed during the path planning process and determining the evaluation results corresponding to each path, the target execution path that meets the task requirements can be quickly identified. Then, based on the evaluation results, the target execution path can be selected for task processing, which can further improve the adaptability of task execution results to task requirements.

[0097] In one possible implementation, D2 includes:

[0098] E1: Get the evaluation function.

[0099] E2: Based on the evaluation function and the resource information corresponding to each path in the multiple paths, determine the evaluation result corresponding to each path.

[0100] The evaluation function refers to the function used to evaluate the path. It can be a performance evaluation function (such as an accuracy evaluation function, a resource consumption evaluation function), a time evaluation function, or a sum of multiple requirements evaluation functions, etc. This application does not impose specific limitations on this.

[0101] Specifically, the PSTranSE algorithm is used to learn the relationships and calculate the probability model for different paths, forming a unified scoring function. The PSTranSE algorithm can determine the critical path under different requirements based on different parameter settings, as shown in Formula 1. When calculating the function, the corresponding parameters μ and X can be adjusted according to the task dependency and task requirements, and multiple path planning methods can be tried to obtain some node paths with higher scores, thus obtaining the critical path corresponding to different requirements.

[0102] (Formula 1)

[0103] in, A comprehensive score is given to the path from node h to node t. It is the set of all paths from node h to node t. Cosine similarity is calculated, where r represents the probability (similarity) that path p is a critical path. , which are system configuration parameters, and X is the key influencing parameter selected by the user (such as time, performance indicators, etc.). Score the path reasoning ability. To traverse all paths p (the set of paths) from entity h to entity t ), ∏ represents the product operation of all elements in the set.

[0104] Therefore, by combining the evaluation function with the resource information corresponding to the path for analysis, the critical path corresponding to different needs can be screened out, and the evaluation efficiency can be further improved by using the evaluation function.

[0105] To more clearly describe this task processing method, the following explanation uses a real-world scenario. (Reference) Figure 3 The diagram shown is a second flowchart illustrating a task processing method provided in this application. First, a basic knowledge graph model is generated based on typical hybrid computing task code. Then, based on historical tasks and reference graph information, multi-graph information fusion processing is performed, including relationship alignment, relationship coordination, and information completion, to improve the knowledge graph model. Next, new hybrid computing task flows submitted by users are processed according to specific requirements, and target parameters are set. Then, for different parameters and planning needs, combined with a scoring function, different path strategies are calculated and trained on the knowledge graph model based on historical running data. Finally, the dynamically improved knowledge graph model is used to identify the new task graph structure and calculate path scores to obtain the critical path under the corresponding target parameters. Finally, during model operation, the model is continuously optimized by incorporating computing task operation monitoring data.

[0106] The following section uses the processing of quantum electronic hybrid computing tasks as an example to illustrate this task processing method in more detail. The following is an embodiment of a quantum electronic hybrid computing task orchestration method based on heterogeneous knowledge graph processing proposed in this application. Its core lies in constructing a multi-dimensional, structured hybrid task processing model through a unified task modeling mechanism and heterogeneous knowledge graph processing. Based on this, multi-path exploration, dynamic path selection, and intelligent backtracking mechanisms are implemented, thereby improving the flexibility, accuracy, and robustness of the hybrid computing task scheduling system.

[0107] Step 1: Construct a basic knowledge graph.

[0108] First, the nodes of the basic knowledge graph are defined: task processing units and attribute information are obtained from typical quantum-classical hybrid computing task information. The definition of task processing units can be updated and maintained by users or the computing power service system. For example, typical quantum-electronic hybrid computing tasks include data preprocessing, quantum circuit execution, quantum post-processing, and model training, which are predefined and include function names, inputs, outputs, and required resources for these typical tasks. Specific attribute information for task processing units is shown in Table 1.

[0109] Table 1

[0110]

[0111] The implementation steps include:

[0112] a. The system administrator defines and registers preset tasks (through decorators) to obtain multiple task processing units based on the relevant code of the quantum electronic hybrid computing task submitted by the user;

[0113] b. Call the knowledge graph modeling tool and load the task processing units for all quantum electronic hybrid computing tasks;

[0114] c. Initialize the result set.

[0115] Then, the dependencies (edges) of the task processing units are established to form a baseline knowledge graph: the input and output information are automatically parsed, and the dependencies are automatically derived by analyzing the name, input and output information of each task processing unit.

[0116] Implementation steps:

[0117] a. Collect the names of all task processing units and their input and output field information;

[0118] b. Iterate through all task processing unit pairs and compare whether the output fields match the input fields;

[0119] c. Construct a knowledge graph for the task, and set the initial critical path probability (1 / N)r based on the edge connections of each node.

[0120] Step 2: Optimize the graph based on heterogeneous data to obtain a fused knowledge graph.

[0121] Building upon a foundational knowledge graph, and combining it with historical task-based knowledge graph libraries and referenced open-source task-based multi-dimensional knowledge graph information, this process fuses multi-graph information to optimize the foundational graph. This includes semantic relationship alignment, relationship coordination and optimization, and information completion. (Includes:)

[0122] a. Semantic-based Relationship Alignment. This approach combines keyword recognition and semantic-based context enhancement to deeply mine the semantic, contextual, and logical constraints of entities and relationships, improving alignment accuracy. It also comprehensively utilizes multimodal information from the knowledge graph (structure, attributes, text, semantics, etc.) to identify semantically identical or similar relationships across different knowledge graphs, supplementing and correcting them to enhance their robustness. For example, keywords such as "data" and "process" that may be involved in data preprocessing are accurately identified by combining them with contextual logical relationships, supplementing the baseline knowledge graph.

[0123] b. Relationship Coordination and Optimization: Addressing the conflict and complementarity issues of the same entity attributes from different knowledge graphs, this involves coordinating and optimizing parameters through a combination of human intervention and machine learning methods. For example, if two data preprocessing processes have a resource-competing relationship, this can be described in the knowledge graph through parallel relationships, and resource allocation parameters can be set using historical data.

[0124] c. Knowledge Graph Information Completion: Utilizing the fused knowledge graph model, inference is performed on missing parameters and relational structures to complete the missing information and support dynamic updates of the knowledge graph. For example, tasks for which prior-to-back relationships have not yet been established, or missing task relationships, can be supplemented through heterogeneous knowledge graph processing.

[0125] Through the above mechanism, the system can build and optimize the knowledge graphs and code information of various quantum electronic hybrid computing tasks / task flows input by users, and organize them into a dynamic knowledge graph with complete attribute information, clear structure and clear semantics. This not only realizes the unified modeling of tasks, but also provides a solid data foundation for subsequent functions such as path optimization, parallel execution and failure recovery.

[0126] Step 3: Multi-path exploration learning and backtracking based on fused knowledge graph.

[0127] To enhance the accuracy and fault tolerance of knowledge graph applications, multi-path exploration learning and backtracking mechanisms are introduced into the fused knowledge graph. This allows the model to explore and learn different solutions based on possible paths and plan alternative paths according to the specific circumstances of task execution, providing more comprehensive and accurate information references for the knowledge graph model.

[0128] First, for the case where there are multiple quantum circuits to choose from in the task processing from the starting node T1 to the ending node T5, multi-path exploration learning is performed:

[0129] Assuming that task processing unit T2 has multiple possible quantum circuit implementations (such as Circuit A and Circuit B), two sub-tasks can be designed. Then, T3 can comprehensively consider the historical execution time and reliability score of different path nodes, and perform post-processing based on these two outputs respectively. Finally, the fusion task T5 can select different combinations of intermediate results according to the user's task requirements.

[0130] In the multi-path exploration-based learning process, the PSTransE algorithm of knowledge graph will be used to construct an evaluation model, perform relationship learning and probability model calculation for different paths. Refer to the description of the evaluation function above, which will not be repeated here.

[0131] The system includes a backtracking mechanism during multi-path exploration: if a path fails or the result quality is substandard, the system should be able to automatically switch to an alternative path. For example, if the output of a path containing node T3 does not meet expectations, the system can select other alternative paths for node T3 to continue execution and monitor their performance.

[0132] The specific implementation method is as follows:

[0133] Each task node can be configured with a "retry policy" and an "alternative path"; during execution, the system monitors the task status and automatically switches paths if it fails.

[0134] Step 4: Task execution path selection and orchestration based on multi-path exploration.

[0135] In the application of knowledge graph models, a flexible path selection and decision-making mechanism is introduced. It supports the selection and adjustment of the parameters μ and X of the evaluation model according to the user's specific requirements, and performs comprehensive path scoring. This supports the arrangement of task execution paths. The parameters can be set according to requirements such as time priority, performance priority, and reliability priority, and the path planning method can be selected and switched according to task requirements.

[0136] (1) Scenario 1: The task requirement is time priority.

[0137] a. Run all path tasks and record the running time;

[0138] b. Write the runtime log to a local file or database;

[0139] c. Load historical log data;

[0140] d. Calculate the historical average time for each task;

[0141] e. Using the above time consumption (1 / T) as the X parameter, and combining it with the knowledge graph model, calculate the comprehensive score for each path;

[0142] f. Output the paths with higher scores for subsequent scheduling. Select the high-scoring paths, such as T1 -> T2A -> T3A -> T5.

[0143] (2) Scenario 2: The task requirement is performance priority.

[0144] In this mode, the critical path and at least two alternative paths are calculated simultaneously. The corresponding performance metric is selected as the X parameter, and the optimal path is chosen to continue execution based on the comprehensive score. For example, the performance metrics for the model training task in this application may include: data integrity + noise removal rate.

[0145] Taking data preprocessing as an example, we define two quality parameters: data integrity and noise removal.

[0146] Data integrity refers to the proportion of missing values ​​in the data. A lower value is better. The noise removal rate can be determined as follows: assuming you have the original noisy data and a standard clean dataset, you can calculate the error (MSE) between the denoised data and the standard data, and then convert it to the noise removal rate.

[0147] The above content uses a benchmark task definition and registration mechanism (which predefines task code function names, key parameters, and inputs and outputs, and requires user task code functions to be submitted according to predefined rules) to abstract various quantum and classical computing tasks into "thinking units" with clear input and output interfaces, and automatically collects task meta-information, including task type, resource requirements, execution logic, etc., to provide basic data support for subsequent knowledge graph modeling.

[0148] By identifying, processing, and learning from relevant heterogeneous knowledge graphs, the accuracy of multi-task code identification and the comprehensiveness of relational analysis in knowledge graph construction can be improved. This further optimizes and adjusts the dependency edges between tasks, thereby constructing a knowledge graph model that integrates information from heterogeneous knowledge graphs. The dynamic knowledge graph construction method based on heterogeneous knowledge graph processing not only clearly expresses the data flow and control flow relationships between tasks but also supports dynamic updates and expansions, adapting to constantly changing task sets and dependency structures, thus ensuring the entire task process has good maintainability and scalability.

[0149] By introducing a knowledge graph-based path evaluation and exploration mechanism, the demands for flexibility and robustness in complex task scenarios can be addressed. During the training of the knowledge graph model, historical models (heterogeneous knowledge graphs) and task logs are dynamically identified and learned. A path evaluation function is used to calculate the comprehensive score of each path under the influence of parameters such as average time consumption, success rate, and result quality. Based on the current task objective (e.g., pursuing speed, performance, or reliability), the path with the highest score is dynamically selected as the critical path. Simultaneously, users can set the weight parameters for path selection according to their actual needs, thus achieving a flexible balance between accuracy and efficiency.

[0150] Therefore, the technical solution of this application organically integrates task modeling, graph construction, heterogeneous processing, path exploration, dynamic decision-making, and fault-tolerance mechanisms to construct an intelligent task scheduling framework for quantum electronic hybrid computing environments. This method not only solves the problems of simple task flow structures, lack of flexibility and intelligence in existing technologies, but also provides a novel solution for task management in complex heterogeneous computing environments. It improves the efficiency of task orchestration, makes resource allocation more rational, optimizes execution paths, and significantly enhances the overall performance and response speed of task execution, demonstrating broad applicability and promotional value. The beneficial effects are as follows:

[0151] (1) The comprehensiveness and accuracy of task identification.

[0152] By employing a dynamic modeling approach that supports heterogeneous knowledge graph processing, a more accurate, efficient, and flexible method for constructing hybrid computing task processing models has been developed. This method enables more comprehensive and accurate learning and processing of heterogeneous knowledge graph information, and automatically collects, organizes, and refines key information for each task, including but not limited to task name, input / output parameter types, and resource requirements. This ensures that each task participating in the computation can be accurately identified and understood by the system. This precision lays a solid foundation for subsequent task management and optimization.

[0153] (2) The richness of knowledge graphs.

[0154] In the multi-task relationship analysis stage, by using semantic-based relationship alignment, coordination optimization, and information completion methods, we can more fully learn and utilize existing knowledge graph resources. Combined with multi-source heterogeneous knowledge graph processing methods, we can comprehensively and accurately mine the relationships between tasks, especially those logical connections formed based on similar input / output parameters and keyword information. This invention can effectively establish a dynamic knowledge graph between tasks.

[0155] (3) Flexibility and adaptability in task handling.

[0156] The dynamic knowledge graph constructed based on this invention can also undergo iterative optimization as hardware infrastructure and network environment change, new tasks are added, or historical task training data is updated, to reflect the latest task status and relationships. Therefore, this framework provides continuous support and services during both the development and operation phases. Particularly in terms of dynamic adjustment, it can automatically adjust the computation process according to actual conditions, improving the system's flexibility and adaptability.

[0157] Based on the above embodiments, this application provides a task processing apparatus, with reference to... Figure 4 The diagram shown is a schematic of a task processing device provided in an embodiment of this application. The device 400 includes an acquisition unit 401, a fusion unit 402, and a determination unit 403.

[0158] The acquisition unit is used to acquire the task to be executed, multi-source heterogeneous data, and a pre-built basic knowledge graph. The nodes of the basic knowledge graph are used to identify the task processing unit, the attributes of the nodes are used to identify the attribute information corresponding to the task processing unit, the edges of the basic knowledge graph are used to identify the dependency relationship, and the multi-source heterogeneous data includes historical task execution data.

[0159] The fusion unit is used to obtain a fused knowledge graph based on the multi-source heterogeneous data and the basic knowledge graph. The attributes of the edges in the fused knowledge graph include the resource information required by the task processing unit.

[0160] The determining unit is used to plan the execution path of the task to be executed based on the fused knowledge graph, and determine the target execution path that meets the task requirements of the task to be executed.

[0161] In one possible implementation, the underlying knowledge graph is constructed in the following way:

[0162] Obtain task information;

[0163] The task information is processed to obtain multiple task processing units;

[0164] Based on the attribute information corresponding to each of the plurality of task processing units, the dependency relationship between the plurality of task processing units is determined, wherein the attribute information includes the output information of the task processing unit;

[0165] The basic knowledge graph is constructed based on the multiple task processing units, the attribute information, and the dependencies.

[0166] In one possible implementation, the fusion unit is used for:

[0167] Based on the aforementioned multi-source heterogeneous data, the basic knowledge graph is supplemented;

[0168] Logical detection is performed on the multi-source heterogeneous data, and the basic knowledge graph is optimized based on the detection results;

[0169] Reasoning and filling in the missing parameters in the aforementioned basic knowledge graph;

[0170] Generate a fusion knowledge graph.

[0171] In one possible implementation, the device further includes a rollback unit for:

[0172] If the task corresponding to the first node in the fused knowledge graph fails to execute, then backtrack to the second node before the first node, and replan the execution path of the task to be executed starting from the second node;

[0173] or,

[0174] If the task corresponding to the first node fails to execute, the task corresponding to the first node is re-executed. When the number of executions reaches a preset number, the process falls back to the second node, and the execution path of the task to be executed is replanned starting from the second node.

[0175] In one possible implementation, the determining unit is configured to:

[0176] Based on the task to be executed, a start node and an end node are determined in the fused knowledge graph. The start node is used to identify the first task processing unit corresponding to the task to be executed, and the end node is used to identify the last task processing unit corresponding to the task to be executed.

[0177] Based on the resource information corresponding to each of the multiple undetermined paths, the evaluation result corresponding to each path is determined, wherein the multiple undetermined paths are multiple paths from the starting node to the ending node;

[0178] Based on the evaluation results, a target execution path that meets the task requirements of the task to be executed is determined.

[0179] In one possible implementation, the determining unit is configured to:

[0180] Obtain the evaluation function;

[0181] Based on the evaluation function and the resource information corresponding to each of the multiple paths, the evaluation result corresponding to each path is determined.

[0182] In one possible implementation, the task requirements include one or more of the following: time requirements, accuracy requirements, resource requirements, and reliability requirements.

[0183] Based on the above embodiments, this application provides a computer device, which includes a processor and a memory:

[0184] The memory is used to store computer programs;

[0185] The processor is used to perform the above-described task processing method according to the computer program.

[0186] Based on the above embodiments, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a computer device, implements the method for performing the above-mentioned task processing.

[0187] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems or apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and relevant parts can be referred to the method section.

[0188] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method of task processing, characterized by, The method comprises: acquiring a to-be-executed task, multi-source heterogeneous data, and a pre-constructed basic knowledge graph, a node of the basic knowledge graph being used to identify a task processing unit, an attribute of the node being used to identify attribute information corresponding to the corresponding task processing unit, an edge of the basic knowledge graph being used to identify a dependency relationship between the task processing units, and the multi-source heterogeneous data including historical task execution data; based on the multi-source heterogeneous data and the basic knowledge graph, obtaining a fused knowledge graph, an attribute of an edge in the fused knowledge graph including resource information required by the task processing units; based on the fused knowledge graph, planning an execution path of the to-be-executed task and determining a target execution path meeting a task demand of the to-be-executed task.

2. The method of claim 1, wherein, The basic knowledge graph is constructed by: acquiring task information; processing the task information to obtain a plurality of task processing units; based on attribute information corresponding to each task processing unit in the plurality of task processing units, determining a dependency relationship between the plurality of task processing units, the attribute information including output information of the task processing units and output information; based on the plurality of task processing units, the attribute information, and the dependency relationship, constructing the basic knowledge graph.

3. The method of claim 1, wherein, The obtaining of the fused knowledge graph based on the multi-source heterogeneous data and the basic knowledge graph comprises: complementing the basic knowledge graph based on the multi-source heterogeneous data; performing logical detection on the multi-source heterogeneous data and optimizing the basic knowledge graph based on a detection result; reasoning and filling in missing parameters in the basic knowledge graph; and generating the fused knowledge graph.

4. The method of claim 1, wherein, In the process of processing the to-be-executed path based on the target execution path, the method further comprises: if a task corresponding to a first node in the fused knowledge graph fails, falling back to a second node before the first node and re-planning an execution path of the to-be-executed task from the second node; or, if the task corresponding to the first node fails, re-executing the task corresponding to the first node, and when a number of executions reaches a preset number, falling back to the second node and re-planning the execution path of the to-be-executed task from the second node.

5. The method of claim 1, wherein, The planning of the execution path of the to-be-executed task based on the fused knowledge graph and the determination of the target execution path meeting the task demand of the to-be-executed task comprise: based on the to-be-executed task, determining a start node and an end node in the fused knowledge graph, the start node being used to identify a first task processing unit corresponding to the to-be-executed task and the end node being used to identify a last task processing unit corresponding to the to-be-executed task; based on the resource information corresponding to each of a plurality of to-be-determined paths, determining an evaluation result corresponding to each path, the plurality of to-be-determined paths being a plurality of paths from the start node to the end node; based on the evaluation result, determining the target execution path meeting the task demand of the to-be-executed task.

6. The method of claim 5, wherein, The determining the evaluation result corresponding to each path in the plurality of paths based on the resource information corresponding to each path in the plurality of paths includes: obtaining an evaluation function; determining the evaluation result corresponding to each path in the plurality of paths based on the evaluation function and the resource information corresponding to each path in the plurality of paths.

7. The method according to any one of claims 1 to 5, characterized in that, The task requirement includes one or more of a time requirement, an accuracy requirement, a resource requirement, and a reliability requirement.

8. A task processing apparatus characterized by comprising: The device includes an obtaining unit, a fusion unit, and a determining unit: The obtaining unit is configured to obtain a to-be-executed task, multi-source heterogeneous data, and a pre-constructed basic knowledge graph, a node of the basic knowledge graph is configured to identify a task processing unit, an attribute of the node is configured to identify attribute information corresponding to the corresponding task processing unit, an edge of the basic knowledge graph is configured to identify a dependency relationship between the task processing units, and the multi-source heterogeneous data includes historical task execution data. The fusion unit is configured to obtain a fused knowledge graph based on the multi-source heterogeneous data and the basic knowledge graph, and an attribute of an edge in the fused knowledge graph includes resource information required by the task processing units. The determining unit is configured to plan an execution path of the to-be-executed task based on the fused knowledge graph, and determine a target execution path that meets a task requirement of the to-be-executed task.

9. A computer device, comprising: The computer device includes a processor and a memory: The memory is configured to store a computer program. The processor is configured to execute the method according to any one of claims 1-7 according to the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is configured to store a computer program, and the computer program implements the method according to any one of claims 1-7 when executed by a computer device.

Images