Task rollback method and device

Abstract

The embodiment of the invention provides a task rollback method and device, and the method comprises the steps: executing at least two initial tasks contained in a target service through updating metadata corresponding to the target service, and obtaining task execution data. And constructing a task dependency graph of the at least two initial tasks, and determining a task execution sequence corresponding to the task dependency graph based on the task execution data. Under the condition that the target task node in the task dependency graph meets the rollback condition, it is shown that the target node in the task dependency graph needs to execute rollback operation. And determining associated task nodes associated with the target task node in the task dependency graph according to the task execution sequence, and updating the task execution data in the metadata dimension. The task rollback operation is executed for the associated task nodes according to the updated task execution data, the number of the associated task nodes needing to execute the task rollback operation can be reduced, the time for executing the task rollback operation is shortened, and waste of computing resources is reduced.

Description

Technical Field

[0001] The embodiments of the present specification relate to the field of computer technology, and in particular to a task rollback method and device. Background Art

[0002] The rollback mechanism of traditional databases usually relies on transaction logs or pre-created data backups. When performing a task rollback operation, it is necessary to restore the previous state of the database by restoring the backup data or replaying the transaction log. The transaction log records all changes to the database, while the backup is usually a complete copy of the database at a specific point in time. Through these two methods, traditional databases can ensure data consistency and transaction atomicity. However, these methods often face large performance bottlenecks under large-scale data processing and high-concurrency tasks, especially in scenarios with large data volumes or frequent operations, task rollback operations often take a long time and have high computational costs. Therefore, there is an urgent need for a more effective task rollback method to solve the above problems. Summary of the invention

[0003] In view of this, an embodiment of the present specification provides a task rollback method. One or more embodiments of the present specification also relate to a task rollback device, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a task rollback method is provided, including: Execute at least two initial tasks included in the target business by updating metadata corresponding to the target business, and obtain task execution data; Constructing a task dependency graph of the at least two initial tasks, and determining a task execution order corresponding to the task dependency graph based on the task execution data; In a case where a target task node in the task dependency graph satisfies a rollback condition, determining, in the task dependency graph, associated task nodes associated with the target task node according to the task execution order; The task execution data is updated in the metadata dimension, and a task rollback operation is performed on the associated task node according to the updated task execution data.

[0005] According to a second aspect of an embodiment of this specification, a task rollback device is provided, including: An execution module is configured to execute at least two initial tasks included in the target business by updating metadata corresponding to the target business, and obtain task execution data; A construction module is configured to construct a task dependency graph of the at least two initial tasks, and determine a task execution order corresponding to the task dependency graph based on the task execution data; A determination module is configured to determine, in the task dependency graph, an associated task node associated with the target task node according to the task execution order when the target task node in the task dependency graph meets the rollback condition; The rollback module is configured to update the task execution data in the metadata dimension, and perform a task rollback operation on the associated task node according to the updated task execution data.

[0006] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned task rollback method are implemented.

[0007] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned task rollback method are implemented.

[0008] According to a fifth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which implement the steps of the above-mentioned task rollback method when executed by a processor.

[0009] An embodiment of the present specification executes at least two initial tasks included in the target business by updating the metadata corresponding to the target business, and obtains task execution data. The execution of at least two initial tasks is achieved by updating the metadata, which can improve the efficiency of task execution. A task dependency graph of at least two initial tasks is constructed, and the task execution order corresponding to the task dependency graph is determined based on the task execution data. When the target task node in the task dependency graph meets the rollback condition, it indicates that the target node in the task dependency graph needs to perform a rollback operation. In the task dependency graph, the associated task nodes associated with the target task node are determined according to the task execution order, and the task execution data is updated in the metadata dimension. Performing a task rollback operation on the associated task nodes according to the updated task execution data can reduce the number of associated task nodes that need to perform the task rollback operation, shorten the time to perform the task rollback operation, and reduce the waste of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic diagram of a task rollback method provided by an embodiment of this specification; Figure 2 is a flowchart of a task rollback method provided by an embodiment of this specification; Figure 3It is a task dependency graph of a task rollback method provided by an embodiment of this specification; Figure 4 It is a task dependency graph reconstruction diagram of a task rollback method provided by an embodiment of this specification; Figure 5 is a process flow chart of a task rollback method provided by an embodiment of this specification; Figure 6 It is a structural diagram of a task rollback device provided by an embodiment of this specification; Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0011] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0012] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0013] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0014] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0015] First, the terms involved in one or more embodiments of this specification are explained.

[0016] Data Lake: A data storage architecture. A data lake is a system or storage that stores data in its natural / original format, usually as object blocks or files. A data lake can hold large amounts of data of different types and formats and support advanced queries and processing for data analytics and machine learning. It is a centralized repository that allows users to store all structured and unstructured data at any scale.

[0017] Iceberg (Apache Iceberg): An open table format for large-scale data analysis scenarios, which can also be regarded as a data lake solution. Iceberg provides a rich table operation interface for the upper-level data processing engine, while shielding the differences in the underlying data storage format. It can serve as an intermediate layer to provide abstraction and integration between the computing engine and the underlying storage.

[0018] Metadata: data about data, or data describing data. It mainly describes the attributes, characteristics, organization, relationships and other information of data, and is used to support data storage, retrieval, management and other functions.

[0019] Traditional database rollback technology has low rollback efficiency, lacks refined control of the task level, and is difficult to effectively manage dependencies in complex clearing task flows, which can easily lead to resource waste and incomplete rollback. Data lake technologies such as Iceberg provide snapshot-based rollback functions, allowing users to restore historical versions of tables. Unlike the rollback mechanism of traditional databases, Iceberg records every change in table data in the form of incremental snapshots. This method significantly improves the efficiency of rollback, especially in big data environments. By operating metadata rather than the data itself to achieve table backup and recovery, Iceberg can avoid the storage cost problem in traditional methods and improve the speed of data recovery. However, Iceberg's rollback capability is still limited to single-table operations, and has not yet provided sufficient support for complex multi-task and multi-table operation scenarios. Traditional database rollback technology and data lake technologies such as Iceberg face problems such as low efficiency, insufficient accuracy and complex management when dealing with complex clearing task flow rollback scenarios, and it is difficult to meet the actual needs of efficient and accurate task rollback in big data environments. Therefore, one or more embodiments of this specification propose a task rollback method to solve the above problems.

[0020] Figure 1 A schematic diagram of a task rollback method provided according to an embodiment of the present specification is shown. An embodiment of the present specification executes at least two initial tasks included in the target business by updating the metadata corresponding to the target business to obtain task execution data. The execution of at least two initial tasks is achieved by updating metadata, which can improve the efficiency of task execution. When the initial task is executed, modifications and deletions are not made directly to the original data, but the data is saved in an incremental modification mode. That is, after each modification and deletion, the historical data still exists, but a new incremental file will be generated to save the definition, structure, location and other information of these data, which is called "metadata". Data changes are achieved by operating metadata without operating the data itself.

[0021] By performing dependency parsing on at least two initial tasks, the dependency relationship and data flow between the initial tasks are obtained, and a task dependency graph of at least two initial tasks is constructed based on the dependency relationship and the data flow. The task execution order corresponding to the task dependency graph is determined based on the task execution data. When the target task node in the task dependency graph meets the rollback condition, it means that the target node in the task dependency graph needs to perform a rollback operation. At this time, the associated task nodes associated with the target task node are determined in the task dependency graph according to the task execution order, and the task execution data is updated in the metadata dimension. Performing a task rollback operation on the associated task nodes according to the updated task execution data can reduce the number of associated task nodes that need to perform the task rollback operation, shorten the time for performing the task rollback operation, and reduce the waste of computing resources.

[0022] In this specification, a task rollback method is provided. This specification also relates to a task rollback device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0023] See also Figure 2 , Figure 2 A flowchart of a task rollback method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0024] Step 202: executing at least two initial tasks included in the target business by updating metadata corresponding to the target business, and obtaining task execution data.

[0025] Specifically, the target business can be a clearing business, a data analysis business, or a data lake management business. Clearing business includes but is not limited to clearing business in the financial field. Clearing business in the financial field can be clearing business in scenarios such as banking and insurance. In the data analysis business, when data scientists and analysts explore different data transformations and try to make changes to data sets, if the analysis leads to bad results, they can perform a rollback to return to the previous state. This helps them make more accurate data analysis and decisions. When validating data or running tests, data rollback provides the function of undoing changes during the test process, thereby ensuring the integrity of the production data set. In the data lake management business, when performing data cleaning and transformation in the data lake, if errors or unexpected results occur, data rollback can be used to undo changes and restore to the previous state. In the process of data integration and synchronization, if problems such as data conflicts or overwriting occur, data rollback can help solve these problems and ensure the accuracy of the data. Metadata is data that describes or records the data of the target business execution process. The initial task is the data processing task under the target business, which is used to achieve the addition, deletion, or modification of data under the target business. The task execution data includes metadata corresponding to at least two initial task executions, including the time when the initial task was submitted, the execution status, and the execution result of the initial task.

[0026] Based on this, when executing at least two initial tasks included in the target business, the metadata corresponding to the target business can be updated in the metadata dimension to implement the execution of the at least two initial tasks. The at least two initial tasks are executed to obtain task execution data including the metadata of the target business.

[0027] In actual applications, at least two initial tasks of the target business can be implemented based on data lake technology. When the initial task is executed, the original data corresponding to the initial task is not directly processed, but the initial task is executed by incremental update of data. The newly added metadata is used to save the definition, structure, location and other information of the original data operation.

[0028] Furthermore, at least two initial tasks usually have task dependencies, that is, the execution of one task needs to depend on the execution result of the previous task. Therefore, when executing at least two initial tasks, it is necessary to distinguish between main tasks and branch tasks, which is specifically implemented as follows: Determine a main task and a branch task from the at least two initial tasks included in the target business; execute the main task by updating the metadata corresponding to the target business in the metadata dimension to obtain main task execution data; execute the branch task by updating the main task execution data in the metadata dimension to obtain the task execution data.

[0029] Specifically, the main task refers to the dependent initial task. The execution of the branch task needs to depend on the execution result of the main task. The branch task continues to be executed based on the execution result of the main task. The main task execution data refers to the data obtained by adding the main task metadata corresponding to the main task execution, the main task execution status data, the main task submission time and other data on the basis of the metadata of the target business.

[0030] Based on this, the main task and the branch task are determined based on the task dependency and the task data flow in at least two initial tasks included in the target business. The main task is executed by updating the metadata corresponding to the target business in the metadata dimension, and the main task metadata, main task execution status data, and main task submission time of executing the main task are added on the basis of the metadata of the target business to obtain the main task execution data. The branch task is executed by updating the main task execution data in the metadata dimension, and the branch task metadata, branch task execution status data, and branch task submission time of executing the branch task are added on the basis of the main task execution data to obtain the task execution data.

[0031] For example, in a batch processing scenario, the target business can be a scientific computing or data processing task. The data processing process contains multiple consecutive subtasks. During or after the execution of any subtask, you can roll back to the previous subtask of the current subtask according to the task rollback requirements, and start again with the previous subtask to continue executing the target business on the basis of the previous subtask. The series of subtasks included in the data processing process are the initial tasks included in the target business. The data processing process contains the main process and branch processes. When executing the target business, you need to first execute the main task on the main process in the metadata dimension and record the task execution information. Then execute the branch task on the branch process based on the task execution information corresponding to the main task.

[0032] To summarize, by updating the metadata corresponding to the target business in the metadata dimension to execute the main task, and by updating the main task in the metadata dimension to execute the data to execute the branch task, the orderly execution of the initial task under the target business is achieved, thereby improving the efficiency and accuracy of task execution.

[0033] Step 204: construct a task dependency graph of the at least two initial tasks, and determine a task execution order corresponding to the task dependency graph based on the task execution data.

[0034] Specifically, after the at least two initial tasks included in the target business are executed by updating the metadata corresponding to the target business and obtaining the task execution data, a task dependency graph of the at least two initial tasks can be constructed, and the task execution order corresponding to the task dependency graph can be determined based on the task execution data, wherein the task dependency graph is a directed acyclic graph. The task dependency graph represents the dependency relationship and data flow between the at least two initial tasks, and the task execution order between the at least two initial tasks can be determined by combining the task dependency graph and the task execution data.

[0035] Based on this, after obtaining the task execution data by executing at least two initial tasks included in the target business through updating the metadata corresponding to the target business, a task dependency graph of at least two initial tasks is constructed according to the dependency relationship and data flow between the at least two initial tasks, and the execution order corresponding to each task node in the task dependency graph is determined based on the task execution data, thereby determining the task execution order corresponding to the task dependency graph.

[0036] Furthermore, considering that there is a task dependency relationship and data flow between at least two initial tasks, the task dependency relationship between at least two initial tasks needs to be fully considered when constructing the task dependency graph. The specific implementation is as follows: Determine the task dependency relationship corresponding to the at least two initial tasks based on the task execution data; construct task nodes based on the at least two initial tasks, and construct a task dependency graph of the at least two initial tasks based on the task dependency relationship and the task nodes.

[0037] Specifically, the task dependency relationship corresponding to at least two initial tasks refers to the dependency relationship between the initial tasks, which can represent the execution order and data flow of the initial tasks. The task node is a graph node, which is represented by a geometric shape such as a circle or a square. The task dependency relationship is represented as a connection line with a pointing direction between the task nodes, and the pointing direction is the data flow direction.

[0038] Based on this, the task dependency relationship corresponding to the at least two initial tasks is determined based on the task submission time of the at least two initial tasks in the task execution data, and the data flow and dependency relationship of the at least two initial tasks. Task nodes are constructed based on the at least two initial tasks, and lines between the task nodes are constructed based on the task dependency relationship to construct a task dependency graph of the at least two initial tasks.

[0039] Continuing with the above example, in a batch processing scenario, when the target business includes 12 initial tasks, such as Task 1 to Task 12, a task dependency graph can be constructed based on the 12 initial tasks, such as Task 1 to Task 12. Based on the task execution data, the task dependency relationships between the 12 initial tasks, such as Task 1 to Task 12, included in the target business can be determined, that is, Task 10 depends on Task 9, Task 8, and Task 12; Task 8 and Task 9 both depend on Task 7, and Task 8 also depends on Task 6; Task 6 depends on Task 3, Task 12 depends on Task 11, and Task 7 depends on Task 4 and Task 5; Task 4 and Task 4 both depend on Task 2; Task 2, Task 3, and Task 11 depend on Task 1. Based on the above dependencies between tasks, the following can be constructed: Figure 3 The task dependency graph shown.

[0040] In summary, task nodes are constructed based on at least two initial tasks, and lines between task nodes are constructed based on task dependencies, so as to construct a task dependency graph of at least two initial tasks and realize visualization of task dependencies of at least two initial tasks.

[0041] Furthermore, when at least two initial tasks included in the target business are executed, when the task execution is completed and submitted, the task submission time will be recorded, and the task execution order can be determined based on the task submission time, which is specifically implemented as follows: Determine the task submission data corresponding to the task dependency graph in the task execution data; determine the task submission times corresponding to at least two graph nodes in the task dependency graph based on the task submission data, and determine the task execution order corresponding to the task dependency graph based on the task submission times of the at least two graph nodes.

[0042] Specifically, the task submission data corresponding to the task dependency graph includes the task execution results, task execution status, task submission time and other data of the initial task corresponding to each task node in the task dependency graph. The task execution order can be determined according to the order of task submission time, and the initial task that is executed first is prioritized.

[0043] Based on this, the task submission data corresponding to the task dependency graph is determined in the task execution data; the task submission time corresponding to at least two graph nodes in the task dependency graph is determined based on the task submission time data included in the task submission data. The task nodes in the task dependency graph are sorted based on the task submission time of at least two graph nodes to obtain the task execution order corresponding to the task dependency graph.

[0044] Using the above example, determine the task submission time corresponding to at least two graph nodes in the task dependency graph in the task execution data. Sort tasks 1 to 12 according to the task submission time to obtain the task execution order: task 1 - task 2 - task 11 - task 5 - task 3 - task 4 - task 6 - task 7 - task 9 - task 12 - task 8 - task 10.

[0045] In summary, the task execution order corresponding to the task dependency graph is determined based on the task submission time of at least two graph nodes respectively, thereby ensuring the accuracy of the task execution order.

[0046] Step 206: When the target task node in the task dependency graph meets the rollback condition, the associated task nodes associated with the target task node are determined in the task dependency graph according to the task execution order.

[0047] Specifically, after constructing the task dependency graph of at least two initial tasks and determining the task execution order corresponding to the task dependency graph based on the task execution data, when the target task node in the task dependency graph meets the rollback condition, the associated task nodes associated with the target task node can be determined in the task dependency graph according to the task execution order, wherein the target task node can be a node that meets the rollback condition, and the target task node meeting the rollback condition can indicate that there is an abnormality in the initial task corresponding to the target task node, or it can indicate that the target task node is a designated node that needs to be rolled back. The task dependency graph can be tested for rollback conditions, and the target task node in the task dependency graph that meets the rollback condition is detected. The associated task node associated with the target task node is the task node corresponding to the initial task that is determined based on the task execution order and continues to be executed after the initial task corresponding to the target task node is executed.

[0048] Based on this, after constructing the task dependency graph of at least two initial tasks and determining the task execution order corresponding to the task dependency graph based on the task execution data, if the target task node in the task dependency graph meets the rollback condition, it means that the initial tasks corresponding to the subsequent task nodes need to be rolled back starting from the target task node. In the task dependency graph, the associated task nodes arranged after the target task node are determined according to the task execution order.

[0049] Further, after determining that the task rollback operation starts from the target task node in the task dependency graph, all associated task nodes executed after the target task node need to be rolled back. The associated task nodes can be determined by performing node detection on subsequent task nodes. The specific implementation is as follows: In the task dependency graph, the subsequent task node of the target task node is determined according to the task execution order; based on the task dependency graph, the target task node and the subsequent task node are detected, and the associated task node associated with the target task node is determined according to the detection result.

[0050] Specifically, the subsequent task node of the target task node is a task node arranged after the target task node based on the task execution order. The purpose of performing node detection on the target task node and the subsequent task node is to detect whether there is a reachable relationship between the subsequent task node and the target task node, that is, to detect whether there is a connection edge relationship between the target task node and the subsequent task node in the task dependency graph.

[0051] Based on this, the subsequent task nodes arranged after the target task node are determined in the task dependency graph according to the task execution order. The target task node and the subsequent task node are detected based on the task dependency graph to detect whether there is a direct dependency relationship between the target task node and the subsequent task node. According to the detection result, the associated task node with a direct dependency relationship with the target task node is determined.

[0052] Using the above example, after determining the task execution order corresponding to the task dependency graph: Task 1-Task 2-Task 11-Task 5-Task 3-Task 4-Task 6-Task 7-Task 9-Task 12-Task 8-Task 10. If Task 4 is determined as the target task node, the subsequent task nodes of Task 4 can be determined based on the task execution order. Figure 4 As shown in (a), according to the task execution order, it can be determined that Task 6, Task 7, Task 9, Task 12, Task 8 and Task 10 are the subsequent task nodes of Task 4. By performing node detection on subsequent task nodes such as Task 6, Task 7, Task 9, Task 12, Task 8 and Task 10, the associated task node corresponding to Task 4 is determined in the subsequent task nodes.

[0053] In summary, in the task dependency graph, the subsequent task nodes arranged after the target task node are determined according to the task execution order, and then the associated task nodes that need to perform the task rollback operation are determined by performing node detection on the target task node and the subsequent task nodes.

[0054] Furthermore, considering that in the task dependency graph, there are nodes in the subsequent task nodes corresponding to the target task node that are not connected to the target task node, such nodes may not be rolled back, and the specific implementation is as follows: Based on the task dependency graph, node connectivity detection is performed on the target task node and at least two subsequent task nodes respectively; and the associated task node connected to the target task node is determined among the at least two subsequent task nodes according to the detection result.

[0055] Specifically, the node connectivity detection is used to detect whether there is an edge connection between the target task node and the subsequent task node in the task dependency graph, that is, whether the target task node and the subsequent task node in the task dependency graph are connected.

[0056] Based on this, node connectivity detection is performed on the target task node and at least two subsequent task nodes based on the task dependency graph. When there are at least two subsequent task nodes, it is necessary to form a detection pair with the target task node and each subsequent task node to perform connectivity detection. According to the detection result, the associated task node connected to the target task node is determined in the at least two subsequent task nodes.

[0057] Using the above example, Figure 4 As shown in (b), after determining that Task 6, Task 7, Task 9, Task 12, Task 8, and Task 10 are the subsequent task nodes of Task 4, the connectivity detection between Task 4, the target task node, and each subsequent task node can be performed based on the task dependency graph. According to the task dependency graph, Task 6 and Task 12 are not connected to Task 4. Then Task 7, Task 9, Task 8, and Task 10 are the associated task nodes connected to Task 4.

[0058] To summarize, according to the detection results, the associated task nodes connected to the target task node are determined among at least two subsequent task nodes, and the nodes in the subsequent task nodes that are not connected to the target task node do not need to be rolled back, thereby reducing the number of associated task nodes that need to perform task rollback operations, shortening the time for performing task rollback operations, and reducing the waste of computing resources.

[0059] Step 208: updating the task execution data in the metadata dimension, and performing a task rollback operation on the associated task node according to the updated task execution data.

[0060] Specifically, when the target task node in the task dependency graph satisfies the rollback condition, after the associated task nodes associated with the target task node are determined in the task dependency graph according to the task execution order, the task execution data can be updated in the metadata dimension, and the task rollback operation can be performed on the associated task nodes according to the updated task execution data, wherein updating the task execution data refers to processing the metadata related to the associated task nodes in the task execution data in the metadata dimension, which can be deleting or invalidating the metadata related to the associated task nodes in the task execution data. The task rollback operation refers to rolling back the initial task corresponding to the associated task node, that is, re-executing the initial task corresponding to the associated task node.

[0061] Based on this, when the target task node in the task dependency graph meets the rollback conditions, after determining the associated task nodes associated with the target task node in the task dependency graph according to the task execution order, the task execution data is updated in the metadata dimension, and the metadata related to the associated task nodes in the task execution data are deleted or invalidated in the metadata dimension, and the task rollback operation is performed on the associated task nodes according to the updated task execution data, and the initial task corresponding to the associated task nodes is re-executed.

[0062] Furthermore, considering that it is necessary to perform a task rollback operation on the associated task node, the task execution data includes the associated task data generated by the initial task execution corresponding to the associated task node. Therefore, it is necessary to update the associated task data so that the associated task node can perform the task rollback operation. The specific implementation is as follows: The associated task data corresponding to the associated task node is updated in the task execution data to obtain updated task execution data.

[0063] Specifically, the associated task data corresponding to the associated task node is the data generated when the initial task corresponding to the associated task node is executed. The associated task data may include metadata, task execution results, task execution status, and task submission time when the initial task corresponding to the associated task node is executed.

[0064] Based on this, the associated task data corresponding to the associated task node in the task execution data is updated in the metadata dimension to obtain the updated task execution data. The updated task execution data includes the target task node and the task execution data corresponding to the predecessor task node of the target task node. The predecessor task node of the target task node is the task node whose task submission time is before the task submission time of the target task node.

[0065] Furthermore, when the associated task data is updated, since the associated task node corresponding to the associated task data needs to perform a task rollback operation, the data in the associated task data related to the execution of the initial task corresponding to the associated task node are all useless data and can be deleted. The specific implementation is as follows: The associated task data corresponding to the associated task node is determined in the task execution data; the associated task execution data included in the associated task data is deleted, and the task status data included in the associated task data is updated.

[0066] Specifically, the associated task execution data contained in the associated task data is the associated task metadata generated by the execution of the initial task corresponding to the associated task node. The task status data contained in the associated task data is the execution status data recorded when the initial task corresponding to the associated task node is executed. The task execution status data can be updated to the task unfinished state.

[0067] Based on this, the associated task data corresponding to the associated task node is determined in the task execution data. The associated task execution data contained in the associated task data is deleted, and the associated task execution data may include the associated task metadata of the initial task corresponding to the associated task node, the task execution process, and the data corresponding to the task execution result. After deleting the associated task execution data contained in the associated task data, the task status data contained in the associated task data may be updated, and the task status may be updated to not executed.

[0068] Using the above example, determine the associated task data corresponding to the associated task nodes (task 7, task 9, task 8, and task 10) in the task execution data. Delete the associated task execution data corresponding to task 7, task 9, task 8, and task 10 in the associated task data. Update the task status data corresponding to task 7, task 9, task 8, and task 10 contained in the associated task data to an unfinished state.

[0069] In summary, deleting the associated task execution data contained in the associated task data can avoid the situation where useless associated task execution data still exists in the task execution data when the task rollback operation is performed on the associated task node, thereby avoiding a waste of storage space.

[0070] Furthermore, when performing a task rollback operation on an associated task node, it is necessary to pull branch task execution data of branch task nodes related to the task execution of the target task node in at least two subsequent task nodes, which is specifically implemented as follows: According to the detection result, a branch task node that is not connected to the target task node is determined in the at least two subsequent task nodes; branch task execution data corresponding to the branch task node is determined in the task execution data; and an associated task corresponding to the associated task node is executed by updating the updated task execution data in the metadata dimension based on the branch task execution data, thereby completing the task rollback operation of the associated task node.

[0071] Specifically, the branch task node is a task node determined based on the task dependency graph among at least two subsequent task nodes, which is not directly connected to the target task node, that is, there is no connecting edge between the branch task node and the target task node in the task dependency graph. The branch task execution data corresponding to the branch task node is the task execution data generated when the initial task corresponding to the branch task node is executed, including task execution metadata, task execution process, and task execution result corresponding data.

[0072] Based on this, a branch task node that is not connected to the target task node is determined in at least two subsequent task nodes according to the detection results, and the branch task execution data corresponding to the branch task node is determined in the task execution data. The branch task execution data corresponding to the branch task node will be pulled in the subsequent task rollback operation of the associated task node. By updating the updated task execution data in the metadata dimension based on the branch task execution data, the execution of the associated task corresponding to the associated task node can be completed, that is, the task rollback operation of the associated task node is completed. When executing the associated task corresponding to the associated task node, the branch task execution data corresponding to the branch task node is pulled and merged according to the node connection relationship in the task dependency graph to ensure that the task rollback operation of the associated task node is successfully completed while ensuring data accuracy.

[0073] Using the above example, Figure 4 As shown in (b), Task 6 and Task 12 in the task dependency graph are branch task nodes. When the associated task nodes (Task 7, Task 9, Task 8, and Task 10) perform a task rollback operation, the records submitted by the branch task nodes in the task execution data (branch task execution data) need to be merged into the rollback process of the associated task nodes to successfully complete the task rollback process.

[0074] To summarize, by updating the updated task execution data in the metadata dimension based on the branch task execution data, the associated tasks corresponding to the associated task nodes are executed, the task rollback operation of the associated task nodes is completed, and the accuracy and credibility of the task rollback of the associated task nodes are improved.

[0075] An embodiment of the present specification executes at least two initial tasks included in the target business by updating the metadata corresponding to the target business, and obtains task execution data. The execution of at least two initial tasks is achieved by updating the metadata, which can improve the efficiency of task execution. A task dependency graph of at least two initial tasks is constructed, and the task execution order corresponding to the task dependency graph is determined based on the task execution data. When the target task node in the task dependency graph meets the rollback condition, it indicates that the target node in the task dependency graph needs to perform a rollback operation. In the task dependency graph, the associated task nodes associated with the target task node are determined according to the task execution order, and the task execution data is updated in the metadata dimension. Performing a task rollback operation on the associated task nodes according to the updated task execution data can reduce the number of associated task nodes that need to perform the task rollback operation, shorten the time to perform the task rollback operation, and reduce the waste of computing resources.

[0076] The following combination Figure 5 Taking the application of the task rollback method provided in this specification in the rollback of liquidation business as an example, the task rollback method is further described. Figure 5 A process flow chart of a task rollback method provided in an embodiment of the present specification is shown, which specifically includes the following steps.

[0077] Step 502: Execute at least two tasks included in the target business by updating metadata corresponding to the target business, and obtain task execution data.

[0078] In the actual clearing business scenario, the target business is the clearing business. A "job" of the clearing business usually consists of multiple tasks, among which the execution of some tasks may depend on the output of multiple other previous tasks, and at the same time, some tasks may exist independently. When executing at least two tasks included in the target business, the tasks can be executed by modifying the metadata, that is, data modification and deletion are not directly modifying and deleting the original data, but writing metadata in an incremental modification mode to save the definition, structure, location and other information of these data, and obtain task execution data including task submission time, task data flow relationship and other data.

[0079] Step 504: Determine a task dependency relationship corresponding to at least two tasks based on the task execution data, construct a task node based on the at least two tasks, and construct a task dependency graph based on the task dependency relationship and the task nodes.

[0080] Based on the task execution data, the clearing business is subjected to dependency analysis to determine the task dependencies corresponding to at least two tasks. By analyzing the data flow relationship between tasks, the dependencies between tasks are accurately identified, and an intuitive directed acyclic graph, namely, the task dependency graph, is constructed. In the task dependency graph, the relationship between task nodes clearly expresses the data flow direction and execution order. Task nodes can be executed in parallel, but their execution must depend on the completion of the parent node connected to them. In the task dependency graph, there must be no data processing dependency between two "unreachable nodes". Through such dependency analysis, the correctness of the execution order and data flow of tasks can be ensured, while improving the efficiency of parallel execution and avoiding unnecessary resource competition and bottlenecks.

[0081] Step 506: Determine the task submission time corresponding to at least two graph nodes in the task dependency graph based on the task submission data corresponding to the task dependency graph, and determine the task execution order corresponding to the task dependency graph based on the task submission time of at least two graph nodes.

[0082] In actual applications, in order to ensure the traceability of tasks, it is necessary to use branch management for version control. Specifically, for the execution of each task, a new task branch is first pulled from the main branch, and relevant operations are performed on the task branch. After the task is executed, the task branch will be merged back to the main branch, and each task needs to save the current commit ID and time. Since the dependency relationship between tasks has been ensured to be clear and unambiguous in the dependency resolution phase, and there is no data processing dependency between the "unreachable nodes" in the graph, there will be no merge conflict when multiple "unreachable nodes" are merged back to the main branch after parallel execution. The branch merge operation is an update at the metadata level, so the execution speed is extremely fast and can usually be completed within seconds. Through this branch management method, the data lake can flexibly perform parallel processing and version control of tasks, thereby supporting efficient task scheduling and rollback operations. The task execution order corresponding to the task dependency graph can be determined according to the submission time of each task.

[0083] Step 508: Determine the target task node to be rolled back in the task dependency graph, and determine the subsequent task node of the target task node according to the task execution order.

[0084] Step 510: Perform node detection on the target task node and subsequent task nodes based on the task dependency graph, and determine the associated task nodes associated with the target task node according to the detection result.

[0085] The purpose of performing node detection on the target task node and the subsequent task nodes is to determine the reachable nodes and unreachable nodes of the target task node in the subsequent task nodes.

[0086] In actual applications, when the liquidation "job" is rolled back to the target task node, the tasks corresponding to the subsequent task nodes need to be re-executed. When re-executing, it is not necessary to re-execute all tasks in the "unfinished" state. The task flow can be finely reconstructed. All "reachable nodes" after the target task node have a dependency relationship with the target task node and need to be re-executed, while unreachable nodes do not need to be re-executed. It is only necessary to find the deepest node among the unreachable nodes of the target task node and merge the commit records of the branches where these nodes are located into the main branch. In this way, the task flow can be successfully completed by operating only the metadata without performing actual data calculations.

[0087] Step 512: Determine the associated task data corresponding to the associated task node in the task execution data, delete the associated task execution data included in the associated task data, and update the task status data included in the associated task data.

[0088] Step 514: Determine a branch task node that is not connected to the target task node in at least two subsequent task nodes according to the detection result, and determine branch task execution data corresponding to the branch task node in the task execution data.

[0089] Step 516: by updating the updated task execution data in the metadata dimension based on the branch task execution data, executing the associated task corresponding to the associated task node, and completing the task rollback operation of the associated task node.

[0090] When performing a task rollback operation, the reference pointer of the main branch can be switched to the commit ID position submitted by the task branch where the target task node is located, so as to achieve the rollback of the main branch. At the same time, all the commit records after the task branch of the target task node are deleted, and the task status corresponding to these deleted commit records is marked as unfinished, so that the status of the main branch is restored to the status after the target task node is executed. The above operation is completed by modifying the metadata and does not directly operate the data, so it can be completed within seconds.

[0091] In summary, the task rollback method provided in this embodiment realizes the accurate rollback and efficient management of complex liquidation task flows through the steps of dependency parsing, version control, branch rollback and task reconstruction. Based on the task dependency graph, the dependency relationship and execution status of the task are automatically parsed, and the scope of tasks that need to be rolled back and reconstructed are accurately identified, effectively avoiding the operation of irrelevant tasks in the traditional global rollback mode. By adjusting the reference pointer of the main branch instead of recalculating the underlying data, the rollback operation only involves metadata management. This method fully reuses the existing task status, quickly generates the required task flow, greatly improves the rollback efficiency, and significantly reduces the consumption of computing and storage resources. It supports the rollback and reconstruction of any task, providing a highly flexible solution for the dynamic adjustment of complex liquidation task flows.

[0092] Corresponding to the above method embodiment, this specification also provides a task rollback device embodiment, Figure 6 FIG. 1 shows a schematic diagram of a task rollback device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises: An execution module 602 is configured to execute at least two initial tasks included in the target business by updating metadata corresponding to the target business, and obtain task execution data; A construction module 604 is configured to construct a task dependency graph of the at least two initial tasks, and determine a task execution order corresponding to the task dependency graph based on the task execution data; A determination module 606 is configured to determine, in the task dependency graph, an associated task node associated with the target task node according to the task execution order when the target task node in the task dependency graph meets the rollback condition; The rollback module 608 is configured to update the task execution data in the metadata dimension, and perform a task rollback operation on the associated task node according to the updated task execution data.

[0093] In an optional embodiment, the determining module 606 is further configured to: Determine the subsequent task node of the target task node according to the task execution order in the task dependency graph; Based on the task dependency graph, node detection is performed on the target task node and the subsequent task node, and the associated task node associated with the target task node is determined according to the detection result.

[0094] In an optional embodiment, the determining module 606 is further configured to: Based on the task dependency graph, node connectivity detection is performed on the target task node and at least two subsequent task nodes respectively; The associated task node connected to the target task node is determined from among the at least two subsequent task nodes according to the detection result.

[0095] In an optional embodiment, the execution module 602 is further configured to: Determining a main task and a branch task from the at least two initial tasks included in the target business; The main task is executed by updating the metadata corresponding to the target business in the metadata dimension to obtain main task execution data; The branch task is executed by updating the main task execution data in the metadata dimension to obtain the task execution data.

[0096] In an optional embodiment, the construction module 604 is further configured to: Determining a task dependency relationship corresponding to the at least two initial tasks based on the task execution data; A task node is constructed based on the at least two initial tasks, and a task dependency graph of the at least two initial tasks is constructed based on the task dependency relationship and the task node.

[0097] In an optional embodiment, the construction module 604 is further configured to: Determining task submission data corresponding to the task dependency graph in the task execution data; The task submission times corresponding to at least two graph nodes in the task dependency graph are determined based on the task submission data, and the task execution order corresponding to the task dependency graph is determined based on the task submission times of the at least two graph nodes.

[0098] In an optional embodiment, the rollback module 608 is further configured to: The associated task data corresponding to the associated task node is updated in the task execution data to obtain updated task execution data.

[0099] In an optional embodiment, the rollback module 608 is further configured to: Determining the associated task data corresponding to the associated task node in the task execution data; The associated task execution data included in the associated task data is deleted, and the task status data included in the associated task data is updated.

[0100] In an optional embodiment, the rollback module 608 is further configured to: Determine, according to the detection result, a branch task node that is not connected to the target task node from among the at least two subsequent task nodes; Determining branch task execution data corresponding to the branch task node in the task execution data; The associated task corresponding to the associated task node is updated and executed by updating the updated task execution data in the metadata dimension based on the branch task execution data, thereby completing the task rollback operation of the associated task node.

[0101] An embodiment of the present specification executes at least two initial tasks included in the target business by updating the metadata corresponding to the target business, and obtains task execution data. The execution of at least two initial tasks is achieved by updating the metadata, which can improve the efficiency of task execution. A task dependency graph of at least two initial tasks is constructed, and the task execution order corresponding to the task dependency graph is determined based on the task execution data. When the target task node in the task dependency graph meets the rollback condition, it indicates that the target node in the task dependency graph needs to perform a rollback operation. In the task dependency graph, the associated task nodes associated with the target task node are determined according to the task execution order, and the task execution data is updated in the metadata dimension. Performing a task rollback operation on the associated task nodes according to the updated task execution data can reduce the number of associated task nodes that need to perform the task rollback operation, shorten the time to perform the task rollback operation, and reduce the waste of computing resources.

[0102] The above is a schematic scheme of a task rollback device of this embodiment. It should be noted that the technical scheme of the task rollback device and the technical scheme of the task rollback method described above are of the same concept, and the details of the technical scheme of the task rollback device that are not described in detail can be found in the description of the technical scheme of the task rollback method described above.

[0103] Figure 7 The block diagram of a computing device 700 according to an embodiment of the present specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0104] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0105] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0106] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 may also be a mobile or stationary server.

[0107] The processor 720 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned task rollback method.

[0108] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned task rollback method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned task rollback method.

[0109] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned task rollback method.

[0110] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the task rollback method described above are of the same concept, and the details not described in detail in the technical scheme of the storage medium can be found in the description of the technical scheme of the task rollback method described above.

[0111] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which implement the steps of the above-mentioned task rollback method when executed by a processor.

[0112] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the task rollback method described above are of the same concept, and the details not described in detail in the technical scheme of the computer program product can be found in the description of the technical scheme of the task rollback method described above.

[0113] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0114] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0115] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0116] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0117] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well.

Claims

1. A task rollback method, characterized in that: include: Execute at least two initial tasks included in the target business by updating metadata corresponding to the target business, and obtain task execution data; Constructing a task dependency graph of the at least two initial tasks, and determining a task execution order corresponding to the task dependency graph based on the task execution data; In a case where a target task node in the task dependency graph satisfies a rollback condition, determining, in the task dependency graph, associated task nodes associated with the target task node according to the task execution order; The task execution data is updated in the metadata dimension, and a task rollback operation is performed on the associated task node according to the updated task execution data.

2. The task rollback method according to claim 1, characterized in that: Determining the associated task nodes associated with the target task node according to the task execution order in the task dependency graph includes: Determine the subsequent task node of the target task node according to the task execution order in the task dependency graph; Based on the task dependency graph, node detection is performed on the target task node and the subsequent task node, and the associated task node associated with the target task node is determined according to the detection result.

3. The task rollback method according to claim 2, characterized in that: In the case where there are at least two subsequent task nodes, performing node detection on the target task node and the subsequent task node based on the task dependency graph, and determining the associated task node associated with the target task node according to the detection result, includes: Based on the task dependency graph, node connectivity detection is performed on the target task node and at least two subsequent task nodes respectively; The associated task node connected to the target task node is determined from among the at least two subsequent task nodes according to the detection result.

4. The task rollback method according to claim 1, characterized in that: The step of executing at least two initial tasks included in the target business by updating metadata corresponding to the target business to obtain task execution data includes: Determining a main task and a branch task from the at least two initial tasks included in the target business; The main task is executed by updating the metadata corresponding to the target business in the metadata dimension to obtain main task execution data; The branch task is executed by updating the main task execution data in the metadata dimension to obtain the task execution data.

5. The task rollback method according to claim 1, characterized in that: The step of constructing a task dependency graph of the at least two initial tasks includes: Determining a task dependency relationship corresponding to the at least two initial tasks based on the task execution data; A task node is constructed based on the at least two initial tasks, and a task dependency graph of the at least two initial tasks is constructed based on the task dependency relationship and the task node.

6. The task rollback method according to claim 1, characterized in that: The determining the task execution order corresponding to the task dependency graph based on the task execution data includes: Determining task submission data corresponding to the task dependency graph in the task execution data; The task submission times corresponding to at least two graph nodes in the task dependency graph are determined based on the task submission data, and the task execution order corresponding to the task dependency graph is determined based on the task submission times of the at least two graph nodes.

7. The task rollback method according to claim 1, characterized in that: The updating of the task execution data in the metadata dimension includes: The associated task data corresponding to the associated task node is updated in the task execution data to obtain updated task execution data.

8. The task rollback method according to claim 7, characterized in that: The updating of the associated task data corresponding to the associated task node in the task execution data includes: Determining the associated task data corresponding to the associated task node in the task execution data; The associated task execution data included in the associated task data is deleted, and the task status data included in the associated task data is updated.

9. The task rollback method according to claim 3, characterized in that: The performing a task rollback operation on the associated task node according to the updated task execution data includes: Determine, according to the detection result, a branch task node that is not connected to the target task node from among the at least two subsequent task nodes; Determining branch task execution data corresponding to the branch task node in the task execution data; The associated task corresponding to the associated task node is updated and executed by updating the updated task execution data in the metadata dimension based on the branch task execution data, thereby completing the task rollback operation of the associated task node.

10. A task rollback device, characterized in that: include: An execution module is configured to execute at least two initial tasks included in the target business by updating metadata corresponding to the target business, and obtain task execution data; A construction module is configured to construct a task dependency graph of the at least two initial tasks, and determine a task execution order corresponding to the task dependency graph based on the task execution data; A determination module is configured to determine, in the task dependency graph, an associated task node associated with the target task node according to the task execution order when the target task node in the task dependency graph meets the rollback condition; The rollback module is configured to update the task execution data in the metadata dimension, and perform a task rollback operation on the associated task node according to the updated task execution data.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the task rollback method described in any one of claims 1 to 9 are implemented.

12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the task rollback method described in any one of claims 1 to 9.

13. A computer program product, characterized in that It includes a computer program or an instruction, which, when executed by a processor, implements the steps of the task rollback method described in any one of claims 1 to 9.

Images