A data processing method and device, electronic equipment and storage medium

By merging blocking event information and directed acyclic graphs and performing unified recovery operations, the problem of blocking devices being unable to accurately determine data quality is solved, and the accuracy and consistency of task instance data generation are achieved.

CN114020507BActive Publication Date: 2026-01-20BEIJING DAJIA INTERNET INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202111273853.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-29
Publication Date
2026-01-20
Estimated Expiration
2042-01-20

AI Technical Summary

Technical Problem

The blocking device cannot accurately determine the current data quality problem of the task instance, causing downstream task instances to generate problematic data, which affects the accuracy of data processing.

Method used

By acquiring blocking information from multiple blocking events, merging it to generate target blocking information, determining the dependencies of task instances based on a directed acyclic graph, and executing a unified recovery operation to control the data generation process of task instances.

Benefits of technology

Accurately identify data quality issues for each task instance, improve the accuracy and effectiveness of data processing, and avoid inconsistencies in task instance states caused by multiple blocking events.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114020507B_ABST
    Figure CN114020507B_ABST
Patent Text Reader

Abstract

The present disclosure relates to a data processing method and device, electronic equipment and storage medium, and relates to the technical field of computer. The method comprises: obtaining blocking information of a plurality of blocking events, wherein the blocking information of a blocking event comprises the identification of at least two task instances corresponding to the blocking event and the dependency relationship between the at least two task instances; merging the blocking information of the plurality of blocking events to obtain target blocking information; performing a recovery operation on the target blocking event to cause the plurality of task instances to generate data according to the target blocking information. In the present disclosure, the electronic equipment can avoid the simultaneous execution of multiple blocking events causing the state of a same task instance to be different, and thus can accurately determine whether the data generated by each task instance has a quality problem, thereby improving the accuracy of data processing.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] At present, when upstream data of a task instance has a quality problem, a blocking device can perform a blocking operation on the task instance to prevent the task instance from generating problem data, and further prevent the problem data from spreading; when the upstream data is restored, the blocking device can also stop the blocking operation to make the task instance generate data again.

[0003] However, in the above method, the task instance can correspond to different projects. Specifically, for some projects, the task instance can be in the process of being blocked; and for other projects, the task instance can be in the process of generating data again. That is, the blocking device can not accurately determine whether the data currently corresponding to the task instance has a quality problem, so when there is a quality problem, the downstream task instance will continue to receive problem data, and further the downstream task instance will also generate problem data, affecting the accuracy of data processing. SUMMARY

[0004] The present disclosure provides a data processing method and device, electronic equipment and storage medium, which solves the technical problem that the blocking device can not accurately determine whether the data currently corresponding to the task instance has a quality problem, and further affects the generation of data by the downstream task instance.

[0005] The technical solutions of the embodiments of the present disclosure are as follows:

[0006] According to a first aspect of the embodiments of the present disclosure, a data processing method is provided. The method can include: obtaining blocking information of a plurality of blocking events, wherein the blocking information of one blocking event includes the identifiers of at least two task instances corresponding to the blocking event, and the dependency relationship between the at least two task instances; merging the blocking information of the plurality of blocking events to obtain target blocking information, the target blocking information being the blocking information of a target blocking event, the target blocking event corresponding to a plurality of task instances, the plurality of task instances including each task instance corresponding to the plurality of blocking events; and performing a recovery operation on the target blocking event to make the plurality of task instances generate data respectively according to the target blocking information.

[0007] Optionally, the data processing method further comprises: generating a directed acyclic graph corresponding to each of the plurality of blocking events based on the blocking information of the plurality of blocking events, wherein the directed acyclic graph corresponding to one blocking event comprises a task instance node and an edge, the task instance node is used to represent a task instance, and the edge is used to connect task instance nodes having a dependency relationship; and the merging the blocking information of the plurality of blocking events to obtain the target blocking information specifically comprises: merging the directed acyclic graphs corresponding to the plurality of blocking events to generate a target directed acyclic graph, the target directed acyclic graph is the directed acyclic graph corresponding to the target blocking event, and the target directed acyclic graph comprises the task instance nodes included in the directed acyclic graphs corresponding to the plurality of blocking events and the edges included in the directed acyclic graphs; and determining the target blocking information based on the target directed acyclic graph.

[0008] Optionally, the data processing method further comprises: executing the target blocking event; and when the target blocking information comprises an identifier of a to-be-identified task instance, controlling the to-be-identified task instance to pause generating data.

[0009] Optionally, the current task instance node is a root node in the target directed acyclic graph or a non-root node in the target directed acyclic graph, and the causing the plurality of task instances to generate data according to the target blocking information specifically comprises: when the current task instance node is the non-root node, determining a number of direct upstream successful nodes of the current task instance node, the direct upstream successful node is a node having a state of recovery success among direct upstream nodes of the current task instance node, and the direct upstream node is an upstream task instance node having a dependency relationship with the current task instance node; and when the number of direct upstream successful nodes is equal to the number of direct upstream nodes, determining that a current task instance starts to generate data, the current task instance being a task instance represented by the current task instance node.

[0010] Optionally, the data processing method further comprises: when the data generation of the current task instance is successful, updating a state of the current task instance to recovery success.

[0011] Optionally, the data processing method further comprises: storing the target blocking information in a database; and after the updating the state of the current task instance to recovery success, the data processing method further comprises: deleting the identifier of the current task instance from the database.

[0012] Optionally, the target blocking information includes priorities of the plurality of task instances, and the data processing method further includes: when the number of the direct upstream successful nodes is equal to the number of the direct upstream nodes and the direct upstream nodes are the same as the direct upstream nodes of the target task instance node, determining whether a priority of a target task instance is higher than a priority of the current task instance, where the target task instance node is a task instance node of the plurality of task instance nodes included in the target directed acyclic graph, except the current task instance node; the target task instance is a task instance represented by the target task instance node; and when the priority of the target task instance is higher than the priority of the current task instance, determining that the target task instance is preferentially started to generate data.

[0013] Optionally, the data processing method further includes: terminating the plurality of blocking events.

[0014] According to a second aspect of the embodiments of the present disclosure, a data processing apparatus is provided. The apparatus can include an obtaining module and a processing module. The obtaining module is configured to obtain blocking information of a plurality of blocking events, where the blocking information of one blocking event includes identification of at least two task instances corresponding to the blocking event and a dependency relationship between the at least two task instances. The processing module is configured to merge the blocking information of the plurality of blocking events to obtain target blocking information, where the target blocking information is blocking information of a target blocking event, the target blocking event corresponds to a plurality of task instances, and the plurality of task instances include each task instance corresponding to the plurality of blocking events. The processing module is further configured to perform a recovery operation on the target blocking event to enable the plurality of task instances to generate data respectively according to the target blocking information.

[0015] Optionally, the data processing apparatus further includes a determining module. The processing module is further configured to generate a directed acyclic graph corresponding to each of the plurality of blocking events based on the blocking information of the plurality of blocking events, where the directed acyclic graph corresponding to one blocking event includes a task instance node and an edge, the task instance node is used to represent a task instance, and the edge is used to connect task instance nodes having a dependency relationship. The processing module is specifically configured to merge the directed acyclic graphs corresponding to the plurality of blocking events to generate a target directed acyclic graph, where the target directed acyclic graph is a directed acyclic graph corresponding to the target blocking event, the target directed acyclic graph includes task instance nodes included in the directed acyclic graphs corresponding to the plurality of blocking events and edges included in the directed acyclic graphs. The determining module is configured to determine the target blocking information based on the target directed acyclic graph.

[0016] Optionally, the processing module is further configured to execute the target blocking event; and the processing module is further configured to control a to-be-identified task instance to suspend data generation when the target blocking information comprises an identifier of the to-be-identified task instance.

[0017] Optionally, the current task instance node is a root node in the target directed acyclic graph or a non-root node in the target directed acyclic graph; and the determining module is specifically configured to determine a number of direct upstream successful nodes of the current task instance node when the current task instance node is the non-root node, the direct upstream successful node being a node with a state of recovery success among direct upstream nodes of the current task instance node, and the direct upstream node being an upstream task instance node having a dependency relationship with the current task instance node; and the determining module is specifically further configured to determine the current task instance to start data generation when the number of the direct upstream successful nodes is equal to the number of the direct upstream nodes, the current task instance being a task instance represented by the current task instance node.

[0018] Optionally, the processing module is further configured to update a state of the current task instance to recovery success when the data generation of the current task instance is successful.

[0019] Optionally, the data processing apparatus further comprises a storage module and a deletion module; the storage module is configured to store the target blocking information into a database; and the deletion module is configured to delete the identifier of the current task instance from the database.

[0020] Optionally, the target blocking information comprises respective priorities of the plurality of task instances; the determining module is further configured to determine whether a priority of a target task instance is higher than a priority of the current task instance when the number of the direct upstream successful nodes is equal to the number of the direct upstream nodes and the direct upstream node is the same as a direct upstream node of a target task instance node, the target task instance node being a task instance node of the target directed acyclic graph except the current task instance node, the target task instance being a task instance represented by the target task instance node; and the determining module is further configured to determine the target task instance to start data generation preferentially when the priority of the target task instance is higher than the priority of the current task instance.

[0021] Optionally, the processing module is further configured to terminate the plurality of blocking events.

[0022] According to a third aspect of the embodiments of the present disclosure, an electronic device can include a processor and a memory configured to store processor-executable instructions; and the processor is configured to execute the instructions to implement any of the optional data processing methods in the first aspect.

[0023] According to a fourth aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores instructions. When the instructions in the computer readable storage medium are executed by an electronic device, the electronic device can perform any one of the optional data processing methods in the first aspect.

[0024] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, and the computer program product includes computer instructions. When the computer instructions are run on an electronic device, the electronic device performs any one of the optional data processing methods in the first aspect.

[0025] The embodiments of the present disclosure provide at least the following beneficial effects:

[0026] Based on any one of the above aspects, in the present disclosure, the electronic device can obtain blocking information of a plurality of blocking events, and combine the blocking information of the plurality of blocking events to obtain target blocking information. Since the target blocking information is the blocking information of a target blocking event, the target blocking event corresponds to a plurality of task instances, and each task instance corresponding to the plurality of blocking events is included in the plurality of task instances. Therefore, the target blocking information includes an identifier of each task instance and a dependency relationship between each task instance and other task instances. Then the electronic device can perform a recovery operation on the newly created target blocking event to respectively make the plurality of task instances generate data according to the target blocking information. In the embodiments of the present disclosure, the electronic device can perform a recovery operation on the target blocking event, that is, perform a data generation operation on the plurality of task instances to respectively make the plurality of task instances generate data. The data generation process of the plurality of task instances can be controlled based on a unified operation (that is, performing a recovery operation on the target blocking event), avoiding the state of a same task instance being different due to the plurality of blocking events being executed at the same time (or after being executed and recovered). Further, it can be accurately determined whether the data generated by each task instance has a quality problem, and the accuracy of data processing can be improved.

[0027] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0028] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the principles of the present disclosure, and do not constitute an undue limitation on the present disclosure.

[0029] Figure 1 A flowchart of a data processing method provided by the embodiments of the present disclosure is shown;

[0030] Figure 2 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0031] Figure 3 A schematic diagram of a directed acyclic graph combined by an embodiment of the present disclosure is shown;

[0032] Figure 4 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0033] Figure 5 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0034] Figure 6 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0035] Figure 7 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0036] Figure 8 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0037] Figure 9 A flowchart of another data processing method provided by an embodiment of the present disclosure is shown;

[0038] Figure 10 A structural schematic diagram of a data processing device provided by an embodiment of the present disclosure is shown;

[0039] Figure 11 A structural schematic diagram of another data processing device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0040] In order for those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be described clearly and completely below in conjunction with the drawings.

[0041] It should be noted that the terms "first", "second", and the like in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0042] It should also be understood that the word "comprising" is used herein to mean the presence of stated features, integers, steps, operations, elements, and / or components, but does not serve as a limitation to an open or inclusive meaning which permits additional features, integers, steps, operations, elements, components, and / or groups that are present in addition to those which are recited.

[0043] The data involved in the present disclosure can be data authorized by a user or sufficiently authorized by parties.

[0044] As described in the background, since the blocking device can not accurately determine whether the data currently corresponding to the task instance has a quality problem, thereby affecting the data generated by the downstream task instance. Based on this, the embodiment of the present disclosure provides a data processing method, which can control the data generation process of multiple task instances based on unified operations (i.e., performing recovery operations on target blocking events), avoid the state of a same task instance being different when multiple blocking events are executed at the same time (or after being executed recovery operations). In turn, it can accurately determine whether the data generated by each task instance has a quality problem, and can improve the accuracy of data processing.

[0045] The data processing method, device, electronic device and storage medium provided by the embodiment of the present disclosure are applied to the scene of generating (or recovering) data of a task instance. When the electronic device obtains blocking information of multiple blocking events, the method provided by the embodiment of the present disclosure can be used to make multiple task instances corresponding to the blocking event generate data.

[0046] The data processing method provided by the embodiment of the present disclosure is exemplarily described below in combination with the drawings:

[0047] It can be understood that the electronic device executing the data processing method provided by the embodiment of the present disclosure can be a mobile phone, a tablet computer, a desktop computer, a laptop computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) \ virtual reality (VR) device, and the like, which can install and use a content community application. The present disclosure does not specially limit the specific form of the electronic device. It can perform human-computer interaction with the user through one or more ways such as a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, or a handwriting device.

[0048] As shown in FIG. 1, Figure 1 The data processing method provided by the embodiment of the present disclosure can include S101-S103.

[0049] S101, the electronic device obtains blocking information of a plurality of blocking events.

[0050] The blocking information of one blocking event includes identification of at least two task instances corresponding to the blocking event, and a dependency relationship between the at least two task instances.

[0051] It should be understood that the relationship between a task and a task instance is similar to the relationship between a program and a process. For a task, each execution of the task generates a task instance, and different task instances produce different data partitions, which can be understood as partitions storing data generated (or obtained after operation) by the task instance.

[0052] It can be understood that the dependency relationship between the at least two task instances is used to represent the upstream and downstream relationship between the at least two task instances. For example, if the first task instance is an upstream task instance of the second task instance (the at least two task instances include the first task instance and the second task instance), the electronic device can determine that there is a dependency relationship between the first task instance and the second task instance.

[0053] In an implementation manner of the embodiment of the present disclosure, the electronic device can obtain the identification of the plurality of blocking events, and then obtain the blocking information of the plurality of blocking events based on the identification of the plurality of blocking events.

[0054] It should be noted that the task instance corresponding to each of the plurality of blocking events can be understood as a blocking instance. That is, if the each of the plurality of blocking events is executed, the task instance corresponding to the each of the plurality of blocking events will be blocked, specifically, the task instance corresponding to the each of the plurality of blocking events is blocked from generating data; if the each of the plurality of blocking events is stopped, the task instance corresponding to the each of the plurality of blocking events is stopped from being blocked, specifically, the task instance corresponding to the each of the plurality of blocking events is enabled to generate data.

[0055] Optionally, the identification of one task instance can be a primary key of the task instance, and the primary key can be a combination of a name of a task corresponding to the task instance and a generation time of the task instance.

[0056] S102, the electronic device merges the blocking information of the plurality of blocking events to obtain target blocking information.

[0057] The target blocking information is blocking information of a target blocking event, the target blocking event corresponds to a plurality of task instances, and the plurality of task instances include each task instance corresponding to the plurality of blocking events.

[0058] It should be understood that the target blocking event can be a new blocking event created by the electronic device. The target blocking event can represent the plurality of blocking events at the same time. Specifically, the target blocking information includes the blocking information of each of the plurality of blocking events.

[0059] In an implementation of the embodiments of the present disclosure, when any two of the plurality of blocking events correspond to the same task instance, the electronic device can perform a merging operation on the blocking information of the plurality of blocking events.

[0060] S103, the electronic device performs a recovery operation on the target blocking event to make the plurality of task instances generate data according to the target blocking information.

[0061] It can be understood that the electronic device performs a recovery operation on the target blocking event, which is used to instruct the plurality of task instances corresponding to the target blocking event to perform a recovery operation, so as to make the plurality of task instances generate data respectively.

[0062] It should be understood that the electronic device performs a recovery operation on the target blocking event to make a certain task instance in the plurality of task instances generate data can include the following two cases:

[0063] In one case, when the task instance has not generated data before, the electronic device performs a recovery operation on the target blocking event, which is used to instruct the task instance to generate data for the first time.

[0064] In another case, when the task instance has generated data before, the electronic device performs a recovery operation on the target blocking event, which is used to instruct the task instance to regenerate data, and it can also be understood as instructing the task instance to perform a re-run recovery operation.

[0065] It should be noted that the above S103 is performed under the condition that the upstream data recovery corresponding to the target blocking event is determined. That is, when the upstream data recovery is performed, the electronic device performs a recovery operation on the target blocking event.

[0066] The technical solutions provided by the above embodiments can bring at least the following beneficial effects: as can be seen from S101-S103, the electronic device can obtain the blocking information of a plurality of blocking events, and combine the blocking information of the plurality of blocking events to obtain target blocking information. Since the target blocking information is the blocking information of a target blocking event, the target blocking event corresponds to a plurality of task instances, and each task instance corresponding to the plurality of blocking events is included in the plurality of task instances. Therefore, the target blocking information includes the identifier of each task instance and the dependency relationship between each task instance and other task instances. Then the electronic device can perform a recovery operation on the newly created target blocking event to respectively make the plurality of task instances generate data according to the target blocking information. In the embodiments of the present disclosure, the electronic device can perform a recovery operation on the target blocking event, that is, perform a data generation operation on the plurality of task instances to respectively make the plurality of task instances generate data, which can control the data generation process of the plurality of task instances based on a unified operation (that is, perform a recovery operation on the target blocking event), and avoid the state of a same task instance being different due to the plurality of blocking events being simultaneously executed (or after being executed). Further, it can accurately determine whether the data generated by each task instance has a quality problem, and can improve the accuracy of data processing.

[0067] In combination Figure 1 As Figure 2 indicated, the data processing method provided by the embodiments of the present disclosure can further include S104.

[0068] S104, the electronic device generates a directed acyclic graph corresponding to each of the plurality of blocking events based on the blocking information of the plurality of blocking events.

[0069] The directed acyclic graph corresponding to each of the plurality of blocking events includes a task instance node and an edge, the task instance node is used to represent a task instance, and the edge is used to represent task instance nodes having a dependency relationship.

[0070] In combination with the description of the above embodiments, it should be understood that the dependency relationship is used to represent the upstream and downstream relationship between the task instance nodes. Specifically, one task instance node (i.e., the end point of the edge or the end point of the arrow) depends on another task instance node (i.e., the start point of the edge or the start point of the arrow), that is, the task instance node is a (direct) downstream task instance node of the another task instance node.

[0071] Continuing as Figure 2 indicated, the combining of the blocking information of the plurality of blocking events to obtain the target blocking information can specifically include S1021-S1022.

[0072] S1021, the electronic device combines the directed acyclic graphs of the plurality of blocking events to generate a target directed acyclic graph.

[0073] The target DAG is a DAG corresponding to the target blocking event, and the target DAG includes task instance nodes included in the DAGs corresponding to the plurality of blocking events and edges included in the DAGs corresponding to the plurality of blocking events.

[0074] In an embodiment of the present disclosure, the electronic device can create a new graph, and then add the task instance nodes included in the DAGs corresponding to the plurality of blocking events and the edges included in the DAGs corresponding to the plurality of blocking events to the new graph to generate the target DAG.

[0075] For example, Figure 3 For an example in which the electronic device combines two DAGs (including the DAG 101 and the DAG 102) into a target DAG (i.e., the DAG 103) in an embodiment of the present disclosure, specifically Figure 3 One letter in the DAG 101 represents one task instance node, for example, the letter A represents the task instance node A.

[0076] As shown in FIG. 1, Figure 3 The DAG 101 includes four task instance nodes (i.e., the task instance node A, the task instance node B, the task instance node D, and the task instance node E) and three edges, and the three edges are respectively used to connect the task instance node A and the task instance node B, the task instance node B and the task instance node D, and the task instance node B and the task instance node E. The DAG 102 includes five task instance nodes (i.e., the task instance node C, the task instance node E, the task instance node F, the task instance node G, and the task instance node H) and four edges, and the four edges are respectively used to connect the task instance node C and the task instance node E, the task instance node C and the task instance node F, the task instance node C and the task instance node G, and the task instance node E and the task instance node F.

[0077] As can be seen from the DAG 103 in FIG. 1, Figure 3 The DAG 103 includes the four task instance nodes and the three edges, and also includes the five task instance nodes and the four edges. That is, the target DAG includes each task instance node included in the two DAGs and each edge included in the two DAGs.

[0078] S1022、The electronic device determines target blocking information based on the target DAG.

[0079] In combination with the description of the above embodiments, it should be understood that the target DAG includes the task instance nodes included in the DAG corresponding to each of the above plurality of blocking events and the edges included in the DAG. In this way, the electronic device can determine the identity of each of the above plurality of task instances and the dependency relationship between each of the task instances and other task instances based on the task instance nodes included in the DAG and the edges included in the DAG, i.e., determine the target blocking information.

[0080] The technical solutions provided by the above embodiments can at least bring the following beneficial effects: as can be seen from S104, S1021-S1022, the electronic device can generate a DAG corresponding to each of the plurality of blocking events based on the blocking information of the plurality of blocking events; then the electronic device merges the DAGs corresponding to each of the plurality of blocking events to generate a target DAG, and determines target blocking information based on the target DAG. In the embodiments of the present disclosure, since the target DAG includes the task instance nodes included in the DAG corresponding to each of the plurality of blocking events and the edges included in the DAG, the electronic device can determine the target blocking information based on the task instance nodes included in the DAG and the edges included in the DAG. The merged blocking information can be accurately and reasonably determined, and the unified processing of the plurality of blocking events can be completed.

[0081] In combination with Figure 1 As Figure 4 indicated, the data processing method provided by the embodiments of the present disclosure further includes S105-S106.

[0082] S105, the electronic device executes a target blocking event.

[0083] It should be understood that the electronic device executing the target blocking event means performing a blocking operation on the target blocking event, specifically, blocking the generation of data for each of the plurality of task instances corresponding to the target blocking event.

[0084] In one implementation form of the embodiments of the present disclosure, when the electronic device executes the target blocking event, the electronic device can update the state of the target blocking event to blocking. Similarly, when the electronic device performs a recovery operation on the target blocking event (i.e., executes S103 described above), the electronic device can update the state of the target blocking event to recovery.

[0085] S106, when the target blocking information includes the identity of a to-be-identified task instance, the electronic device controls the to-be-identified task instance to pause generating data.

[0086] It is understandable that the task instance to be identified can be one of the multiple task instances mentioned above, or it can be a task instance other than the multiple task instances mentioned above.

[0087] When the target blocking information includes the identifier of the task instance to be identified, it indicates that the task instance to be identified is one of the multiple task instances. In this way, the electronic device can block the data generation of the task instance to be identified, that is, control the task instance to pause data generation.

[0088] When the target blocking information does not include the identifier of the task instance to be identified, it indicates that the task instance to be identified is a task instance other than the multiple task instances. In this case, the electronic device can control the task instance to be identified to start generating (or continue generating) data.

[0089] The technical solution provided by the above embodiments can bring at least the following beneficial effects: As shown in S105-S106, the electronic device can execute a target blocking event, and when the target blocking information includes the identifier of the task instance to be identified, it controls the task instance to be identified to pause data generation. In this embodiment of the present disclosure, when the target blocking information includes the identifier of the task instance to be identified, it indicates that the task instance to be identified is one of multiple task instances corresponding to the target blocking event. In this way, the electronic device can block the data generation of the task instance to be identified, that is, control the task instance to be identified to pause data generation. It can block each of the multiple task instances based on a unified standard (i.e., the target blocking event), thereby improving the effectiveness of data processing.

[0090] Combination Figure 2 ,like Figure 5 As shown, in one implementation of this embodiment, the current task instance node is the root node in the target directed acyclic graph or a non-root node in the target directed acyclic graph. The above-mentioned generation of data by multiple task instances according to the target blocking information may specifically include S1031-S1032.

[0091] S1031. When the current task instance node is a non-root node, the electronic device determines the number of successful upstream nodes of the current task instance node.

[0092] Among them, the directly upstream successful node is the node whose status is successfully recovered among the directly upstream nodes of the current task instance node, and the directly upstream node is the upstream task instance node that has a dependency relationship with the current task instance node.

[0093] It should be understood that when the current task instance node is a non-root node, it means that there is another task instance node upstream of the current task instance node, and the other task instance node directly (or indirectly) has a dependency relationship with the current task instance node. The upstream task instance node described above which has a dependency relationship with the current task instance node is the upstream node which has an edge between the current task instance node.

[0094] It can be understood that the electronic device can obtain the state of each of the plurality of task instance nodes (i.e., the nodes represented by the plurality of task instance nodes described above), i.e., the current state of each of the plurality of task instance nodes. When the state is recovery success, it means that the data generated by the task instance node does not have a quality problem, so that the downstream task instance node of the task instance node has the opportunity to start generating data.

[0095] S1032, when the number of directly upstream successful nodes is equal to the number of directly upstream nodes, the electronic device determines that the current task instance starts to generate data.

[0096] The current task instance is the task instance represented by the current task instance node.

[0097] It should be understood that when the number of directly upstream successful nodes is equal to the number of directly upstream nodes, it means that the data generated by all directly upstream task instance nodes of the current task instance node does not have a quality problem, at which time the electronic device can determine that the current task instance can start to generate data. Otherwise, i.e., when the number of upstream directly successful nodes is not equal to the number of directly upstream nodes, it means that among all the directly upstream task instance nodes, there are still some task instance nodes that generate data with quality problems, so the current task instance cannot start to generate data.

[0098] For example, in combination with the example in the above Figure 3 , it is assumed that the current task instance node is task instance node E in the directed acyclic graph 103, and when the number of directly upstream successful nodes of the task instance node E is equal to 2 (the directly upstream nodes of the task instance node E are task instance node B and task instance node C), the electronic device determines that the task instance represented by the task instance node E starts to generate data.

[0099] In an implementation manner of the embodiment of the present disclosure, when the current task instance node is the root node described above, it means that there is no blocking instance upstream of the current task instance node (i.e., there is only a blocking instance downstream of it). In the case that the upstream data corresponding to the target blocking event is recovered successfully (or in the case that the input data of the current task instance node does not have a quality problem), the electronic device can determine that the current task instance starts to generate data.

[0100] The technical solutions provided by the above embodiments can bring at least the following beneficial effects: as can be seen from S1031-S1032, when the current task instance node is a non-root node, the electronic device can determine the number of direct upstream successful nodes of the current task instance node, and when the number of direct upstream successful nodes is equal to the number of direct upstream nodes, the electronic device can determine that the current task instance starts to generate data. In the embodiments of the present disclosure, when the number of direct upstream successful nodes is equal to the number of direct upstream nodes, it indicates that the data generated by all direct upstream task instance nodes of the current task instance node does not have quality problems, and thus the electronic device can determine that the current task instance starts to generate data. Whether each of the plurality of task instances can start to generate data can be accurately and effectively determined, and accurate data generated by downstream task instances is ensured.

[0101] In combination Figure 5 As Figure 6 indicated, the data processing method provided by the embodiments of the present disclosure further includes S107.

[0102] S107, when the data generation of the current task instance is successful, the electronic device updates the state of the current task instance to recovery success.

[0103] It should be understood that when a task instance (for example, the current task instance) starts to generate data, the state of the task instance can be (or can be updated to be) running, which means that the task instance is generating data (or is resuming the data generation process).

[0104] When the data generation of the current task instance is successful, it indicates that the data generated by the current task instance does not have quality problems, and thus the electronic device can update the state of the current task instance to recovery success.

[0105] Optionally, when a task instance is blocked from generating data, the state of the task instance is blocked. When the data generation of the task instance fails, the state of the task instance can be recovery failure. At this time, the electronic device can instruct the task instance to continue to generate data until correct (i.e., without quality problems) data is generated.

[0106] The technical solutions provided by the above embodiments can bring at least the following beneficial effects: as can be seen from S107, when the data generation of the current task instance is successful, it indicates that the data generated by the current task instance does not have quality problems, and thus the electronic device can update the state of the current task instance to recovery success. The electronic device can accurately determine the current state of each of the plurality of task instances, and thus can accurately determine whether each of the plurality of task instances can start to generate data.

[0107] In combinationFigure 6 As shown in Figure 7 S108, the electronic device stores the target blocking information into a database.

[0108] S108, the electronic device stores the target blocking information into a database.

[0109] It should be understood that the electronic device stores the target blocking information into the database, that is, stores the identification of each task instance in the plurality of task instances and the dependency relationship between each task instance and other task instances into the database.

[0110] As shown in Figure 7 S109, the electronic device deletes the identification of the current task instance from the database.

[0111] S109, the electronic device deletes the identification of the current task instance from the database.

[0112] It can be understood that when the state of the current task instance is updated to recovery success, it indicates that the data generated by the current task instance does not have quality problems and the data has been generated successfully. Therefore, the electronic device can delete the identification of the current task instance from the database, so that the electronic device does not need to block the current task instance.

[0113] Optionally, the electronic device can periodically schedule the identifications of the various task instances currently stored in the database to block the various task instances from generating data.

[0114] The technical solutions provided by the above embodiments can at least bring the following beneficial effects: as can be known from S108-S109, the electronic device can store the target blocking information into the database, and after updating the state of the current task instance to recovery success, the electronic device can also delete the identification of the current task instance from the database. In the embodiments of the present disclosure, when the state of the current task instance is updated to recovery success, it indicates that the data generated by the current task instance does not have quality problems and the data has been generated successfully. Therefore, the electronic device can delete the identification of the current task instance from the database, so that the electronic device does not need to block the current task instance, which can improve the blocking accuracy of each task instance in the plurality of task instances and reduce the probability of obtaining problem data by a downstream task instance.

[0115] As shown in Figure 5 As shown in Figure 8 In an implementation manner of the embodiments of the present disclosure, the target blocking information includes the priority of each task instance in the plurality of task instances, and the data processing method provided by the embodiments of the present disclosure further includes S110-S111.

[0116] S110, when the number of direct upstream successful nodes is equal to the number of direct upstream nodes, and the direct upstream node is the same as the direct upstream node of the target task instance node, the electronic device determines whether the priority of the target task instance is higher than the priority of the current task instance.

[0117] The target task instance node is a task instance node in the plurality of task instance nodes included in the target directed acyclic graph, except the current task instance node, and the target task instance is a task instance represented by the target task instance node.

[0118] It should be understood that when the direct upstream node (of the current task instance node) is the same as the direct upstream node of the target task instance node, it means that the target task instance node has the same direct upstream node as the current task instance node, or it can be understood that the target task instance node has the same upstream task instance node with the same dependency relationship as the current task instance node.

[0119] It can be understood that different tasks can correspond to different priorities, and the priority of a task can be understood as the attribute information of the task. Different task instances generated by executing the same task have the same priority, and the priority of the different task (i.e., the priority of each task instance) can be pre-configured.

[0120] S111, when the priority of the target task instance is higher than the priority of the current task instance, the electronic device determines that the target task instance starts to generate data in priority.

[0121] It should be understood that in the process of starting data generation of each task instance, the resources that can be allocated by the electronic device can be limited, at which time the task instance with higher priority should be processed in priority, i.e., the task instance with higher priority starts to generate data in priority.

[0122] Alternatively, when the priority of the current task instance is higher than the priority of the target task instance, the electronic device can determine that the current task instance starts to generate data in priority, i.e., the current task instance is processed in priority.

[0123] For example, in combination with the above Figure 3 For example, it is assumed that the current task instance is a task instance represented by the task instance node F included in the directed acyclic graph 103, and the target task instance is a task instance represented by the task instance node G included in the directed acyclic graph 103. Since the priority of the task instance represented by the task instance node G is higher than the priority of the task instance represented by the task instance node F, the electronic device determines that the task instance represented by the task instance node G starts to generate data in priority.

[0124] The technical solutions provided by the above embodiments can bring at least the following beneficial effects: as can be known from S110-S111, when the number of the direct upstream successful nodes of the current task instance node is equal to the number of the direct upstream nodes of the current task instance node, and the direct upstream nodes are the same as the direct upstream nodes of the target task instance node, it is indicated that the target task instance node and the current task instance node have the same direct upstream nodes and the data of the same direct upstream nodes does not have a quality problem (or is all recovered), and at this time, the electronic device can determine whether the priority of the target task instance is higher than the priority of the current task instance. When the priority of the target task instance is higher than the priority of the current task instance, it is indicated that the priority of the target task instance is higher, and thus the electronic device can determine that the target task instance starts to generate data in priority. In the embodiments of the present disclosure, when two task instance nodes have the same direct upstream nodes, and the number of the direct upstream successful nodes of one of the task instance nodes is equal to the number of the direct upstream nodes of the task instance node, the electronic device can determine the respective priorities of the two task instances, and preferentially processes the task instance with higher priority, so that the processing order of the multiple task instances can be reasonably determined, and the effectiveness of data processing is improved.

[0125] In combination Figure 1 As Figure 9 indicated, the data processing method provided by the embodiments of the present disclosure further includes S112.

[0126] In S112, the electronic device terminates the multiple blocking events.

[0127] It should be understood that each of the multiple blocking events described above can be being executed (i.e., at least two task instances corresponding to each of the blocking events pause data generation), and the state of each of the blocking events can also be updated to be in recovery (i.e., at least two task instances corresponding to each of the blocking events start to generate data). In the embodiments of the present disclosure, the electronic device terminates the multiple blocking events to terminate the pause (or start) of data generation of each of the task instances corresponding to the multiple blocking events, so that the electronic device can only execute the target blocking event (or perform a recovery operation on the target blocking event) and not execute the multiple blocking events, that is, the data generation process of the multiple task instances only depends on the execution operation of the target blocking event, rather than executing the multiple blocking events (or performing a recovery operation on the multiple blocking events) to make one of the multiple task instances generate or pause to generate related data multiple times, so that the efficiency of data processing can be improved. Moreover, based on the multiple blocking events blocking the same task instance multiple times or recovering the same task instance multiple times, a large amount of resources will be consumed. Therefore, based on the unified operation of making the multiple task instances generate data, the energy consumption can be reduced and the running speed of the electronic device can be improved.

[0128] It should be noted that the execution order of S103 and S113 is not limited in the embodiments of the present disclosure. For example, S103 can be executed first and then S113, or S113 can be executed first and then S103, or S103 and S113 can be executed simultaneously. For the convenience of description, Figure 9 In the embodiments of the present disclosure, S103 is executed first and then S113 is executed.

[0129] It can be understood that in actual implementation, the electronic device described in the embodiments of the present disclosure can include one or more hardware structures and / or software modules for implementing the foregoing corresponding data processing method, and these execution hardware structures and / or software modules can constitute an electronic device. Those skilled in the art should easily realize that, in combination with the algorithm steps of the examples described in the embodiments disclosed herein, the present disclosure can be realized in the form of hardware or a combination of hardware and computer software. Whether a certain function is realized in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present disclosure.

[0130] Based on such understanding, the embodiments of the present disclosure also correspondingly provide a data processing apparatus, Figure 10 The structure schematic diagram of the data processing apparatus provided by the embodiments of the present disclosure is shown. As Figure 10 shown, the data processing apparatus 20 can include an acquisition module 201 and a processing module 202.

[0131] The acquisition module 201 is configured to acquire blocking information of a plurality of blocking events, wherein the blocking information of one blocking event includes the identification of at least two task instances corresponding to the blocking event and the dependency relationship between the at least two task instances.

[0132] The processing module 202 is configured to merge the blocking information of the plurality of blocking events to obtain target blocking information, the target blocking information being the blocking information of a target blocking event, the target blocking event corresponding to a plurality of task instances, the plurality of task instances including each task instance corresponding to the plurality of blocking events.

[0133] The processing module 202 is further configured to perform a recovery operation on the target blocking event to make the plurality of task instances generate data respectively according to the target blocking information.

[0134] Optionally, the data processing apparatus 20 further includes a determination module 203.

[0135] The processing module 202 is further configured to generate a directed acyclic graph corresponding to each of the plurality of blocking events based on the blocking information of the plurality of blocking events, wherein the directed acyclic graph corresponding to one blocking event comprises task instance nodes and edges, the task instance nodes are used to represent task instances, and the edges are used to connect task instance nodes having a dependency relationship.

[0136] The processing module 202 is specifically configured to merge the directed acyclic graphs corresponding to the plurality of blocking events to generate a target directed acyclic graph, the target directed acyclic graph is the directed acyclic graph corresponding to the target blocking event, and the target directed acyclic graph comprises the task instance nodes comprised by the directed acyclic graphs corresponding to the plurality of blocking events and the edges comprised by the directed acyclic graphs.

[0137] The determining module 203 is configured to determine the target blocking information based on the target directed acyclic graph.

[0138] Optionally, the processing module 202 is further configured to execute the target blocking event.

[0139] The processing module 202 is further configured to control a to-be-identified task instance to suspend data generation when the target blocking information comprises an identifier of the to-be-identified task instance.

[0140] Optionally, the current task instance node is a root node in the target directed acyclic graph or a non-root node in the target directed acyclic graph.

[0141] The determining module 203 is specifically configured to determine a number of direct upstream successful nodes of the current task instance node when the current task instance node is the non-root node, the direct upstream successful node is a node having a state of recovery success among direct upstream nodes of the current task instance node, and the direct upstream node is an upstream task instance node having a dependency relationship with the current task instance node.

[0142] The determining module 203 is further specifically configured to determine that a current task instance starts to generate data when the number of direct upstream successful nodes is equal to the number of direct upstream nodes, and the current task instance is a task instance represented by the current task instance node.

[0143] Optionally, the processing module 202 is further configured to update a state of the current task instance to recovery success when the data generation of the current task instance is successful.

[0144] Optionally, the data processing apparatus 20 further comprises a storage module 204 and a deletion module 205.

[0145] The storage module 204 is configured to store the target blocking information into a database.

[0146] The deleting module 205 is configured to delete the identification of the current task instance from the database.

[0147] Optionally, the target blocking information includes a priority of each of the plurality of task instances.

[0148] The determining module 203 is further configured to determine whether a priority of a target task instance is higher than a priority of the current task instance, when the number of the direct upstream successful nodes is equal to the number of the direct upstream nodes, and the direct upstream nodes are the same as direct upstream nodes of a target task instance node, wherein the target task instance node is a task instance node of the plurality of task instance nodes included in the target directed acyclic graph, except the current task instance node; the target task instance is a task instance represented by the target task instance node.

[0149] The determining module 203 is further configured to determine that the target task instance is to be preferentially started to generate data, when the priority of the target task instance is higher than the priority of the current task instance.

[0150] Optionally, the processing module 202 is further configured to terminate the plurality of blocking events.

[0151] As described above, the embodiments of the present disclosure can divide the functional modules of the data processing apparatus according to the method examples described above. The integrated modules described above can be implemented in the form of hardware or in the form of software functional modules. In addition, it should be noted that the division of modules in the embodiments of the present disclosure is illustrative, and is only a logical functional division. Actual implementation can have another division manner. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module.

[0152] As to the data processing apparatus in the above embodiments, the specific manner in which each module performs operations and the beneficial effects thereof have been described in detail in the foregoing method embodiments, which will not be described here.

[0153] Figure 11 is a structural schematic diagram of another data processing apparatus provided by the present disclosure. As shown in Figure 11 The data processing apparatus 30 can include at least one processor 301 and a memory 303 for storing processor-executable instructions. The processor 301 is configured to execute the instructions in the memory 303 to implement the data processing method in the above embodiments.

[0154] In addition, the data processing apparatus 30 can further include a communication bus 302 and at least one communication interface 304.

[0155] The processor 301 can be a central processing unit (CPU), a micro-processing unit, an ASIC, or one or more integrated circuits for controlling the execution of programs of the present disclosure.

[0156] The communication bus 302 can include a path that carries information between the aforementioned components.

[0157] The communication interface 304, using any transceiver-like device, can be used to communicate with other devices or computer networks, such as an Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0158] The memory 303 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but not limited to. The memory can exist independently, connected to the processing unit through a bus. The memory can also be integrated with the processing unit.

[0159] The memory 303 is configured to store instructions for implementing the present disclosure, and the processor 301 is configured to control the execution of the instructions. The processor 301 is configured to execute the instructions stored in the memory 303, thereby realizing the functions in the method of the present disclosure.

[0160] In a specific implementation, as an embodiment, the processor 301 can include one or more CPUs, such as CPU0 and CPU1 in Figure 11 .

[0161] In a specific implementation, as an embodiment, the data processing apparatus 30 can include multiple processors, such as Figure 11The data processing apparatus 30 can include a processor 301 and a processor 307. Each of the processors can be a single-CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).

[0162] In a specific implementation, as an embodiment, the data processing apparatus 30 can further include an output device 305 and an input device 306. The output device 305 communicates with the processor 301 and can display information in various ways. For example, the output device 305 can be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, a projector, or the like. The input device 306 communicates with the processor 301 and can accept user input in various ways. For example, the input device 306 can be a mouse, a keyboard, a touch screen device, a sensor device, or the like.

[0163] Those skilled in the art can understand that the structure shown in the above is not a limitation on the data processing apparatus 30, and the data processing apparatus 30 can include more or fewer components than shown, or combine certain components, or use different component arrangements. Figure 11

[0164] In addition, the present disclosure also provides a computer readable storage medium, including instructions, when the instructions are executed by a processor, the processor executes the data processing method provided by the above embodiments.

[0165] In addition, the present disclosure also provides a computer program product, including instructions, when the instructions are executed by a processor, the processor executes the data processing method provided by the above embodiments.

[0166] Other embodiments of the present disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the aspects of the present disclosure disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure following the general principles thereof and including such departures from the present disclosure that come within known, accepted, or customary practice in the art to which the present disclosure pertains. The specification and examples are to be regarded as exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.​

Claims

1. A data processing method, characterized in that, include: Obtain blocking information for multiple blocking events, wherein the blocking information for a blocking event includes the identifiers of at least two task instances corresponding to the blocking event, and the dependency relationship between the at least two task instances, and the blocking event is used to block the corresponding task instance from generating data; Based on the blocking information of the multiple blocking events, a directed acyclic graph corresponding to each of the multiple blocking events is generated. The directed acyclic graph corresponding to a blocking event includes task instance nodes and edges. The task instance nodes are used to represent task instances, and the edges are used to connect task instance nodes with dependencies. Merge the directed acyclic graphs corresponding to the multiple blocking events to generate a target directed acyclic graph. The target directed acyclic graph is the directed acyclic graph corresponding to the target blocking event. The target directed acyclic graph includes the task instance nodes included in the directed acyclic graphs corresponding to the multiple blocking events and the edges included in the directed acyclic graphs. The target blocking information is determined based on the target directed acyclic graph. The target blocking information is the blocking information of the target blocking event. The target blocking event corresponds to multiple task instances. The multiple task instances include each task instance corresponding to the multiple blocking events. Execute the target blocking event; When the target blocking information includes the identifier of the task instance to be identified, the task instance to be identified is controlled to pause data generation; the current task instance node is the root node in the target directed acyclic graph or a non-root node in the target directed acyclic graph. Perform a recovery operation on the target blocking event. When the current task instance node is the non-root node, determine the number of successful upstream nodes of the current task instance node. The successful upstream nodes are the nodes in the state of successful recovery among the direct upstream nodes of the current task instance node. The direct upstream nodes are the upstream task instance nodes that have a dependency relationship with the current task instance node. When the number of successful upstream nodes equals the number of direct upstream nodes, the current task instance is determined to start generating data, where the current task instance is the task instance represented by the current task instance node.

2. The data processing method according to claim 1, characterized in that, The method further includes: When the data for the current task instance is successfully generated, the status of the current task instance is updated to "Recovery Successful".

3. The data processing method according to claim 2, characterized in that, The method further includes: Store the target blocking information in the database; After updating the status of the current task instance to successful recovery, the method further includes: Delete the identifier of the current task instance from the database.

4. The data processing method according to claim 1, characterized in that, The target blocking information includes the priority of each of the multiple task instances, and the method further includes: When the number of successful upstream nodes is equal to the number of direct upstream nodes, and the direct upstream nodes are the same as the direct upstream nodes of the target task instance node, it is determined whether the priority of the target task instance is higher than the priority of the current task instance. Here, the target task instance node is the task instance node other than the current task instance node among the multiple task instance nodes included in the target directed acyclic graph; the target task instance is the task instance represented by the target task instance node. When the priority of the target task instance is higher than the priority of the current task instance, the target task instance is determined to start generating data first.

5. The data processing method according to any one of claims 1-4, characterized in that, The method further includes: Terminate the multiple blocking events.

6. A data processing apparatus, characterized in that, It includes an acquisition module, a determination module, and a processing module; The acquisition module is configured to acquire blocking information of multiple blocking events, wherein the blocking information of a blocking event includes the identifiers of at least two task instances corresponding to the blocking event, and the dependency relationship between the at least two task instances, and the blocking event is used to block the corresponding task instance from generating data; The processing module is further configured to generate a directed acyclic graph corresponding to each of the multiple blocking events based on the blocking information of the multiple blocking events. The directed acyclic graph corresponding to a blocking event includes task instance nodes and edges. The task instance nodes are used to represent task instances, and the edges are used to connect task instance nodes with dependencies. The processing module is specifically configured to merge the directed acyclic graphs corresponding to the multiple blocking events to generate a target directed acyclic graph. The target directed acyclic graph is the directed acyclic graph corresponding to the target blocking event. The target directed acyclic graph includes the task instance nodes included in the directed acyclic graphs corresponding to the multiple blocking events and the edges included in the directed acyclic graphs. The determining module is configured to determine the target blocking information based on the target directed acyclic graph, thereby obtaining the target blocking information. The target blocking information is the blocking information of the target blocking event. The target blocking event corresponds to multiple task instances, and the multiple task instances include each task instance corresponding to the multiple blocking events. The processing module is also configured to execute the target blocking event; The processing module is further configured to control the task instance to pause data generation when the target blocking information includes the identifier of the task instance to be identified; the current task instance node is the root node in the target directed acyclic graph or a non-root node in the target directed acyclic graph. The processing module is also configured to perform a recovery operation on the target blocking event; The determining module is specifically configured to determine the number of successful upstream nodes of the current task instance node when the current task instance node is the non-root node. The successful upstream nodes are the nodes in the current task instance node whose status is successful recovery. The successful upstream nodes are upstream task instance nodes that have a dependency relationship with the current task instance node. The determining module is further configured to determine that the current task instance starts generating data when the number of successful upstream nodes is equal to the number of direct upstream nodes, wherein the current task instance is the task instance represented by the current task instance node.

7. The data processing apparatus according to claim 6, characterized in that, The processing module is also configured to update the status of the current task instance to "recovery successful" when the data of the current task instance is successfully generated.

8. The data processing apparatus according to claim 7, characterized in that, The data processing device further includes a storage module and a deletion module; The storage module is configured to store the target blocking information into a database; The deletion module is configured to delete the identifier of the current task instance from the database.

9. The data processing apparatus according to claim 6, characterized in that, The target blocking information includes the priority of each of the multiple task instances; The determining module is further configured to determine whether the priority of the target task instance is higher than the priority of the current task instance when the number of successful direct upstream nodes is equal to the number of direct upstream nodes, and the direct upstream nodes are the same as the direct upstream nodes of the target task instance node. Here, the target task instance node is a task instance node other than the current task instance node among the multiple task instance nodes included in the target directed acyclic graph; the target task instance is the task instance represented by the target task instance node. The determining module is further configured to determine that the target task instance starts generating data first when the priority of the target task instance is higher than the priority of the current task instance.

10. The data processing apparatus according to any one of claims 6-9, characterized in that, The processing module is also configured to terminate the plurality of blocking events.

11. An electronic device, characterized in that, The electronic device includes: processor; A memory configured to store processor-executable instructions; The processor is configured to execute the instructions to implement the data processing method as described in any one of claims 1-5.

12. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions in the computer-readable storage medium are executed by an electronic device, the electronic device is able to perform the data processing method as described in any one of claims 1-5.

13. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method as described in any one of claims 1-5.

Citation Information

Patent Citations

  • Scheduling and failure recovery method for task processing flow

    CN106547613A

  • Intelligent distributed calculation management system and method based on DAG (Directed Acyclic Graph)

    CN108388474A