Data processing method and device, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to the completion of execution of at least one front task corresponding to a first task, and confirming the current concurrency of the first task; in response to the fact that the current concurrency of the first task is larger than a preset threshold value, executing other preposed tasks corresponding to the first task, and updating the current concurrency of the first task based on at least one completed preposed task; when the current concurrency of the first task is responded to be a preset threshold value, it is confirmed that all front tasks of the first task are executed, and the first task is executed; thus, under the condition that multiple tasks are called back at the same time, the concurrency is updated by combining with the optimistic lock, the post-task is triggered only when the concurrency meets a specific condition, and the problem that the process is stuck due to the fact that the post-task is triggered repeatedly is avoided.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular, to a data processing method, apparatus, electronic device, and storage medium. Background Art

[0002] In a credit scheduling platform, for a task with prerequisite tasks, it needs to be triggered after all prerequisite tasks are executed; if at least two prerequisite tasks are completed simultaneously, the corresponding successor tasks will be repeatedly triggered but can only be completed once, resulting in a situation where the process gets stuck. Summary of the Invention

[0003] The present disclosure provides a data processing method, apparatus, electronic device, and storage medium to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a data processing method is provided, the method comprising: In response to at least one prerequisite task corresponding to a first task being executed, confirm the current concurrency degree of the first task; In response to the current concurrency degree of the first task being greater than a preset threshold, execute other prerequisite tasks corresponding to the first task, and update the current concurrency degree of the first task based on the at least one prerequisite task that has been completed; In response to the current concurrency degree of the first task being the preset threshold, confirm that all prerequisite tasks of the first task have been executed, and execute the first task.

[0005] In the above solution, the method further comprises: Based on the relationships between tasks in a batch task association table corresponding to a batch task, confirm the concurrency degree of each task included in the batch task; the concurrency degree is used to represent the number of prerequisite tasks of the corresponding task; Based on the concurrency degree of each task, confirm a concurrency lock table; Wherein, the batch task includes at least the first task and at least one prerequisite task corresponding to the first task.

[0006] In the above solution, the step of based on the relationships between tasks in a batch task association table corresponding to a batch task, confirm the concurrency degree of each task included in the batch task, includes performing the following operations on each task: Based on a regular expression, match in the batch task association table to determine the number of occurrences of the prerequisite task corresponding to each task in the batch task association table; Based on the number of occurrences of the prerequisite task in the batch task association table, confirm the concurrency degree of the corresponding task.

[0007] In the above solution, the confirmation of the concurrent lock table based on the concurrency degree of each task includes: Confirm the concurrent lock table based on the batch task identifier, the identifier corresponding to each task, the concurrency degree of each task, and the version number; Wherein, if the concurrent lock table includes the batch task and other batch tasks, the identifier corresponding to each task is confirmed based on the representation of the task and the batch task identifier to which it belongs.

[0008] In the above solution, the confirmation of the current concurrency degree of the first task includes: Match based on the identifier of the first task in the concurrent lock table; Confirm the current concurrency degree corresponding to the first task in the concurrent lock table, or confirm the current concurrency degree and the current version number corresponding to the first task in the concurrent lock table.

[0009] In the above solution, the update of the current concurrency degree of the first task based on at least one completed pre-task includes at least one of the following: Adjust the current concurrency degree based on the number of at least one completed pre-task, so that the adjusted current concurrency degree is the same as the number of unexecuted pre-tasks; Adjust the current version number corresponding to the first task based on the number of updates to the current concurrency degree.

[0010] In the above solution, the method further includes: Perform verification based on the current concurrency degree and the current version number of the first task; In response to the sum of the current concurrency degree and the current version number being less than or equal to the number of at least one pre-task corresponding to the first task, send an alarm message.

[0011] According to a second aspect of the present disclosure, there is provided a data processing apparatus, the apparatus includes: A concurrency degree confirmation unit, configured to confirm the current concurrency degree of the first task in response to the execution completion of at least one pre-task corresponding to the first task; A processing unit, configured to execute other pre-tasks corresponding to the first task in response to the current concurrency degree of the first task being greater than a preset threshold, and update the current concurrency degree of the first task based on at least one completed pre-task; The processing unit is further configured to confirm that all pre-tasks of the first task have been executed and execute the first task in response to the current concurrency degree of the first task being the preset threshold.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including: At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the present disclosure.

[0013] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method described in the present disclosure.

[0014] In the data processing method of the present disclosure, upon completion of execution of at least one pre-task corresponding to a first task, the current concurrency degree of the first task is confirmed; upon the current concurrency degree of the first task being greater than a preset threshold, other pre-tasks corresponding to the first task are executed, and the current concurrency degree of the first task is updated based on the at least one pre-task that has been completed; upon the current concurrency degree of the first task being the preset threshold, it is confirmed that all pre-tasks of the first task have been completed, and the first task is executed; thus, in the case of multiple tasks being called back simultaneously, by combining an optimistic lock to update the concurrency degree, the subsequent task is triggered only when the concurrency degree meets specific conditions, avoiding the problem of the subsequent task being repeatedly triggered and causing the process to freeze.

[0015] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present disclosure will become readily understandable. In the drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, wherein: In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0017] Figure 1 Shows a judgment flowchart in the related art; Figure 2 Shows a first alternative flowchart of the data processing method provided by an embodiment of the present disclosure; Figure 3 Shows a second alternative flowchart of the data processing method provided by an embodiment of the present disclosure; Figure 4 Shows a third alternative flowchart of the data processing method provided by an embodiment of the present disclosure; Figure 5 Shows a fourth alternative flowchart of the data processing method provided by an embodiment of the present disclosure; Figure 6 The figure shows an optional structural schematic diagram of a data processing device provided by an embodiment of the present disclosure; Figure 7 The figure shows a composition structural schematic diagram of an electronic device according to an embodiment of the present disclosure. Detailed implementation manners

[0018] To make the objectives, features, and advantages of the present disclosure more obvious and understandable, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present disclosure.

[0019] In the following descriptions, "some embodiments" are involved, which describe a subset of all possible embodiments. However, it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0020] In the following descriptions, the terms "first / second" involved are only used to distinguish similar objects, and do not represent a specific order for the objects. It can be understood that "first / second" can be interchanged with a specific order or sequence when allowed, so that the embodiments of the present disclosure described here can be implemented in an order other than those illustrated or described here.

[0021] Unless otherwise defined, all technical and scientific terms used in the present disclosure have the same meanings as those commonly understood by those skilled in the technical field to which the present disclosure belongs. The terms used in the present disclosure are only for the purpose of describing the embodiments of the present disclosure, and are not intended to limit the present disclosure.

[0022] It should be understood that in various embodiments of the present disclosure, the magnitudes of the serial numbers of the various implementation processes do not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation processes of the embodiments of the present disclosure.

[0023] In the batch tasks of a credit system, the tasks are interrelated. One task may be a prerequisite task for another task, that is, the other task can only be continued after the prerequisite task is completed. For example, after the three tasks of debt repayment history shell call, co-debt burden shell call, and capital hunger shell call are completed, the machine learning data deletion task is triggered; then the three tasks of debt repayment history shell call, co-debt burden shell call, and capital hunger shell call are the prerequisite tasks for the machine learning data deletion task, and the machine learning data deletion task cannot be executed until the above three tasks are completed.

[0024] In the related art, after the last pre-task is completed, if it is determined that the other two pre-tasks are also completed, the post-task (i.e., the machine learning data deletion task) is called back, and the machine learning data deletion task is triggered. Although it is verified during the callback whether all pre-tasks are completed, and only if all are completed will the subsequent task be triggered. However, considering that if the two pre-tasks arrive simultaneously, the callbacks of both tasks will pass the verification and trigger the subsequent task; moreover, after one task triggers the post-task and the post-task is completed, it may cause the other post-task to be unable to execute, resulting in a process jam.

[0025] For example, task A corresponds to pre-task B, pre-task C, and pre-task D; when pre-task B has been completed, if pre-task C and pre-task D are completed simultaneously (or completed at very close times), the system, based on the completion of pre-task C, confirms that pre-task B, pre-task C, and pre-task D are all completed and triggers the execution of task A; in addition, the system also, based on the completion of pre-task D, confirms that pre-task B, pre-task C, and pre-task D are all completed and triggers the execution of task A; task A is triggered twice. If the function of task A is to delete data, after the first execution is completed, since the data has been deleted, the second execution cannot be carried out, resulting in the second-triggered task A not being able to be completed, and further causing a process jam.

[0026] Figure 1 The flowchart of judgment in the related art is shown.

[0027] As Figure 1 shown, if task A and task B trigger callbacks simultaneously, the system will separately search for subsequent tasks based on task A and task B, and confirm whether all corresponding pre-tasks are completed. If they are completed, the subsequent task will be triggered; therefore, the subsequent task will be triggered twice.

[0028] In view of the deficiencies in the related art, the embodiments of the present disclosure provide a data processing method. Before verifying whether the pre-task of the subsequent task is completed, concurrent judgment is combined with an optimistic lock to solve some or all of the above technical problems.

[0029] Figure 2 The first optional flowchart of the data processing method provided by the embodiments of the present disclosure is shown, and will be described according to each step.

[0030] Step S201, in response to the completion of at least one pre-task corresponding to the first task, confirm the current concurrency of the first task.

[0031] In some embodiments, the first task corresponds to at least one pre-task, that is, the first task can only be executed after all pre-tasks are completed.

[0032] In some embodiments, in response to any one of at least one prerequisite task corresponding to the first task being completed, the current concurrency degree of the first task is confirmed.

[0033] The current concurrency degree of the first task is used to represent the number of prerequisite tasks of the first task that are not completed, or the number of prerequisite tasks that have been completed but not updated; after the prerequisite task is completed, the carrier for implementing the data processing method (hereinafter referred to as the carrier) determines the current concurrency degree of the first task based on the identifier of the prerequisite task or the identifier of the first task.

[0034] In some alternative embodiments, the carrier may confirm the current concurrency degree of the first task based on a concurrency lock table; the concurrency lock table includes at least one of the identifier of the first task, the current concurrency degree of the first task, and the version number.

[0035] The carrier may be a computer program, an electronic circuit, a database, a mobile application, an electronic device, a cloud computing platform, a distributed system, an artificial intelligence framework, a mathematical model, an automation tool, a microcontroller, etc., which are software or hardware capable of implementing algorithms and method flows.

[0036] Step S202, in response to the current concurrency degree of the first task being greater than a preset threshold, execute other prerequisite tasks corresponding to the first task, and update the current concurrency degree of the first task based on at least one completed prerequisite task.

[0037] In some embodiments, in response to the current concurrency degree of the first task being greater than a preset threshold, it is confirmed that all prerequisite tasks of the first task have not been completed, execute the unexecuted prerequisite tasks corresponding to the first task, and update the current concurrency degree of the first task based on at least one completed prerequisite task.

[0038] In some embodiments, the carrier may update the current concurrency degree corresponding to the first task in the concurrency lock table; optionally, the current concurrency degree corresponding to the first task may be updated based on this task after any one prerequisite task is completed; or, the current concurrency degree corresponding to the first task may be updated based on all completed prerequisite tasks after multiple prerequisite tasks are completed.

[0039] In some embodiments, if none of the prerequisite tasks corresponding to the first task have been executed, the initial concurrency degree of the first task is the same as the number of prerequisite tasks; the current concurrency degree is updated as the prerequisite tasks are completed, and for each completed prerequisite task, the current concurrency degree is decreased by 1.

[0040] In specific implementation, when any of the previous tasks is confirmed to be completed, the carrier can directly update the current concurrency degree by subtracting 1; or when any of the previous tasks is confirmed to be completed, the carrier can confirm the number of all the previous tasks that have been completed, and determine the update value of the current concurrency degree based on the initial concurrency degree and the number of all the previous tasks that have been completed, and update the update value to the concurrency lock table.

[0041] In some embodiments, step S202 is implemented based on the optimistic lock mechanism, that is, the concurrency degree of the first task is updated based on the optimistic lock after the previous task is completed.

[0042] Step S203, in response to the current concurrency degree of the first task being a preset threshold, confirm that all the previous tasks of the first task have been executed, and execute the first task.

[0043] In some embodiments, the preset threshold may be 1. That is, if the current concurrency degree is greater than the preset threshold, it means that at least two tasks have not been executed, or have been executed but the current concurrency degree of the first task has not been updated; if the current concurrency degree is equal to 1, it means that all the previous tasks of the first task have been executed, but the carrier has not updated the current concurrency degree based on the last completed previous task. At this time, the current concurrency degree of the first task may not be updated, and the first task can be directly executed.

[0044] In this way, through the data processing method provided by the embodiments of the present disclosure, the current concurrency degree corresponding to the subsequent task will be updated after the previous task is completed. When the current concurrency degree is the preset threshold, it means that all the previous tasks have been executed, and the subsequent task can be called back and triggered, avoiding the problem that the subsequent task is repeatedly triggered and the process gets stuck.

[0045] Figure 3 Fig. 2 shows a second optional flowchart of the data processing method provided by the embodiments of the present disclosure, which will be described according to each step.

[0046] Step S301, based on the relationship between tasks in the batch task association table corresponding to the batch task, confirm the concurrency degree of each task included in the batch task.

[0047] In some embodiments, the batch task includes tasks composed of multiple tasks with an association relationship. Among the multiple tasks with an association relationship, at least one task is a previous task of another task, and a certain task is a subsequent task of other tasks.

[0048] In some embodiments, the batch task association table includes the association relationship between any task and other tasks and the identifier of each task. For example, the identifier of the second task, the identifier of at least one previous task of the second task, and the identifier of the subsequent task of the second task in the task association table.

[0049] In some embodiments, the carrier may match in the batch task association table based on a regular expression to determine the number of prerequisite tasks corresponding to each task, and confirm the concurrency degree of the corresponding task based on the number of prerequisite tasks corresponding to each task. For example, if the number of prerequisite tasks of the second task is a, then the concurrency degree of the second task is a.

[0050] In other embodiments, the carrier may match in the batch task association table based on a regular expression to determine the number of occurrences of the prerequisite tasks corresponding to each task in the batch task association table, and confirm the concurrency degree of the corresponding task based on the number of occurrences.

[0051] In specific implementation, "prerequisite task field" may be used as a keyword to match in the batch task association table to confirm the number of occurrences of the prerequisite task field corresponding to each task, which is the concurrency degree of the task. For example, for the second task match, if the number of occurrences of the prerequisite task field is determined to be a, then the concurrency degree of the second task is a. It should be noted that the prerequisite task field does not refer to a specific prerequisite task, but an attribute or category, and for any task, the "prerequisite task field" is the same character.

[0052] Step S302, confirm the concurrency lock table based on the concurrency degree of each task.

[0053] In some embodiments, the carrier may confirm the tasks with a concurrency degree greater than 0 in the batch task association table, and determine the concurrency lock table based on the tasks, the identifiers of the tasks, and the concurrency degree. In the concurrency lock table, it includes the concurrency degrees of multiple tasks included in at least one batch task.

[0054] In the concurrency lock table, the identifier of the task included in the batch task is determined by the identifier of the batch task and the identifier of the task; the concurrency lock table may further include the version number corresponding to each task, and the version number is used to represent the number of update times of the concurrency degree of the corresponding task.

[0055] For example, the concurrency lock table includes the concurrency degrees of batch task J and batch task K. Batch task J includes tasks 1, 2, and 3, and batch task K includes tasks 4 and 5. Then, in the concurrency lock table, the identifier of task 1 is determined by concatenating the identifier of batch task J and the identifier of task 1; the identifier of task 2 is determined by concatenating the identifier of batch task J and the identifier of task 2.

[0056] Step S303, in response to the completion of execution of at least one prerequisite task corresponding to the first task, confirm the current concurrency degree of the first task.

[0057] In some embodiments, the carrier may match in the concurrent lock table based on the identifier of the first task to obtain the current concurrency degree and the current version number of the first task; alternatively, the carrier may match in the concurrent lock table based on the identifier of the first task and the identifier of the batch task to which the first task belongs to obtain the current concurrency degree and the current version number of the first task.

[0058] Step S304, in response to the current concurrency degree of the first task being greater than a preset threshold, execute other pre-tasks corresponding to the first task, and update the current concurrency degree of the first task based on at least one completed pre-task.

[0059] In some embodiments, if the current concurrency degree of the first task is greater than a preset threshold, it indicates that at least one pre-task corresponding to the first task is not completed. Then, execute other uncompleted pre-tasks corresponding to the first task, and update the current concurrency degree of the first task based on the completed pre-tasks. Optionally, the carrier may subtract 1 from the current concurrency degree to determine the new value as the updated concurrency degree of the first task; the carrier may also add 1 to the current version number to determine the new value as the updated version number of the first task.

[0060] In some embodiments, the updated concurrency degree is the same as the number of unexecuted completed pre-tasks of the first task; the updated version number is the number of executed completed pre-tasks of the first task + 1.

[0061] Step S305, in response to the current concurrency degree of the first task being the preset threshold, confirm that all pre-tasks of the first task are executed, and execute the first task.

[0062] In some embodiments, the preset threshold is 1. In response to the current concurrency degree of the first task being the preset threshold, it indicates that the carrier has not updated the concurrency value in the concurrent degree table for the most recently completed pre-task, but has updated the concurrency value for all other pre-tasks. Thus, it can be determined that all pre-tasks of the first task are executed, and the first task can be executed.

[0063] In this way, through the data processing method provided by the embodiments of the present disclosure, the current concurrency degree corresponding to the post-task is updated after the pre-task is completed. When the current concurrency degree is the preset threshold, it indicates that all pre-tasks are executed, and the post-task can be called back and triggered, avoiding the problem that the post-task is repeatedly triggered and the process is blocked.

[0064] Figure 4 The third optional flowchart of the data processing method provided by the embodiments of the present disclosure is shown, and will be described according to each step.

[0065] Step S401: In response to the completion of the execution of the third task, confirm the current concurrency degree of the subsequent task.

[0066] In some embodiments, in response to the completion of the execution of the third task, the carrier confirms whether there is a subsequent task for the third task from the batch task association table. If there is a subsequent task, based on the identifier of the batch task and the identifier of the subsequent task, confirm the current concurrency degree of the subsequent task in the concurrency lock table.

[0067] Step S402: In response to the current concurrency degree of the subsequent task being greater than the preset threshold, execute other prerequisite tasks corresponding to the subsequent task, and update the current concurrency degree of the subsequent task based on at least one completed prerequisite task.

[0068] Step S403: In response to the current concurrency degree of the subsequent task being the preset threshold, confirm that all prerequisite tasks of the subsequent task have been completed, and execute the first task.

[0069] Figure 5 FIG. 4 shows a fourth alternative flowchart of the data processing method provided by the embodiments of the present disclosure, and will be described according to each step.

[0070] Step S501: Add a concurrency degree field to the batch task.

[0071] In some embodiments, the carrier adds a concurrency degree field to the script of each task included in the batch task. The concurrency degree field represents the number of prerequisite tasks of the current task. Specifically, it may include: “ALTER TABLE VUE_XXL_JOB.FLOW_JOB_RELATION ADD con_currence NUMBER(20) DEFAULT 0 NOT NULL; COMMENT ON COLUMN VUE_XXL_JOB.FLOW_JOB_RELATION.con_currence IS 'Concurrency degree, default is 0, that is, the number of prerequisite nodes of this node';”.

[0072] Step S502: Initialize the concurrency degree of each task in the batch task.

[0073] In some embodiments, the carrier initializes the concurrency degree based on the number of prerequisite tasks of each task stored in the batch task association table, and determines that the initial concurrency degree of each task is the number of its corresponding prerequisite tasks. Specifically, it may include: update VUE_XXL_JOB.Flow_Job_Relation fr set fr.con_currence = (select A.VALUE from (SELECT COUNT(1) VALUE, m.flow_id flow_id, m.job_id job_id, m.node_id node_id FROM (select REGEXP_SUBSTR(TS.Pre_Node_Ids, '[^,]+', 1, L) NAME, ts.flow_id, ts.job_id, ts.node_id from VUE_XXL_JOB.Flow_Job_Relation TS, (SELECT LEVEL L FROM DUAL CONNECT BY LEVEL<= 1000) where L(+)<= LENGTH(TS.Pre_Node_Ids) LENGTH(REPLACE(TS.Pre_Node_Ids, ',')) + 1) M WHERE M.NAME IS NOT NULL GROUP BY m.flow_id, m.job_id, m.node_id ORDER BY COUNT(1) DESC) A where A.flow_id = fr.flow_id and A.job_id = fr.job_id and A.node_id = fr.node_id) where fr.pre_node_ids is not null; Specifically, TS.Pre_Node_Ids is a field in the VUE_XXL_JOB.Flow_Job_Relation table, storing a list of IDs of pre-tasks, usually a comma-separated string (such as "1001,1002,1003").

[0074] In the above code, the REGEXP_SUBSTR(TS.Pre_Node_Ids, '[^,]+', 1, L) is used to split the Pre_Node_Ids field to extract each pre-task ID. Then, the number of pre-tasks for each (flow_id, job_id, node_id) combination is calculated based on COUNT(1). Finally, fr.con_currence is updated with this count value, which is the concurrency degree.

[0075] In specific implementation, the carrier can use a regular expression to confirm the occurrence times of the pre-task field, add 1 to it, and finally obtain the concurrency degree of this task.

[0076] In some embodiments, the carrier constructs a concurrency lock table based on the identifier of the batch task, the identifier of the task, and the concurrency degree of the task.

[0077] Step S503, insert an optimistic lock.

[0078] In some embodiments, in response to the start of the execution of the batch task, that is, when the start node is triggered, the carrier traverses the batch task and inserts an optimistic lock record into the concurrency lock table. The initial optimistic lock record is the concurrency degree of the current task, and the initial version number is 1.

[0079] In some alternative embodiments, the carrier inserts the concurrency lock table into the script, specifically including: List <flowjobrelation>flowRelation = xxlJobAdminconfig.getAdminconfig().getFlowJobRelationDao().findFlowRelationonconcurrence((long)jobLog.getFlowId());flowRelation.forEach(f->f logger.info("The current task needs to insert an optimistic lock, jobId={}, flowserialNo={},", f.getJobId(), jobLog.getFlowserialNo()); Xx1JobCallBackLock xx1JobCallBackLock = new XxlJobcallBackLock(f.getJobId() , shardSerialNo: jobLog.getFlowserialNo() + "$" + f.getJobId() , f.getconcurrence(), version: 1); / / The initial version number is 1 Xx1JobAdminconfig.getAdminconfig().getXx1JobcallBackLockDao().save(xx1JobcallBackLock); }); Step S504, in response to the completion of the previous task, perform concurrency verification.

[0080] In some embodiments, in response to the completion of the previous task, the carrier confirms the current concurrency and the current version number of the subsequent task in the concurrency lock table based on the identifier of the batch task and the identifier of the subsequent task corresponding to the previous task; and updates the optimistic lock record; specifically, updating the optimistic lock record includes decrementing the concurrency by 1 and incrementing the version number by 1; in response to the successful update of the optimistic lock, confirm whether the concurrency of the subsequent task before the update is 1, if it is 1, then execute Step S505, otherwise continue to execute other previous tasks.

[0081] Step S505, confirm whether all the previous tasks of the subsequent task are completed.

[0082] In some embodiments, the carrier confirms whether all the previous tasks of the subsequent task are completed. If all are completed, trigger the subsequent task; if not all are completed, continue to execute the unexecuted previous tasks.

[0083] Thus, through the data processing method provided by the embodiments of the present disclosure, after the pre-task is completed, the current concurrency degree corresponding to the post-task is updated. When the current concurrency degree is a preset threshold, it indicates that all pre-tasks have been executed, and the post-task can be called back and triggered, avoiding the problem that the post-task is repeatedly triggered and the process is stuck.

[0084] Figure 6 FIG. shows an optional structural schematic diagram of a data processing device provided by the embodiments of the present disclosure, which will be described according to each part.

[0085] In some embodiments, the data processing device 600 includes a concurrency confirmation unit 601 and a processing unit 602.

[0086] The concurrency confirmation unit 601 is configured to confirm the current concurrency degree of the first task in response to the completion of at least one pre-task corresponding to the first task; The processing unit 602 is configured to execute other pre-tasks corresponding to the first task in response to the current concurrency degree of the first task being greater than a preset threshold, and update the current concurrency degree of the first task based on the at least one completed pre-task; The processing unit 602 is further configured to confirm that all pre-tasks of the first task have been executed and execute the first task in response to the current concurrency degree of the first task being the preset threshold.

[0087] In some optional embodiments, the data processing device 600 further includes a determination unit 603.

[0088] The determination unit 603 is configured to confirm the concurrency degree of each task included in the batch task based on the relationship between tasks in the batch task association table corresponding to the batch task; the concurrency degree is used to represent the number of pre-tasks of the corresponding task; confirm the concurrency lock table based on the concurrency degree of each task; wherein, the batch task includes at least the first task and at least one pre-task corresponding to the first task.

[0089] The determination unit 603 is specifically configured to perform the following operations on each task: Match in the batch task association table based on a regular expression to determine the number of pre-tasks corresponding to each task; Confirm the concurrency degree of the corresponding task based on the number of pre-tasks corresponding to each task.

[0090] The determination unit 603 is specifically configured to confirm the concurrency lock table based on the batch task identifier, the identifier corresponding to each task, the concurrency degree of each task, and the version number. Among them, if the concurrent lock table includes the batch task and other batch tasks, the identifier corresponding to each task is confirmed based on the identifier of the task and the identifier of the batch task to which it belongs.

[0091] The concurrency confirmation unit 601 is specifically configured to match based on the identifier of the first task in the concurrent lock table; Confirm the current concurrency corresponding to the first task in the concurrent lock table, or confirm the current concurrency and the current version number corresponding to the first task in the concurrent lock table.

[0092] The processing unit 602 is specifically configured to perform at least one of the following: Update the current concurrency based on the number of at least one pre-task that has been completed, so that the updated current concurrency is the same as the number of pre-tasks that have not been completed; Adjust the current version number corresponding to the first task based on the number of updates to the current concurrency.

[0093] The processing unit 602 is further configured to perform a check based on the current concurrency and the current version number of the first task; In response to the sum of the current concurrency and the current version number being less than or equal to the number of at least one pre-task corresponding to the first task, send an alarm message.

[0094] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0095] Figure 7 A schematic block diagram of an exemplary electronic device 800 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processing, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0096] As Figure 7 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 802 or the computer program loaded from the storage unit 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. The input / output (I / O) interface 805 is also connected to the bus 804.

[0097] Multiple components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disc, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0098] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the data processing method. For example, in some embodiments, the data processing method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the data processing method described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the data processing method in any other appropriate manner (e.g., by means of firmware).

[0099] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0100] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0101] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0102] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0103] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0104] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, or a server of a distributed system, or a server incorporating a blockchain.

[0105] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0106] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this disclosure, "a plurality" means two or more unless otherwise specifically defined.

[0107] As described above, it is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims described above.< / flowjobrelation>

Claims

1. A data processing method, characterized in that, The method includes: In response to the completion of at least one prerequisite task corresponding to the first task, confirm the current concurrency degree of the first task; In response to the current concurrency degree of the first task being greater than a preset threshold, execute other prerequisite tasks corresponding to the first task, and update the current concurrency degree of the first task based on the at least one completed prerequisite task; In response to the current concurrency degree of the first task being the preset threshold, confirm that all prerequisite tasks of the first task have been completed, and execute the first task.

2. The method according to claim 1, characterized in that, The method further includes: Based on the relationships between tasks in the batch task association table corresponding to the batch task, confirm the concurrency degree of each task included in the batch task; the concurrency degree is used to characterize the number of prerequisite tasks of the corresponding task; Confirm the concurrency lock table based on the concurrency degree of each task; Wherein, the batch task includes at least the first task and at least one prerequisite task corresponding to the first task.

3. The method according to claim 2, characterized in that, The confirming the concurrency degree of each task included in the batch task based on the relationships between tasks in the batch task association table corresponding to the batch task includes performing the following operations on each task: Match in the batch task association table based on a regular expression to determine the number of prerequisite tasks corresponding to each task; Based on the number of prerequisite tasks corresponding to each task, confirm the concurrency degree of the corresponding task.

4. The method according to claim 2, wherein The confirming the concurrency lock table based on the concurrency degree of each task includes: Confirm the concurrency lock table based on the batch task identifier, the identifier corresponding to each task, the concurrency degree of each task, and the version number; Wherein, if the batch task and other batch tasks are included in the concurrency lock table, the identifier corresponding to each task is confirmed based on the identifier of the task and the identifier of the batch task to which the task belongs.

5. The method according to claim 1, characterized in that, The confirming the current concurrency degree of the first task includes: Match in the concurrency lock table based on the identifier of the first task; Confirm the current concurrency degree corresponding to the first task in the concurrency lock table, or confirm the current concurrency degree and the current version number corresponding to the first task in the concurrency lock table.

6. The method according to claim 1, characterized in that, The updating the current concurrency degree of the first task based on the at least one completed prerequisite task includes at least one of the following: Update the current concurrency degree based on the number of at least one completed prerequisite task, so that the updated current concurrency degree is the same as the number of uncompleted prerequisite tasks; Adjust the current version number corresponding to the first task based on the number of updates to the current concurrency degree.

7. The method according to claim 1, characterized in that, The method further includes: Perform verification based on the current concurrency degree and the current version number of the first task; In response to the sum of the current concurrency degree and the current version number being less than or equal to the number of at least one prerequisite task corresponding to the first task, send an alarm message.

8. A data processing device, characterized in that, The device includes: A concurrency degree confirmation unit, configured to confirm the current concurrency degree of the first task in response to the completion of at least one prerequisite task corresponding to the first task; A processing unit, configured to execute other prerequisite tasks corresponding to the first task in response to the current concurrency degree of the first task being greater than a preset threshold, and update the current concurrency degree of the first task based on the at least one completed prerequisite task; The processing unit is further configured to, in response to the current concurrency degree of the first task reaching a preset threshold, confirm that all the prerequisite tasks of the first task have been completed, and execute the first task.

9. An electronic device, characterized in that, Comprising: At least one processor; And a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1-7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause a computer to execute the method according to any one of claims 1-7.

Images