Data processing task scheduling method, device, electronic device and medium

Abstract

The embodiments of the present disclosure disclose a data processing task scheduling method, device, electronic device and medium. A specific implementation of the method includes: obtaining a business data processing task information set; determining a task dependency information set; performing task scheduling processing on the business data processing task information set to obtain task scheduling information; determining at least one business data processing task information corresponding to the task execution exception; locating the abnormal root cause of the abnormal data processing task information set to obtain a task abnormal root cause locating information set; performing task resource allocation processing on the abnormal data processing task information set to obtain a task resource allocation information set; re-task scheduling processing on the abnormal data processing task information set and the remaining data processing task information set to obtain target task scheduling information; generating a task trigger information set for the business data processing task information set. This implementation can improve the efficiency of task execution and improve the stability of the task execution system.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a method, device, electronic device, and medium for scheduling data processing tasks. Background Art

[0002] With the advent of the big data era, the number and complexity of data processing tasks are increasing, and it is becoming increasingly important to effectively manage and coordinate the call and execution of data processing tasks. For the scheduling of data processing tasks, the usual approach is to determine the task priority set of the data processing task information set. Then, according to the task priority set, the data processing task information set is scheduled to obtain task scheduling information. Finally, according to the task scheduling information, the data processing task set is executed.

[0003] However, it is found in practice that when the above method is used to schedule data processing tasks, the following technical problems often occur: Task scheduling is performed only through the task priority of the data processing task information set, and the influencing factors considered are relatively simple, which can easily lead to low task scheduling efficiency, waste of system resources and low user experience.

[0004] In the process of adopting technical solutions to solve the above-mentioned technical problem 1, the following technical problem 2 is often accompanied: when the task scheduling information represents the task execution anomaly, how to accurately and efficiently detect the abnormal data processing task information set in the task execution anomaly to ensure the execution of the business data processing task and the stability of the operating system. For the above-mentioned technical problem 2, the conventional solution is generally: obtain the task scheduling log information set corresponding to the task execution anomaly. Then, the task scheduling log information set is input into the multivariate time series anomaly detection model to obtain the abnormal data processing task information set. However, the above conventional solution still has the following problems: due to the use of task scheduling log information as the input of the model, the calling structure of the business data processing task is not taken into account, and the multivariate time series anomaly detection model only detects anomalies within a period of time, and does not take into account the spatial correlation between different time series, resulting in low anomaly detection accuracy, increasing the occurrence of missed detection and false detection, prolonging the task execution time and reducing the stability and security of the system that executes the business data processing task set.

[0005] In the process of adopting technical solutions to solve the above-mentioned technical problem one, the following technical problem three is often accompanied: how to schedule the execution order of the business data processing task set and the allocation of resources for the business data processing tasks to improve the execution efficiency of the business data processing task set and reduce the waste of task resources. For the above-mentioned technical problem three, the conventional solution is generally: through the gray wolf optimization algorithm, the business data processing task set is task scheduled, task scheduling information is obtained, and the business data processing task set is executed according to the task scheduling information. However, the above-mentioned conventional solution still has the following problems: Since the gray wolf optimization algorithm uses random initialization to obtain the initial population, there is a lot of randomness, which can easily cause the algorithm's optimization efficiency to be low as the number of executions increases, and it falls into a local optimal solution. The allocation of business data processing task execution resources is unreasonable, resulting in low business data processing task execution efficiency, increased task execution time, waste of task execution resources and poor user experience.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0007] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0008] Some embodiments of the present disclosure propose a data processing task scheduling method, apparatus, electronic device, and medium to solve one or more of the technical problems mentioned in the above background technology section.

[0009] In a first aspect, some embodiments of the present disclosure provide a data processing task scheduling method, comprising: obtaining a business data processing task information set corresponding to a user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, and user item acquisition information, and the business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks; determining a task dependency relationship information set between the business data processing task information included in the business data processing task information set; performing task scheduling processing on the business data processing task set corresponding to the business data processing task information set according to the task dependency relationship information set to obtain task scheduling information; in response to determining that the task scheduling information represents a task execution abnormality, determining at least one business data processing task information corresponding to the task execution abnormality as the abnormal data processing task information set; An abnormal root cause is located for each abnormal data processing task information in the information set to generate task abnormal root cause location information and obtain a task abnormal root cause location information set; based on the above-mentioned task abnormal root cause location information set, task resource allocation processing is performed on the above-mentioned abnormal data processing task information set and the remaining data processing task information set to obtain a task resource allocation information set, wherein the above-mentioned remaining data processing task information set is the task information set in the above-mentioned business data processing task information set that is located after the above-mentioned abnormal data processing task information set and depends on it; based on the above-mentioned task resource allocation information set, task scheduling processing is re-performed on the above-mentioned abnormal data processing task information set and the remaining data processing task information set to obtain target task scheduling information; based on the above-mentioned task scheduling information and the above-mentioned target task scheduling information, a task trigger information set for the above-mentioned business data processing task information set is generated, and based on the above-mentioned task trigger information set and the above-mentioned task dependency information set, the above-mentioned business data processing task is executed.

[0010] In a second aspect, some embodiments of the present disclosure provide a data processing task scheduling device, including: an acquisition unit, configured to acquire a business data processing task information set corresponding to a user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, and user item acquisition information, and the business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks; a first determination unit, configured to determine a task dependency relationship information set between the business data processing task information included in the business data processing task information set; a first task scheduling unit, configured to perform task scheduling processing on the business data processing task set corresponding to the business data processing task information set according to the task dependency relationship information set, and obtain task scheduling information; a second determination unit, configured to determine at least one business data processing task information corresponding to the task execution abnormality in response to determining that the task scheduling information represents a task execution abnormality, as the abnormal data processing task information set; an abnormality root cause locating unit, configured to The abnormal root cause of each abnormal data processing task information in the above-mentioned abnormal data processing task information set is located to generate task abnormal root cause location information and obtain a task abnormal root cause location information set; a task resource allocation unit is configured to perform task resource allocation processing on the above-mentioned abnormal data processing task information set and the remaining data processing task information set according to the above-mentioned task abnormal root cause location information set to obtain a task resource allocation information set, wherein the above-mentioned remaining data processing task information set is the task information set in the above-mentioned business data processing task information set that is located after the above-mentioned abnormal data processing task information set and depends on it; a second task scheduling unit is configured to re-task schedule processing on the above-mentioned abnormal data processing task information set and the remaining data processing task information set according to the above-mentioned task resource allocation information set to obtain target task scheduling information; an execution unit is configured to generate a task trigger information set for the above-mentioned business data processing task information set according to the above-mentioned task scheduling information and the above-mentioned target task scheduling information, and execute the above-mentioned business data processing task according to the above-mentioned task trigger information set and the above-mentioned task dependency information set.

[0011] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0012] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method described in any implementation manner in the first aspect is implemented.

[0013] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the data processing task scheduling method of some embodiments of the present disclosure can improve the efficiency of task execution and improve the stability of the task execution system. Specifically, the reason for the low efficiency of related task scheduling, the waste of system resources and the low user experience is that the task scheduling is performed only through the task priority of the data processing task information set, and the influencing factors considered are relatively single, which is easy to cause low task scheduling efficiency, waste of system resources and low user experience. Based on this, the data processing task scheduling method of some embodiments of the present disclosure can first obtain a business data processing task information set corresponding to the user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, user item acquisition information, and the business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks. Here, the business data processing task information set facilitates the subsequent generation of task scheduling information. Secondly, determine the task dependency relationship information set between the business data processing task information included in the business data processing task information set. Here, the execution order between the business protective gear processing tasks can be more comprehensively understood, so as to improve the task execution efficiency in the future. Again, according to the above-mentioned task dependency information set, the business data processing task set corresponding to the above-mentioned business data processing task information set is subjected to task scheduling processing to obtain task scheduling information. Here, the factors affecting task execution are fully considered, which can improve the accuracy of task scheduling information and the efficiency of task execution. Subsequently, in response to determining that the above-mentioned task scheduling information represents a task execution exception, at least one business data processing task information corresponding to the task execution exception is determined as an abnormal data processing task information set. Here, determining the abnormal data processing task information set that causes the task execution exception can improve the accuracy of determining the exception, shorten the search range of abnormality detection, and facilitate improving the efficiency of subsequent root cause location. Next, each abnormal data processing task information in the above-mentioned abnormal data processing task information set is subjected to abnormal root cause location to generate task abnormal root cause location information, and obtain a task abnormal root cause location information set. Here, the root cause of the abnormality can be accurately determined, and the time to recover the abnormality can be shortened. Afterwards, according to the above-mentioned task abnormality root cause location information set, the above-mentioned abnormal data processing task information set and the remaining data processing task information set are processed for task resource allocation to obtain a task resource allocation information set, wherein the above-mentioned remaining data processing task information set is the task information set in the above-mentioned business data processing task information set that is located after the above-mentioned abnormal data processing task information set and depends on it. Here, the resources required for task execution can be reasonably allocated to improve the smooth execution of tasks and reduce the waste of resources. Then, according to the above-mentioned task resource allocation information set, the above-mentioned abnormal data processing task information set and the remaining data processing task information set are re-tasked to obtain the target task scheduling information.Here, rescheduling can further ensure the smooth execution of the business data processing task set and improve the execution efficiency. Finally, according to the above task scheduling information and the above target task scheduling information, a task trigger information set for the above business data processing task information set is generated, and according to the above task trigger information set and the above task dependency information set, the above business data processing task is executed. Here, the smooth execution of the task can be guaranteed, the waste of operating resources can be reduced, the user experience can be improved, and the stability of the system for executing the task can be improved. It can be obtained that the data processing task scheduling method can improve the efficiency of task execution and the stability of the system for executing the task. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0015] Figure 1 is a flow chart of some embodiments of the data processing task scheduling method according to the present disclosure;

[0016] Figure 2 is a schematic diagram of the structure of some embodiments of the data processing task scheduling device according to the present disclosure;

[0017] Figure 3 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 The process 100 of some embodiments of the data processing task scheduling method according to the present disclosure is shown. The data processing task scheduling method comprises the following steps:

[0025] Step 101: Obtain a business data processing task information set corresponding to a user credit business data set.

[0026] In some embodiments, the execution subject (e.g., electronic device) of the data processing task scheduling method can obtain the business data processing task information set corresponding to the user credit business data set through a wired connection method or a wireless connection method, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, and user item acquisition information, and the business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks. Among them, the user credit business data in the user credit business data set can be data related to user credit information. For example, the user credit business data can be user credit investigation data. The business data processing task information in the business data processing task information set can be information about offline tasks or online tasks for data processing and processing of the user credit business data set. For example, the business data processing task can be a user credit assessment task, or it can be a pre-processing execution of the user credit business data set. The user value flow information (fund transaction information) can be the flow information of the user acquiring items. The user item acquisition information can be information about items purchased by the user. The dependent processing task can be a dependent condition required when executing the business data processing task. For example, the above-mentioned dependent processing task may be that the user credit service dataset needs to be obtained before the user credit service dataset is pre-processed.

[0027] Step 102: determine a task dependency relationship information set between the business data processing task information included in the business data processing task information set.

[0028] In some embodiments, the execution subject may determine a task dependency information set between the business data processing task information included in the business data processing task information set. The task dependency information in the task dependency information set may be the dependency of the required resources and the task execution order of the business data processing task during the execution process. The task dependency information may represent the order of execution of the business data processing task.

[0029] In some optional implementations of some embodiments, the above-mentioned determination of the task dependency relationship information set between the business data processing task information included in the above-mentioned business data processing task information set may include the following steps:

[0030] The first step is to determine the task input data, task output data and task scheduling resource information of each business data processing task information in the above-mentioned business data processing task information set, and obtain a task input data set, a task output data set and a task scheduling resource information set. Among them, the above-mentioned task input data can be the user credit business information that needs to be processed during the execution of the business data processing task. The above-mentioned task output data can be the output data obtained after the business data processing task processes the input user credit business information. The above-mentioned task scheduling resource information can be the resource information required to execute the business data processing task. For example, the above-mentioned resource information can include but is not limited to at least one of the following: memory resources, CPU resources, and network resources.

[0031] The second step is to determine the task call dependency information set of the business data processing task information included in the business data processing task information set based on the task input data set and the task output data set. The task call dependency information in the task call dependency information set can be the call sequence of the business data processing task execution before and after and the call sequence of simultaneous execution.

[0032] As an example, the execution subject may determine at least one task input data in the task input data set that is identical to the task input data in the task input data set, and obtain a target task input data set. Next, the business data processing task corresponding to the task output data identical to each target task input data in the target task input data set is determined as the preceding business data processing task of the business data processing task information corresponding to the target task input data, and a preceding business data processing task information set is obtained. Subsequently, the front-to-back order of the input data of the preceding business data processing task information corresponding to each business data processing task information in the business data processing task information set corresponding to the target task input data set is determined as a task preceding and following call dependency information set. Afterwards, at least one task input data with the same task input data is filtered out from the task input data set, and an identical task input data set is obtained. Finally, the parallel execution call information set and the task preceding and following call dependency information set of at least one business data processing task information corresponding to the identical task input data set are determined as a task call dependency information set.

[0033] The third step is to obtain a system resource information set for the business data processing task information set. The system resource information in the system resource information set may be resource information of a system server that executes the business data processing task corresponding to the business data processing task information set. For example, the system resource information set may include but is not limited to at least one of the following: memory resources of the system server, CPU resources of the system server, and input and output resources of the system server.

[0034] The fourth step is to determine the task resource dependency information set between the business data processing task information included in the business data processing task information set based on the task scheduling resource information set, the system resource information set and the task call dependency information set. The task resource dependency information in the task resource dependency information set may be information after adjusting the task call dependency information because the total task scheduling resource information set required for executing multiple business data processing tasks simultaneously when executing the business data processing task set through the task pre- and post-call dependency information set is greater than the system resource information set.

[0035] As an example, the execution subject may first determine at least one business data processing task information corresponding to the parallel execution dependency information in the task call dependency information set to obtain a parallel business data processing task information set. Then, the sum of the task scheduling resource information sets required by the parallel business data processing task information set is determined as the parallel task resource information set. Finally, in response to determining that the parallel task resource information set is greater than the system resource information set, the parallel execution dependency information in the task call dependency information set is randomly deleted to obtain a task resource dependency information set.

[0036] Step 5: According to the task resource dependency information set, determine the task time dependency information set between the business data processing task information included in the business data processing task information set. The task time dependency information in the task time dependency information set may be the time dependency information for executing the business data processing task.

[0037] As an example, the execution subject may first determine the task time dependency information set between the business data processing task information included in the business data processing task information set using a resource calling algorithm based on the task resource dependency information set. The resource calling algorithm may be a greedy algorithm or a dynamic programming algorithm.

[0038] The sixth step is to determine the task type information and task priority information of each business data processing task information in the business data processing task information set, and obtain a task type information set and a task priority information set. The task type information in the task type information set may be information on the type of execution of the business data processing task. The execution type may include parallel execution and serial execution. The task priority information may characterize the urgency of the business data processing task.

[0039] In the seventh step, the task time dependency information set, the task type information set and the task priority information set are determined as the task dependency relationship information set between the business data processing task information included in the business data processing task information set.

[0040] Step 103 , according to the task dependency information set, perform task scheduling processing on the business data processing task set corresponding to the business data processing task information set to obtain task scheduling information.

[0041] In some embodiments, the execution subject may perform task scheduling on the business data processing task set corresponding to the business data processing task information set according to the task dependency information set to obtain task scheduling information. The task scheduling information may be the execution order of the business data processing task set under the resource constraints and time constraints of the system server executing the business data processing task set.

[0042] As an example, the execution subject may utilize a topological sorting algorithm based on a directed acyclic graph to perform task scheduling on the business data processing task set corresponding to the business data processing task information set according to the task dependency information set to obtain task scheduling information.

[0043] In some optional implementations of some embodiments, performing task scheduling processing on the business data processing task set corresponding to the business data processing task information set according to the task dependency information set to obtain task scheduling information may include the following steps:

[0044] In the first step, a topology graph is constructed for the task dependency information set to obtain a task dependency call topology graph, wherein the task dependency call topology graph may be a topology graph that displays the task dependency information in a graph-like form.

[0045] The second step is to perform topological sorting on the task dependency call topology graph to obtain a task dependency topology sequence set, wherein the task dependency topology sequence in the task dependency topology sequence set may be the calling order of at least one business data processing task of a serial call in the task dependency call topology graph.

[0046] The third step is to sequentially add the business data processing task information set to the executable task queue according to the task dependency topology sequence set, wherein the executable task queue may be a system stack storing the task dependency topology sequence set.

[0047] As an example, the execution subject may sequentially add the business data processing task information set to the executable task queue according to the execution order of the tasks of each task dependency topology sequence in the task dependency topology sequence set.

[0048] The fourth step is to perform visual monitoring processing on the executable task queue to obtain a task monitoring information set, wherein the task monitoring information in the task monitoring information set may be monitoring information for monitoring the execution of the business data processing task.

[0049] In the fifth step, in response to determining that there is task monitoring information representing an abnormality in the task monitoring information set, at least one business data processing task information corresponding to at least one task monitoring information representing an abnormality is determined to be executed again to obtain an abnormal call information set. The task monitoring information representing an abnormality may be task monitoring information corresponding to the alarm information sent, or task monitoring information that failed to execute.

[0050] In the sixth step, the abnormal call information set and at least one task monitoring information representing normality are determined as task scheduling information.

[0051] In some optional implementations of some embodiments, in response to determining that there is task monitoring information representing an abnormality in the task monitoring information set, determining that at least one business data processing task information corresponding to at least one task monitoring information representing the abnormality is executed again to obtain the abnormal call information set may include the following steps:

[0052] The first step is to perform the following sending steps for each task monitoring information in the at least one task monitoring information according to the business data processing task information corresponding to the task monitoring information:

[0053] Sub-step 1: Determine the number of times the business data processing task information has been executed.

[0054] Sub-step 2: re-execute the business data processing task information to obtain a re-execution result, wherein the re-execution result can represent the execution status of the business data processing task corresponding to the business data processing task information.

[0055] Sub-step 3, in response to determining that the re-execution result indicates successful execution, returning task call information, wherein the task call information may be call information indicating successful execution of the business data processing task.

[0056] Sub-step 4, in response to determining that the above re-execution result indicates an execution failure, and the number of executions is greater than or equal to a preset execution threshold, sending the task call log information indicating the execution failure to the target terminal. The above preset execution threshold may be a preset maximum value of re-execution. For example, the above preset execution threshold may be 3. The above target terminal may be a server terminal for querying execution results.

[0057] In the second step, in response to determining that the re-execution result indicates an execution failure and the number of executions is less than the preset execution threshold, the sending step is executed again.

[0058] The third step is to determine the task call information group or the task call log information group as an abnormal call information set.

[0059] Step 104 , in response to determining that the task scheduling information represents a task execution abnormality, at least one piece of business data processing task information corresponding to the task execution abnormality is determined as an abnormal data processing task information set.

[0060] In some embodiments, the execution subject may determine at least one business data processing task information corresponding to the task execution anomaly in response to determining that the task scheduling information represents a task execution anomaly, as an abnormal data processing task information set. In practice, in response to determining that the task scheduling information represents a task execution anomaly, the execution subject may first obtain a task execution log information set corresponding to the task scheduling information. Then, the task execution log information set is input into a task execution anomaly detection model to obtain an abnormal data processing task information set. Among them, the task execution anomaly detection model may be an LSTM (Long Short-Term Memory).

[0061] In some optional implementations of some embodiments, in response to determining that the task scheduling information represents a task execution exception, determining at least one business data processing task information corresponding to the task execution exception as an abnormal data processing task information set may include the following steps:

[0062] The first step is to obtain a task execution resource call information set corresponding to the task scheduling information in response to determining that the task scheduling information indicates a task execution abnormality. The task execution resource call information in the task execution resource call information set may be information about resources called during the execution of the business data processing task.

[0063] The second step is to perform data standardization processing on the above-mentioned task execution resource call information set to obtain a standardized task execution resource call information set. It should be noted that since the calculation units of the task execution resource call information included in the task execution resource call information set are different, it is necessary to perform standardization processing to avoid the problem of low accuracy in determining the abnormal data processing task information due to different calculation units. In practice, the above-mentioned execution subject can perform the following data standardization processing steps for each task execution resource call information in the above-mentioned task execution resource call information set: First, determine the resource call mean and resource call variance of the above-mentioned task execution resource call. Then, determine the difference between the resource call index corresponding to the above-mentioned task execution resource call information and the above-mentioned resource call mean as the resource call difference. Among them, the above-mentioned resource call index can be the consumption of resources corresponding to the task execution resource call information required to execute the above-mentioned business data processing task. Finally, determine the ratio of the above-mentioned resource call difference to the above-mentioned resource call variance as the standardized task execution resource call information.

[0064] The third step is to extract features from the standardized task execution resource call information set to obtain a task execution feature vector set. The task execution feature vectors in the task execution feature vector set can represent the resource call information of the standardized task execution resource call information. In practice, the execution subject can input the standardized task execution resource call information into a multi-layer perceptron network to obtain a task execution feature vector set.

[0065] The fourth step is to determine the call resource association values ​​between the various business data processing tasks included in the above-mentioned task execution resource call information set, and obtain a call resource association value set. Among them, the above-mentioned call resource association values ​​can characterize the degree of association between the various indicators that reflect the task running status when the various business data processing tasks included in the task execution resource call information are running. The above-mentioned various indicators may include but are not limited to at least one of the following: memory load indicator, CPU load indicator and network delay indicator. The above-mentioned call resource association values ​​can use the Pearson correlation coefficient.

[0066] Step 5: According to the resource call associated value set, determine the weight value of each task execution feature vector in the task execution feature vector set to obtain the task execution weight feature vector set. The task dependency weight feature vector in the task dependency weight feature vector set may be a task dependency feature vector with a weight value.

[0067] As an example, the execution subject may perform feature multiplication on the task execution feature vector set and the call resource associated value set to obtain a task execution weight feature vector set.

[0068] The sixth step is to construct a dependency vector for the above-mentioned task dependency information set to obtain a task dependency feature vector set. Among them, the task dependency feature vector in the above-mentioned task dependency feature vector set can represent the task dependency information between the above-mentioned business data processing task sets. The task dependency feature vector in the above-mentioned task dependency feature vector set is a one-dimensional vector. In practice, the above-mentioned execution subject can use a graph convolution model to construct a dependency vector for the above-mentioned task dependency information set to obtain a task dependency feature vector set.

[0069] In the seventh step, feature concatenation is performed on the above task dependency feature vector set and the above task execution weight feature vector set to obtain a concatenated task feature vector set.

[0070] In the eighth step, the above-mentioned spliced ​​task feature vector set is input into the task anomaly encoder included in the task resource call anomaly detection model to obtain the task resource call hidden feature vector set, wherein the above-mentioned task resource call anomaly detection model also includes: a task call timing extraction model and a task anomaly decoder. Among them, the above-mentioned task resource call anomaly detection model can be a variational autoencoder that extracts the dynamic dependency relationship of the spliced ​​task feature vector included in the input spliced ​​task feature vector set. The above-mentioned task anomaly encoder can be a model that maps the input spliced ​​task feature vector to a low-dimensional space and learns the feature information of the spliced ​​task feature vector so that the task anomaly decoder can reconstruct the feature. For example, the above-mentioned task anomaly encoder can be an encoding network based on a self-attention mechanism. The above-mentioned task call timing extraction model can be a timing model that extracts the timing information of the feature vector output by the above-mentioned task anomaly encoder. The above-mentioned task call timing extraction model can be a two-layer gated recurrent unit (GRU). The above-mentioned task anomaly decoder can be a decoder that reconstructs the spliced ​​task feature vector set according to the feature vector input by the above-mentioned task call timing extraction model. The above-mentioned task anomaly decoder can be a decoder constructed with at least 3 layers of fully connected layers. The training of the task resource call anomaly detection model may be unsupervised training through a loss function. The loss function may be a confidence lower limit loss function composed of the reconstructed spliced ​​task feature vector set and the reconstruction error and KL divergence of the spliced ​​task feature vector set. The KL divergence may characterize that the distribution of the task resource call potential feature vector set generated by the task resource call anomaly detection model is close to a standard normal distribution.

[0071] The ninth step is to determine the mean and standard deviation of each task resource call hidden feature vector in the task resource call hidden feature vector set as the task resource call mean set and the task resource call standard deviation set. In practice, the execution subject may first input the task resource call hidden feature vector set into the Softmax activation function to obtain the task resource call feature vector set. Then, the task resource call feature vector set is input into a multi-layer perceptron to obtain the task resource call mean set and the task resource call standard deviation set.

[0072] In the tenth step, a task resource call potential feature vector set is obtained based on the task resource call mean set and the task resource call standard deviation set. The task resource call potential feature vector in the task resource call potential feature vector set can represent the low-dimensional abstract internal structure and characteristics of the concatenated task feature vector set captured by the task anomaly encoder.

[0073] As an example, the execution subject may first reparameterize the task resource call mean set and the task resource call standard deviation set to obtain a random feature vector set. Among them, the random feature vectors in the random feature vector set may be feature vectors randomly sampled from a standard normal distribution. Then, each random feature vector in the random feature vector set and the task resource call standard deviation set are determined to be element-wise multiplied with the random feature vector to obtain a random task feature vector set. Finally, each random task feature vector in the random task feature vector set and the task resource call mean set are feature-added to obtain a task resource call potential feature vector set.

[0074] In the eleventh step, the task resource call potential feature vector set and the historical task execution resource call information set are input into the task call timing extraction model to obtain the timing task resource call potential feature vector set, wherein the historical task execution resource call information set may be a call information set located in the time period before the task execution resource call information set. The time period may be consistent with the time period corresponding to the task execution resource call information set. The timing task resource call potential feature vector in the timing task resource call potential feature vector set may characterize the timing information of the task resource call potential feature vector set and the historical task execution resource call information set.

[0075] In the twelfth step, the above-mentioned time-series task resource call potential feature vector set is input into the above-mentioned task anomaly decoder to obtain the task call reconstruction feature vector set.

[0076] Step 13: Determine the feature similarity value of each task call reconstruction feature vector in the task call reconstruction feature vector set and the spliced ​​task feature vector corresponding to the task call reconstruction feature vector in the spliced ​​task feature vector set to obtain a feature similarity value set. The feature similarity value in the feature similarity value set may be the Euclidean distance between the task call reconstruction feature vector and the corresponding spliced ​​task feature vector.

[0077] Step 14: Filter out at least one feature similarity value greater than or equal to a preset feature similarity threshold from the feature similarity value set to obtain a target feature similarity value set. The preset feature similarity threshold may be a maximum value of a preset Euclidean distance. For example, the preset feature similarity threshold may be 0.8.

[0078] In the fifteenth step, each task execution resource call information and each task dependency information set corresponding to the above target feature similarity value set are determined as an abnormal data processing task information set, and the above business data processing task set is executed according to the abnormal data processing task information set to ensure the stability of the operating system that executes the business data processing task set and the task execution efficiency.

[0079] The above technical solution and its related contents, combined with step "step 108" as an inventive point of the embodiment of the present disclosure, solve the technical problem 2 mentioned in the background technology "due to the use of task scheduling log information as the input of the model, the calling structure of the business data processing task is not taken into account, and the multivariate time series anomaly detection model only detects anomalies within a period of time, and does not take into account the spatial correlation between different time series, resulting in low anomaly detection accuracy, increased occurrence of missed detection and false detection, prolonged task execution time and reduced stability and security of the system executing the business data processing task set". The factors that lead to low anomaly detection accuracy, increased occurrence of missed detection and false detection, prolonged task execution time and reduced stability and security of the system executing the business data processing task set are often as follows: due to the use of task scheduling log information as the input of the model, the calling structure of the business data processing task is not taken into account, and the multivariate time series anomaly detection model only detects anomalies within a period of time, and does not take into account the spatial correlation between different time series. If the above factors are solved, the effect of improving anomaly detection accuracy, reducing missed detection and false detection, shortening task execution time, and improving stability and security of the system executing the business data processing task set can be achieved. In order to achieve this effect, the present disclosure first obtains the task execution resource call information set corresponding to the task execution exception and performs standardized processing, which can avoid the problem of low accuracy in determining the abnormal data processing task information due to different calculation units. Secondly, the semantic features of the standardized task execution resource call information set and the association between the call resources are extracted, and the dependency feature vectors between the business data processing tasks are constructed, which can alleviate the dimension of the dependency feature vector, while considering the semantic information and structural information of the task execution resource call information set, which can improve the comprehensiveness and quality of feature extraction. Subsequently, the spliced ​​task feature vector is input into the task anomaly encoder included in the task resource call anomaly detection model, and the dynamic dependency relationship between the resources corresponding to the business data processing tasks can be extracted. Then, the above-mentioned task resource call potential feature vector set and the historical task execution resource call information set are input into the task call timing extraction model, which can extract the long-term and short-term dependencies between different time series, improve the detection accuracy and reduce the occurrence of false detection and missed detection. Finally, the above-mentioned time-series task resource call potential feature vector set is input into the above-mentioned task exception decoder, and the abnormal data processing task information set is determined according to the similarity value of the feature vectors before and after reconstruction, and the root cause of the abnormal data processing task information set is located and resources are allocated to smoothly execute the above-mentioned business data processing task set, which can shorten the task execution time and improve the stability and security of the system that executes the business data processing task set.

[0080] Step 105 , performing abnormal root cause location on each abnormal data processing task information in the abnormal data processing task information set to generate task abnormal root cause location information and obtain a task abnormal root cause location information set.

[0081] In some embodiments, the above-mentioned execution subject may locate the root cause of the abnormality for each abnormal data processing task information in the above-mentioned abnormal data processing task information set to generate task abnormality root cause location information and obtain a task abnormality root cause location information set. Among them, the task abnormality root cause location information in the above-mentioned task abnormality root cause location information set may be the task execution index information that causes the above-mentioned task scheduling information to execute abnormally. The above-mentioned task execution index information may include but is not limited to at least one of the following: CPU utilization, memory occupancy rate and task response time. In practice, the above-mentioned execution subject may use the PageRank algorithm and the random walk algorithm to locate the root cause of the abnormality for each abnormal data processing task information in the above-mentioned abnormal data processing task information set to generate task abnormality root cause location information and obtain a task abnormality root cause location information set.

[0082] In some optional implementations of some embodiments, the above-mentioned performing abnormal root cause location on each abnormal data processing task information in the above-mentioned abnormal data processing task information set to generate task abnormal root cause location information and obtain the task abnormal root cause location information set may include the following steps:

[0083] The first step is to determine the inverse matrix of the covariance matrix of the time series task resource call potential feature vector set corresponding to the above abnormal data processing task information set as the abnormal covariance inverse matrix.

[0084] The second step is to determine the feature difference between any two time series task resource call potential feature vectors in the time series task resource call potential feature vector set corresponding to the above abnormal data processing task information set, and obtain the feature difference feature vector set.

[0085] The third step is to determine the ratio of the first preset value and the arithmetic square root of the product of each characteristic difference eigenvector in the characteristic difference eigenvector set, the inverse abnormal covariance matrix and the inverse eigenvector of each characteristic difference eigenvector as the task execution dependency distance value, and obtain the task execution dependency distance value set. Among them, the task execution dependency distance value can characterize the dependency relationship between the business data processing task information included in the business data processing task information set. The first preset value can be a pre-set value. For example, the first preset value can be 1.

[0086] The fourth step is to determine the initial task node abnormality value of each business data processing task information in the business data processing task information set according to the abnormal data processing task information set and the task execution dependency distance value set, and obtain the initial task node abnormality value set. The initial task node abnormality value can represent the degree to which the business data processing task information is determined to be abnormal.

[0087] As an example, the execution subject may first select at least one task execution dependency distance value greater than or equal to a preset dependency distance value threshold from the task execution dependency distance value set. The preset dependency distance value threshold may be a pre-set maximum value for determining whether there is a dependency relationship between any two business data processing tasks. Then, the spectrum method (RussellRao) is used to determine the initial task node abnormal value of each business data processing task information in the business data processing task information set according to the abnormal data processing task information set and at least one task dependency relationship information corresponding to the at least one task execution dependency distance value, and obtain the initial task node abnormal value set.

[0088] The fifth step is to add each initial task node abnormal value in the above initial task node abnormal value set and the feature similarity value corresponding to the above initial task node abnormal value in the above feature similarity value set as the task node abnormal value to obtain the task node abnormal value set.

[0089] Step 6: Encode the task node abnormal value set to obtain a task node abnormal feature vector set. The task node abnormal feature vector in the task node abnormal feature vector set can represent the abnormal information of the business data processing task information. The encoding process can be embedded encoding using the word2vec algorithm.

[0090] In the seventh step, the task node abnormal feature vector set, the task execution dependency feature vector set and the time series task resource call potential feature vector set corresponding to the abnormal data processing task information set are feature spliced ​​to obtain the spliced ​​task abnormal feature vector set.

[0091] In the eighth step, the above-mentioned spliced ​​task abnormality feature vector set is input into the first abnormality localization graph convolutional network included in the task execution abnormality root cause localization model to obtain the first task node root cause localization feature vector set, wherein the above-mentioned task execution abnormality root cause localization model also includes: a second abnormality localization graph convolutional network, a third abnormality localization graph convolutional network and a fully connected layer. Among them, the first task node root cause localization feature vector in the above-mentioned first task node root cause localization feature vector set can represent the semantic information of the above-mentioned business data processing task information and the dependency structure information of the business data processing task information set other than the business data processing task information. The above-mentioned task execution abnormality root cause localization model can be a graph convolutional neural network model that learns the dynamic dependency relationship between each spliced ​​task abnormality feature vector in the input spliced ​​task abnormality feature vector set and the spliced ​​task abnormality feature vector with a direct connection relationship to output the abnormal root cause localization information. The above-mentioned first abnormality localization graph convolutional network, the above-mentioned second abnormality localization graph convolutional network, and the above-mentioned third abnormality localization graph convolutional network can be a graph network composed of a convolutional layer, a layer containing a RELU activation function, and a DropOut layer.

[0092] In the ninth step, the root cause positioning feature vector set of the first task node is input into the second anomaly positioning graph convolutional network to obtain the root cause positioning feature vector set of the second task node.

[0093] In the tenth step, the root cause positioning feature vector set of the second task node is input into the third anomaly positioning graph convolutional network to obtain the root cause positioning feature vector set of the third task node.

[0094] In the eleventh step, the root cause location feature vector set of the third task node is input into the fully connected layer to obtain the task anomaly root cause location information set.

[0095] Step 106 , based on the task abnormality root cause location information set, perform task resource allocation processing on the abnormal data processing task information set and the remaining data processing task information set to obtain a task resource allocation information set.

[0096] In some embodiments, the execution subject may perform task resource allocation processing on the abnormal data processing task information set and the remaining data processing task information set according to the task abnormality root cause location information set to obtain a task resource allocation information set, wherein the remaining data processing task information set is a task information set in the business data processing task information set that is located after the abnormal data processing task information set and depends on it. The task resource allocation information following the task resource allocation information set may be the resources allocated to the abnormal data processing task information to ensure the normal execution of the abnormal data processing task corresponding to the abnormal data processing task information.

[0097] As an example, the execution subject may utilize a resource allocation graph algorithm to perform task resource allocation processing on the abnormal data processing task information set according to the task abnormality root cause location information set to obtain a task resource allocation information set.

[0098] Step 107 , re-perform task scheduling processing on the abnormal data processing task information set and the remaining data processing task information set according to the task resource allocation information set to obtain target task scheduling information.

[0099] In some embodiments, the execution subject may re-schedule the abnormal data processing task information set and the remaining data processing task information set according to the task resource allocation information set to obtain target task scheduling information. The target task scheduling information may be scheduling information of the execution order and status of the abnormal data processing task corresponding to the abnormal data processing task information set and the remaining data processing task information corresponding to the remaining data processing task information set.

[0100] As an example, the execution subject may utilize the DAG task scheduling algorithm to re-schedule the abnormal data processing task information set and the remaining data processing task information set according to the task resource allocation information set to obtain target task scheduling information.

[0101] In some optional implementations of some embodiments, the above-mentioned re-scheduling of the above-mentioned abnormal data processing task information set and the remaining data processing task information set according to the above-mentioned task resource allocation information set to obtain the target task scheduling information may include the following steps:

[0102] The first step is to generate an initial gray wolf population based on the above-mentioned abnormal data processing task information set and the above-mentioned remaining data processing task information set, wherein each initial gray wolf individual in the above-mentioned initial gray wolf population may have a gray wolf upper bound value and a gray wolf lower bound value. Each initial gray wolf individual in the above-mentioned initial gray wolf population represents a task scheduling scheme for rationally allocating resources and execution order for the above-mentioned abnormal data processing task information and the above-mentioned remaining data processing task information set. The above-mentioned gray wolf upper bound value may be the maximum value of the initial gray wolf individual. The above-mentioned gray wolf lower bound value may be the minimum value of the initial gray wolf individual.

[0103] As an example, the execution subject may utilize a chaos algorithm to initialize the abnormal data processing task information set and the remaining data processing task information set to obtain an initial gray wolf population.

[0104] The second step is to generate an initial reverse gray wolf population for the initial gray wolf population according to the gray wolf upper bound value set and the gray wolf lower bound value set. The initial reverse gray wolf population may be a population opposite to the initial gray wolf population.

[0105] As an example, the execution subject may perform the following determination steps for each initial gray wolf individual in the initial gray wolf population: determine the sum of the gray wolf upper bound value and the gray wolf lower bound value of the initial gray wolf individual as the gray wolf individual value. Determine the difference between the gray wolf individual value and the random initial value corresponding to the initial gray wolf individual as the initial reverse gray wolf individual.

[0106] The third step is to generate a task scheduling time fitness function on the task resource allocation information set for the abnormal data processing task information set and the remaining data processing task information set. The task scheduling time fitness function may be to minimize the execution time of the abnormal data processing task information set and the remaining data processing task information set on the task resource allocation information set.

[0107] The fourth step is to input the above-mentioned initial gray wolf population and the above-mentioned initial reverse gray wolf population into the above-mentioned task scheduling time fitness function to obtain an initial gray wolf fitness value set and an initial reverse gray wolf fitness value set.

[0108] The fifth step is to screen each initial gray wolf fitness value in the initial gray wolf fitness value set and the initial reverse gray wolf fitness value corresponding to the initial gray wolf fitness value in the initial reverse gray wolf fitness value set to obtain a target gray wolf fitness value set, and determine at least one initial gray wolf individual and at least one initial reverse gray wolf individual corresponding to the target gray wolf fitness value set as the target gray wolf population. The target gray wolf fitness value in the target gray wolf fitness value set may be the fitness value with the smallest value among the initial gray wolf fitness value and the corresponding initial reverse gray wolf fitness value.

[0109] Step 6. Based on the target gray wolf population, perform the following determination steps:

[0110] Sub-step 1, generating a first gray wolf individual, a second gray wolf individual and a third gray wolf individual according to the target gray wolf fitness value set. The first gray wolf individual may be the target gray wolf individual corresponding to the target gray wolf fitness value with the largest value in the target gray wolf fitness value set. The second gray wolf individual may be the target gray wolf individual corresponding to the target gray wolf fitness value with the largest value after removing the target gray wolf fitness value corresponding to the first gray wolf individual from the target gray wolf fitness value set. The third gray wolf individual may be the target gray wolf individual corresponding to the target gray wolf fitness value with the largest value after removing the target gray wolf fitness value group corresponding to the first gray wolf individual and the second gray wolf individual from the target gray wolf fitness value set.

[0111] As an example, the execution subject may first sort the target gray wolf fitness value set from large to small to obtain a target gray wolf fitness value sequence. Then, the target gray wolf fitness values ​​at the first position, the second position, and the third position are selected from the target gray wolf fitness value sequence as the first gray wolf individual, the second gray wolf individual, and the third gray wolf individual.

[0112] Sub-step 2, removing the first gray wolf individual, the second gray wolf individual and the third gray wolf individual from the target gray wolf population to obtain a gray wolf population after removal.

[0113] Sub-step 3, determining the individual distance values ​​of the first gray wolf individual, the second gray wolf individual, the third gray wolf individual and each removed gray wolf individual in the removed gray wolf population, and obtaining the first individual distance value set, the second individual distance value set and the third individual distance value set. Among them, the first individual distance value in the above-mentioned first individual distance value set can be the product of the position of the first gray wolf individual and the random factor, and the difference between the removed gray wolf individual. The above-mentioned random factor can be any value in the range of [0, 2]. The second individual distance value in the above-mentioned second individual distance value set can be the product of the position of the second gray wolf individual and the random factor, and the difference between the removed gray wolf individual. The third individual distance value in the above-mentioned third individual distance value set can be the product of the position of the third gray wolf individual and the random factor, and the difference between the removed gray wolf individual.

[0114] Sub-step 4, determining the number of times the above determination step has been executed.

[0115] Sub-step 5, updating the individual search convergence factor to obtain the updated individual search convergence factor. The individual search convergence factor can characterize the search step length of the initialized gray wolf individual. The individual search convergence factor can be: First, determine the difference between the power of the ratio of the number of times e has been executed and the preset execution threshold and the first preset value as the first value. The preset execution threshold can be the maximum value that can be executed by the preset determination step. Secondly, determine the difference between e and the first preset value as the second value. Then, determine the ratio of the first value to the second value and the product of the second preset value as the third value. The second preset value can be a preset value. For example, the second preset value can be 2. Finally, determine the difference between the second preset value and the third value as the individual search convergence factor.

[0116] Sub-step 6, based on the first individual distance value set, the second individual distance value set, the third individual distance value set and the updated individual search convergence factor, the position of each removed gray wolf individual in the removed gray wolf population is updated to obtain the updated removed gray wolf population.

[0117] As an example, the execution subject may perform the following update steps for each removed gray wolf individual in the removed gray wolf population: first, determine the difference between the second preset value, the product of the updated individual search convergence factor and the random factor, and the updated individual search convergence factor as the individual search factor. Secondly, determine the difference between the first gray wolf individual and the product of the individual search factor and the first individual distance value as the first removed gray wolf position. Subsequently, determine the difference between the second gray wolf individual and the product of the individual search factor and the second individual distance value as the second removed gray wolf position. Then, determine the difference between the third gray wolf individual and the product of the individual search factor and the third individual distance value as the third removed gray wolf position. Finally, determine the average of the first removed gray wolf position, the second removed gray wolf position, and the third removed gray wolf position as the updated removed gray wolf individual.

[0118] Sub-step 7, inputting the updated gray wolf population without the gray wolf population into the above-mentioned task scheduling time fitness function to obtain an updated gray wolf fitness value set.

[0119] Sub-step 8, the target gray wolf fitness value corresponding to the first gray wolf individual, the target gray wolf fitness value corresponding to the second gray wolf individual, the target gray wolf fitness value corresponding to the third gray wolf individual, and the updated gray wolf fitness value set are screened to obtain the updated first gray wolf individual, the updated second gray wolf individual, and the updated third gray wolf individual. Among them, the updated first gray wolf individual, the updated second gray wolf individual, and the updated third gray wolf individual can be the fitness values ​​that are ranked in the top three after sorting from large to small in the target gray wolf fitness value corresponding to the first gray wolf individual, the target gray wolf fitness value corresponding to the second gray wolf individual, the target gray wolf fitness value corresponding to the third gray wolf individual, and the updated gray wolf fitness value set.

[0120] Sub-step 9, performs Cauchy mutation update on the updated first gray wolf individual to obtain the first gray wolf individual after Cauchy mutation. In practice, the execution subject may first determine the product of the updated individual search convergence factor and the Cauchy distribution random number as the first Cauchy value. The Cauchy distribution random number may be a random number centered at 0 and having a scale parameter of 1 that conforms to the Cauchy distribution. Then, the sum of the first Cauchy value and the first preset value is determined as the second Cauchy value. Finally, the product of the second Cauchy value and the updated first gray wolf individual is determined as the first gray wolf individual after Cauchy mutation.

[0121] Sub-step 10, in response to determining that the number of executions is greater than or equal to the preset execution threshold, the task scheduling information corresponding to the first gray wolf individual after the Cauchy mutation is determined as the target task scheduling information, and the abnormal value processing task corresponding to the abnormal data processing task information set and the remaining data processing task corresponding to the remaining data processing task information set are executed according to the target task scheduling information. The preset execution threshold may be a pre-set maximum value that can be executed in the determination step. For example, the preset execution threshold may be 200.

[0122] The seventh step, in response to determining that the number of executions is less than the above-mentioned preset execution threshold, the first gray wolf individual after the Cauchy mutation, the second gray wolf individual after the update, the third gray wolf individual after the update, and the gray wolf population removed after the update are determined as the target gray wolf population, and the sum of the number of executions and the first preset value is determined as the number of executions, so as to execute the above-mentioned determination step again.

[0123] The above technical scheme and its related contents, as an inventive point of an embodiment of the present disclosure, solve the technical problem three mentioned in the background technology: "Since the gray wolf optimization algorithm adopts random initialization to obtain the initial population, there is a lot of randomness, which easily causes the algorithm's optimization efficiency to be low as the number of executions increases, and it falls into the local optimal solution. The allocation of business data processing task execution resources is unreasonable, resulting in low business data processing task execution efficiency, increased task execution time, waste of task execution resources and poor user experience." The factors that lead to low business data processing task execution efficiency, increased task execution time, waste of task execution resources and poor user experience are often as follows: Since the gray wolf optimization algorithm adopts random initialization to obtain the initial population, there is a lot of randomness, which easily causes the algorithm's optimization efficiency to be low as the number of executions increases, and it falls into the local optimal solution. The allocation of business data processing task execution resources is unreasonable. If the above factors are solved, the effect of improving the execution efficiency of business data processing tasks, shortening task execution time, reducing the waste of task execution resources and improving poor user experience can be achieved. In order to achieve this effect, the present invention first determines the initial reverse gray wolf population of the initial gray wolf population, and screens the initial gray wolf population and the initial reverse gray wolf population according to the initial gray wolf fitness value set and the initial reverse gray wolf fitness value set to obtain the target gray wolf population, which can improve the quality of the target gray wolf population, so that the target gray wolf population is distributed as evenly as possible in the search space, which is convenient for improving the convergence rate of the target gray wolf population and reducing the number of subsequent executions. Secondly, according to the first gray wolf individual, the second gray wolf individual, the third gray wolf individual and the individual search convergence factor, the gray wolf population after removal is updated. Since the individual search convergence factor has a nonlinear convergence property, it has a lower decay rate in the early stage of the determination step cycle, which can improve the global search performance of the gray wolf population, which is convenient for better optimization of the global optimal solution, and has a higher decay rate in the later stage of the determination step cycle, which can improve the local search performance of the gray wolf population, which is convenient for better optimization of the local optimal solution. Then, the Cauchy mutation update is performed on the updated first gray wolf individual, which can improve the diversity of the position of the first gray wolf individual, obtain sufficient disturbance in the early stage of the cycle to enhance the search ability, and reduce the mutation rate in the later stage of the cycle to avoid the turbulence of the optimal solution and accelerate the convergence speed of the gray wolf population. Finally, the task scheduling information corresponding to the first gray wolf individual after the Cauchy mutation is determined as the target task scheduling information, and the abnormal value processing task set corresponding to the abnormal data processing task information set and the remaining data processing task set corresponding to the remaining data processing task information set are executed according to the above target task scheduling information. Under the task resource allocation information set, the execution resources of the abnormal value processing task set and the remaining data processing task set are reasonably allocated, which can improve the execution efficiency, shorten the execution time and reduce the waste of execution resources.

[0124] Step 108, generating a task triggering information set for the business data processing task information set according to the task scheduling information and the above target task scheduling information, and executing the business data processing task according to the task triggering information set and the task dependency information set.

[0125] In some embodiments, the execution subject may generate a task trigger information set for the business data processing task information set according to the task scheduling information and the target task scheduling information, and execute the business data processing task according to the task trigger information set and the task dependency information set. The task trigger information in the task trigger information set may be information for the scheduled execution of the business data processing task, so as to improve the automatic execution of the business data processing task.

[0126] As an example, the execution subject may extract at least one business data processing task information without a preceding dependency from the task scheduling information and the target task scheduling information. Then, the scheduled start time of the at least one business data processing task information is set as the task trigger information set. Finally, the business data processing task is executed according to the task trigger information set and the task dependency information set.

[0127] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: the data processing task scheduling method of some embodiments of the present disclosure can improve the efficiency of task execution and improve the stability of the task execution system. Specifically, the reason for the low efficiency of related task scheduling, the waste of system resources and the low user experience is that the task scheduling is performed only through the task priority of the data processing task information set, and the influencing factors considered are relatively single, which is easy to cause low task scheduling efficiency, waste of system resources and low user experience. Based on this, the data processing task scheduling method of some embodiments of the present disclosure can first obtain a business data processing task information set corresponding to the user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, user item acquisition information, and the business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks. Here, the business data processing task information set facilitates the subsequent generation of task scheduling information. Secondly, determine the task dependency relationship information set between the business data processing task information included in the business data processing task information set. Here, the execution order between the business protective gear processing tasks can be more comprehensively understood, so as to improve the task execution efficiency in the future. Again, according to the above-mentioned task dependency information set, the business data processing task set corresponding to the above-mentioned business data processing task information set is subjected to task scheduling processing to obtain task scheduling information. Here, the factors affecting task execution are fully considered, which can improve the accuracy of task scheduling information and the efficiency of task execution. Subsequently, in response to determining that the above-mentioned task scheduling information represents a task execution exception, at least one business data processing task information corresponding to the task execution exception is determined as an abnormal data processing task information set. Here, determining the abnormal data processing task information set that causes the task execution exception can improve the accuracy of determining the exception, shorten the search range of abnormality detection, and facilitate improving the efficiency of subsequent root cause location. Next, each abnormal data processing task information in the above-mentioned abnormal data processing task information set is subjected to abnormal root cause location to generate task abnormal root cause location information, and obtain a task abnormal root cause location information set. Here, the root cause of the abnormality can be accurately determined, and the time to recover the abnormality can be shortened. Afterwards, according to the above-mentioned task abnormality root cause location information set, the above-mentioned abnormal data processing task information set and the remaining data processing task information set are processed for task resource allocation to obtain a task resource allocation information set, wherein the above-mentioned remaining data processing task information set is the task information set in the above-mentioned business data processing task information set that is located after the above-mentioned abnormal data processing task information set and depends on it. Here, the resources required for task execution can be reasonably allocated to improve the smooth execution of tasks and reduce the waste of resources. Then, according to the above-mentioned task resource allocation information set, the above-mentioned abnormal data processing task information set and the remaining data processing task information set are re-tasked to obtain the target task scheduling information.Here, rescheduling can further ensure the smooth execution of the business data processing task set and improve the execution efficiency. Finally, according to the above task scheduling information and the above target task scheduling information, a task trigger information set for the above business data processing task information set is generated, and according to the above task trigger information set and the above task dependency information set, the above business data processing task is executed. Here, the smooth execution of the task can be guaranteed, the waste of operating resources can be reduced, the user experience can be improved, and the stability of the system for executing the task can be improved. It can be obtained that the data processing task scheduling method can improve the efficiency of task execution and the stability of the system for executing the task.

[0128] Further references Figure 2 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a data processing task scheduling device. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the data processing task scheduling device can be specifically applied to various electronic devices.

[0129] like Figure 2As shown, a data processing task scheduling device 200 includes: an acquisition unit 201, a first determination unit 202, a first task scheduling unit 203, a second determination unit 204, an abnormal root cause location unit 205, a task resource allocation unit 206, a second task scheduling unit 207 and an execution unit 208. The acquisition unit 201 is configured to: acquire a business data processing task information set corresponding to a user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, user item acquisition information, and the business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks. The first determination unit 202 is configured to: determine a task dependency relationship information set between the business data processing task information included in the business data processing task information set. The first task scheduling unit 203 is configured to: perform task scheduling processing on the business data processing task set corresponding to the business data processing task information set according to the task dependency relationship information set to obtain task scheduling information. The second determination unit 204 is configured to: in response to determining that the above-mentioned task scheduling information represents a task execution abnormality, determine at least one business data processing task information corresponding to the task execution abnormality as an abnormal data processing task information set. The abnormal root cause location unit 205 is configured to: perform abnormal root cause location on each abnormal data processing task information in the above-mentioned abnormal data processing task information set to generate task abnormal root cause location information and obtain a task abnormal root cause location information set. The task resource allocation unit 206 is configured to: perform task resource allocation processing on the above-mentioned abnormal data processing task information set and the remaining data processing task information set according to the above-mentioned task abnormal root cause location information set to obtain a task resource allocation information set, wherein the above-mentioned remaining data processing task information set is a task information set in the above-mentioned business data processing task information set that is located after the above-mentioned abnormal data processing task information set and depends on it. The second task scheduling unit 207 is configured to: re-perform task scheduling processing on the above-mentioned abnormal data processing task information set and the remaining data processing task information set according to the above-mentioned task resource allocation information set to obtain target task scheduling information. The execution unit 208 is configured to: generate a task trigger information set for the business data processing task information set according to the task scheduling information and the target task scheduling information, and execute the business data processing task according to the task trigger information set and the task dependency information set.

[0130] It is understandable that the units recorded in the data processing task scheduling device 200 are similar to those in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the data processing task scheduling device 200 and the units included therein, and will not be described in detail here.

[0131] Reference below Figure 3 , which shows a structural schematic diagram of an electronic device (eg, an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0132] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 are also stored. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0133] Typically, the following devices may be connected to the I / O interface 305: input devices 306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 308 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 309. The communication devices 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 3 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0134] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0135] It should be noted that the computer-readable medium in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0136] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0137] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently and not be installed in the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a business data processing task information set corresponding to the user credit business data set, wherein the above-mentioned user credit business data set includes at least one of the following: user identity information, user value flow information, user item acquisition information, and the above-mentioned business data processing task information set includes at least one of the following: dependent processing tasks, discrete trigger tasks, and timed processing tasks; determines a task dependency relationship information set between the business data processing task information included in the above-mentioned business data processing task information set; performs task scheduling processing on the business data processing task set corresponding to the above-mentioned business data processing task information set according to the above-mentioned task dependency relationship information set to obtain task scheduling information; in response to determining that the above-mentioned task scheduling information represents a task execution abnormality, determines at least one business data processing task information corresponding to the task execution abnormality as the abnormal data processing task information set; Performing abnormal root cause location on each abnormal data processing task information in the abnormal data processing task information set to generate task abnormal root cause location information and obtain a task abnormal root cause location information set; performing task resource allocation processing on the abnormal data processing task information set and the remaining data processing task information sets according to the task abnormal root cause location information set to obtain a task resource allocation information set, wherein the remaining data processing task information set is a task information set in the business data processing task information set that is located after the abnormal data processing task information set and depends on it; performing task scheduling processing on the abnormal data processing task information set and the remaining data processing task information set according to the task resource allocation information set to obtain target task scheduling information; generating a task trigger information set for the business data processing task information set according to the task scheduling information and the target task scheduling information, and executing the business data processing task according to the task trigger information set and the task dependency information set.

[0138] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0139] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an acquisition unit, a first determination unit, a first task scheduling unit, a second determination unit, an abnormal root cause location unit, a task resource allocation unit, a second task scheduling unit and an execution unit. Among them, the names of these units do not constitute a limitation on the units themselves in certain cases, for example, the acquisition unit may also be described as "a unit for acquiring a business data processing task information set corresponding to a user credit business data set".

[0141] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0142] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A data processing task scheduling method, comprising: Acquire a business data processing task information set corresponding to a user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, and user item acquisition information; the business data processing task information set includes at least one of the following: a dependent processing task, a discrete trigger task, and a timed processing task; and the user value flow information is flow information of user acquisition items; Determine a task dependency relationship information set between the business data processing task information included in the business data processing task information set; According to the task dependency information set, performing task scheduling processing on the business data processing task set corresponding to the business data processing task information set to obtain task scheduling information; In response to determining that the task scheduling information represents a task execution abnormality, determining at least one piece of business data processing task information corresponding to the task execution abnormality as an abnormal data processing task information set; Performing abnormal root cause location on each abnormal data processing task information in the abnormal data processing task information set to generate task abnormal root cause location information and obtain a task abnormal root cause location information set; According to the task abnormality root cause location information set, task resource allocation processing is performed on the abnormal data processing task information set and the remaining data processing task information set to obtain a task resource allocation information set, wherein the remaining data processing task information set is a task information set in the business data processing task information set that is located after the abnormal data processing task information set and depends on it; According to the task resource allocation information set, re-perform task scheduling processing on the abnormal data processing task information set and the remaining data processing task information set to obtain target task scheduling information; A task trigger information set for the business data processing task information set is generated according to the task scheduling information and the target task scheduling information, and the business data processing task set is executed according to the task trigger information set and the task dependency information set.

2. The method according to claim 1, wherein: The step of determining a task dependency relationship information set between the business data processing task information included in the business data processing task information set includes: Determine the task input data, task output data and task scheduling resource information of each business data processing task information in the business data processing task information set, and obtain a task input data set, a task output data set and a task scheduling resource information set; Determine, according to the task input data set and the task output data set, a task call dependency information set of the business data processing task information included in the business data processing task information set; Acquire a system resource information set for the business data processing task information set; Determine a task resource dependency information set between the business data processing task information included in the business data processing task information set according to the task scheduling resource information set, the system resource information set and the task calling dependency information set; Determining, according to the task resource dependency information set, a task time dependency information set between the business data processing task information included in the business data processing task information set; Determine task type information and task priority information of each business data processing task information in the business data processing task information set to obtain a task type information set and a task priority information set; The task time dependency information set, the task type information set and the task priority information set are determined as a task dependency relationship information set between the business data processing task information included in the business data processing task information set.

3. The method according to claim 1, wherein: The step of performing task scheduling processing on the business data processing task set corresponding to the business data processing task information set according to the task dependency information set to obtain task scheduling information includes: Constructing a topological graph for the task dependency information set to obtain a task dependency call topological graph; Performing topological sorting on the task dependency call topology graph to obtain a task dependency topology sequence set; According to the task dependency topology sequence set, the business data processing task information set is sequentially added to the executable task queue; Performing visual monitoring processing on the executable task queue to obtain a task monitoring information set; In response to determining that there is task monitoring information representing an abnormality in the task monitoring information set, determining to re-execute at least one business data processing task information corresponding to at least one task monitoring information representing an abnormality, to obtain an abnormal call information set; The abnormal call information set and at least one task monitoring information representing normality are determined as task scheduling information.

4. The method according to claim 3, wherein: In response to determining that there is task monitoring information representing an abnormality in the task monitoring information set, determining to re-execute at least one business data processing task information corresponding to at least one task monitoring information representing an abnormality, and obtaining an abnormal call information set, including: For each task monitoring information in the at least one task monitoring information, the following sending steps are performed according to the business data processing task information corresponding to the task monitoring information: Determine the number of times the business data processing task information has been executed; Re-execute the business data processing task information to obtain a re-execution result; In response to determining that the re-execution result indicates successful execution, returning task call information; In response to determining that the re-execution result indicates an execution failure, and the number of executions is greater than or equal to a preset execution threshold, sending task call log information indicating the execution failure to a target terminal; In response to determining that the re-execution result indicates an execution failure and the number of executions is less than the preset execution threshold, executing the sending step again; The obtained task call information group or the obtained task call log information group is determined as an abnormal call information set.

5. The method according to claim 1, wherein: The abnormal root cause location is performed on each abnormal data processing task information in the abnormal data processing task information set to generate task abnormal root cause location information to obtain the task abnormal root cause location information set, including: Determine the inverse matrix of the covariance matrix of the time series task resource calling the potential feature vector set corresponding to the abnormal data processing task information set as the abnormal covariance inverse matrix; Determine the feature difference between any two time series task resource call potential feature vectors in the time series task resource call potential feature vector set corresponding to the abnormal data processing task information set, and obtain a feature difference feature vector set; Determine a ratio of a first preset value to the arithmetic square root of the product of each characteristic difference eigenvector in the characteristic difference eigenvector set, an inverse abnormal covariance matrix and an inverse eigenvector of each characteristic difference eigenvector as a task execution dependency distance value, and obtain a task execution dependency distance value set; Determine the initial task node abnormality value of each business data processing task information in the business data processing task information set according to the abnormal data processing task information set and the task execution dependency distance value set, and obtain the initial task node abnormality value set; Each initial task node abnormality value in the initial task node abnormality value set and the feature similarity value corresponding to the initial task node abnormality value in the feature similarity value set are added together as the task node abnormality value to obtain a task node abnormality value set, wherein the feature similarity value set is obtained by the following steps: in response to determining that the task scheduling information represents a task execution abnormality, obtaining a task execution resource call information set corresponding to the task scheduling information; performing data standardization processing on the task execution resource call information set to obtain a standardized task execution resource call information set; performing feature extraction on the standardized task execution resource call information set to obtain a task execution feature vector set; determining the call resource association value between each business data processing task included in the task execution resource call information set to obtain a call resource association value set; according to the call resource association value set, determining the weight value of each task execution feature vector in the task execution feature vector set to obtain a task execution weight feature vector set; performing dependency vector construction on the task dependency relationship information set to obtain a task dependency feature vector set; performing feature splicing on the task dependency feature vector set and the task execution weight feature vector set to obtain a spliced ​​task feature vector set; The feature vector set is input into the task anomaly encoder included in the task resource call anomaly detection model to obtain the task resource call hidden feature vector set, wherein the task resource call anomaly detection model also includes: a task call timing extraction model and a task anomaly decoder; determining the mean and standard deviation of each task resource call hidden feature vector in the task resource call hidden feature vector set as the task resource call mean set and the task resource call standard deviation set; generating a task resource call potential feature vector set based on the task resource call mean set and the task resource call standard deviation set; comparing the task resource call potential feature vector set with the historical task execution data The source call information set is input into the task call timing extraction model to obtain a time sequence task resource call potential feature vector set, wherein the historical task execution resource call information set may be a call information set located in a time period before the task execution resource call information set; the time sequence task resource call potential feature vector set is input into the task exception decoder to obtain a task call reconstruction feature vector set; the feature similarity value between each task call reconstruction feature vector in the task call reconstruction feature vector set and the spliced ​​task feature vector corresponding to the task call reconstruction feature vector in the spliced ​​task feature vector set is determined to obtain a feature similarity value set; Encoding the task node abnormal value set to obtain a task node abnormal feature vector set; The task node abnormal feature vector set, the task execution dependency distance value set and the time series task resource call potential feature vector set corresponding to the abnormal data processing task information set are subjected to feature splicing to obtain a spliced ​​task abnormal feature vector set; Inputting the spliced ​​task abnormality feature vector set into the first abnormality localization graph convolutional network included in the task execution abnormality root cause localization model to obtain the first task node root cause localization feature vector set, wherein the task execution abnormality root cause localization model also includes: a second abnormality localization graph convolutional network, a third abnormality localization graph convolutional network and a fully connected layer; Inputting the first task node root cause location feature vector set into the second anomaly location graph convolutional network to obtain a second task node root cause location feature vector set; Inputting the second task node root cause location feature vector set into the third anomaly location graph convolutional network to obtain the third task node root cause location feature vector set; The third task node root cause location feature vector set is input into the fully connected layer to obtain a task anomaly root source information set.

6. A data processing task scheduling device, comprising: an acquisition unit, configured to acquire a business data processing task information set corresponding to a user credit business data set, wherein the user credit business data set includes at least one of the following: user identity information, user value flow information, and user item acquisition information; the business data processing task information set includes at least one of the following: a dependent processing task, a discrete trigger task, and a timed processing task; and the user value flow information is flow information of a user acquiring an item; A first determining unit is configured to determine a task dependency relationship information set between the business data processing task information included in the business data processing task information set; A first task scheduling unit is configured to perform task scheduling processing on the business data processing task set corresponding to the business data processing task information set according to the task dependency information set to obtain task scheduling information; A second determining unit is configured to, in response to determining that the task scheduling information represents a task execution abnormality, determine at least one piece of business data processing task information corresponding to the task execution abnormality as an abnormal data processing task information set; an abnormality root cause locating unit, configured to locate the abnormality root cause of each abnormal data processing task information in the abnormal data processing task information set, so as to generate task abnormality root cause locating information and obtain a task abnormality root cause locating information set; A task resource allocation unit is configured to perform task resource allocation processing on the abnormal data processing task information set and the remaining data processing task information set according to the task abnormality root cause location information set to obtain a task resource allocation information set, wherein the remaining data processing task information set is a task information set in the business data processing task information set that is located after the abnormal data processing task information set and depends on it; A second task scheduling unit is configured to re-schedule the abnormal data processing task information set and the remaining data processing task information set according to the task resource allocation information set to obtain target task scheduling information; The execution unit is configured to generate a task trigger information set for the business data processing task information set according to the task scheduling information and the target task scheduling information, and to execute the business data processing task set according to the task trigger information set and the task dependency information set.

7. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

8. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images