A method and device for locating task dependency bottlenecks

Abstract

The present invention discloses a method and device for locating task dependency bottlenecks, and relates to the field of big data technology. A specific implementation of the method includes: generating a task dependency chain according to the dependency relationship between tasks at different levels; querying the parent task corresponding to the target task to be optimized from the task dependency chain, and recording the target task and its corresponding parent task in a task array; wherein the parent task includes a direct parent task and an indirect parent task; generating an adjacency matrix based on the task array, and calculating each path corresponding to the target task according to the adjacency matrix; generating the character strings corresponding to each path respectively, and screening out the path with the smallest character string as the task dependency bottleneck. This implementation can solve the technical problem of being unable to quickly locate the dependency bottleneck of a task.

Description

Technical Field

[0001] The present invention relates to the field of big data technology, and in particular to a method and device for locating task dependency bottlenecks. Background Art

[0002] The big data platform is the basic service platform for big data business, providing a one-stop, self-service big data processing full-process solution for the realization of big data business. Among them, the task scheduling data development platform supports data processing and circulation. A large number of data processing tasks are distributed on the platform. From data access to target applications, it is necessary to go through multi-level task conversions, and there are direct or indirect dependencies between these tasks.

[0003] In the process of implementing the present invention, the inventors found that there are at least the following problems in the prior art:

[0004] Only the direct parent task of each layer can be searched separately. If you want to improve the timeliness of the task, you can only move forward layer by layer to find which tasks need to be optimized or solve one task when you find a problem with the timeliness of a task. This is time-consuming and labor-intensive, and it is impossible to perform predictive optimization and adjustment. Due to the excessive number of layers and branches in the task chain, the task chain is too long and a complete and direct dependency chain has not been formed. It is impossible to clearly find which tasks have a critical impact on the target application, so it is impossible to quickly locate the dependency bottleneck of the task, which makes it difficult to optimize long chain tasks. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a method and apparatus for locating a task dependency bottleneck, so as to solve the technical problem that the task dependency bottleneck cannot be quickly located.

[0006] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for locating a task dependency bottleneck is provided, comprising:

[0007] Generate task dependency chains based on the dependencies between tasks at different levels;

[0008] Querying the parent task corresponding to the target task to be optimized from the task dependency chain, and recording the target task and its corresponding parent task in a task array; wherein the parent task includes a direct parent task and an indirect parent task;

[0009] Generate an adjacency matrix based on the task array, and calculate each path corresponding to the target task according to the adjacency matrix;

[0010] Generate a string corresponding to each path, and select the path with the smallest string as the task dependency bottleneck.

[0011] Optionally, the target task and its corresponding parent task are recorded in a task array, including:

[0012] An initialization task is added to all tasks without a parent task, and the initialization task, the target task and the target task are recorded in a task array.

[0013] Optionally, in the task array, the initialization task is located at the first position, and the target task is located at the second position;

[0014] The order of tasks corresponding to the rows and columns of the adjacency matrix is ​​the same as the order of tasks in the task array.

[0015] Optionally, generating an adjacency matrix according to the task array includes:

[0016] Initialize an n*n adjacency matrix, where n is the length of the task array;

[0017] For the xth task in the task array, determine whether the direct parent task of the xth task is the same as the yth task in the task array; if so, set the value of the xth row and yth column in the adjacency matrix to 1; if not, set the value of the xth row and yth column in the adjacency matrix to 0; if the xth task has no parent task, set the value of the xth row and 1st column in the adjacency matrix to 1.

[0018] Optionally, calculating each path corresponding to the target task based on the adjacency matrix includes:

[0019] The adjacency matrix is ​​calculated using a depth-first search algorithm or a breadth-first search algorithm, thereby calculating each path corresponding to the target task.

[0020] Optionally, respectively generating a string corresponding to each of the paths, and selecting a path with the smallest string as the task dependency bottleneck, includes:

[0021] For any path, generate a character string corresponding to the path according to the parent tasks of each task in the path;

[0022] The strings corresponding to the paths are sorted in order from small to large, and the path with the smallest string is selected as the task dependency bottleneck.

[0023] Optionally, generating a character string corresponding to the path according to the parent task of each task in the path includes:

[0024] For any task in the path, the direct parent task of the task is recorded in the direct parent task array; if the direct parent task of the task is the same as the upstream task of the task, a character is generated according to the position of the direct parent task of the task in the direct parent task array, and the character is appended to the character string corresponding to the path;

[0025] Among them, in the direct parent task array, each task is arranged in descending order according to the execution end time.

[0026] Optionally, generating a character according to the position of the direct parent task of the task in the direct parent task array, and appending the character to the string corresponding to the path includes:

[0027] The subscript corresponding to the position of the direct parent task of the task in the direct parent task array is appended to the character string corresponding to the path.

[0028] In addition, according to another aspect of an embodiment of the present invention, a device for locating a task dependency bottleneck is provided, comprising:

[0029] A generation module is used to generate a task dependency chain based on the dependency relationships between tasks at different levels;

[0030] A query module, used to query the parent task corresponding to the target task to be optimized from the task dependency chain, and record the target task and its corresponding parent task in a task array; wherein the parent task includes a direct parent task and an indirect parent task;

[0031] A calculation module, used to generate an adjacency matrix based on the task array, and calculate each path corresponding to the target task according to the adjacency matrix;

[0032] The positioning module is used to generate the character strings corresponding to the paths respectively, and select the path with the smallest character string as the task dependency bottleneck.

[0033] Optionally, the query module is further used to:

[0034] An initialization task is added to all tasks without a parent task, and the initialization task, the target task and the target task are recorded in a task array.

[0035] Optionally, in the task array, the initialization task is located at the first position, and the target task is located at the second position;

[0036] The order of tasks corresponding to the rows and columns of the adjacency matrix is ​​the same as the order of tasks in the task array.

[0037] Optionally, the calculation module is further used for:

[0038] Initialize an n*n adjacency matrix, where n is the length of the task array;

[0039] For the xth task in the task array, determine whether the direct parent task of the xth task is the same as the yth task in the task array; if so, set the value of the xth row and yth column in the adjacency matrix to 1; if not, set the value of the xth row and yth column in the adjacency matrix to 0; if the xth task has no parent task, set the value of the xth row and 1st column in the adjacency matrix to 1.

[0040] Optionally, the calculation module is further used for:

[0041] The adjacency matrix is ​​calculated using a depth-first search algorithm or a breadth-first search algorithm, thereby calculating each path corresponding to the target task.

[0042] Optionally, the positioning module is further used for:

[0043] For any path, generate a character string corresponding to the path according to the parent tasks of each task in the path;

[0044] The strings corresponding to the paths are sorted in order from small to large, and the path with the smallest string is selected as the task dependency bottleneck.

[0045] Optionally, the positioning module is further used for:

[0046] For any task in the path, the direct parent task of the task is recorded in the direct parent task array; if the direct parent task of the task is the same as the upstream task of the task, a character is generated according to the position of the direct parent task of the task in the direct parent task array, and the character is appended to the character string corresponding to the path;

[0047] Among them, in the direct parent task array, each task is arranged in descending order according to the execution end time.

[0048] Optionally, the positioning module is further used for:

[0049] The subscript corresponding to the position of the direct parent task of the task in the direct parent task array is appended to the character string corresponding to the path.

[0050] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, including:

[0051] one or more processors;

[0052] a storage device for storing one or more programs,

[0053] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of the above embodiments.

[0054] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any one of the above embodiments is implemented.

[0055] An embodiment of the above invention has the following advantages or beneficial effects: because an adjacency matrix is ​​generated based on a task array, each path corresponding to the target task is calculated according to the adjacency matrix, and then the character strings corresponding to each path are respectively generated, thereby screening out the path with the smallest character string as the task dependency bottleneck, thus overcoming the technical problem of the inability to quickly locate the dependency bottleneck of the task in the prior art. For tasks with multiple levels and multiple links, the embodiment of the present invention calculates all paths of the target task through the adjacency matrix, and then quickly locates the dependency bottleneck of the task through the priority of the path, thereby quickly improving the timeliness of the link task.

[0056] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0058] Figure 1 is a schematic diagram of the main process of the method for locating task dependency bottlenecks according to an embodiment of the present invention;

[0059] Figure 2 is a schematic diagram of a task dependency chain according to an embodiment of the present invention;

[0060] Figure 3 is a schematic diagram of a task dependency chain according to another embodiment of the present invention

[0061] Figure 4 is a schematic diagram of a task dependency chain after adding an initialization task according to an embodiment of the present invention;

[0062] Figure 5 is a schematic diagram of the main process of a method for locating task dependency bottlenecks according to a reference embodiment of the present invention;

[0063] Figure 6 is a schematic diagram of main modules of an apparatus for locating task dependency bottlenecks according to an embodiment of the present invention;

[0064] Figure 7 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0065] Figure 8 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0067] The embodiment of the present invention forms a complete task dependency chain from the tasks distributed on the scheduling task data development platform, selects the target task according to the business priority, and uses the algorithm to calculate all the paths of the task. Finally, the path of the target task and the priorities corresponding to these paths are output, and the criticality of the dependency bottleneck is judged according to the priority of the path. The task with the highest path priority is the most critical task dependency bottleneck.

[0068] Figure 1 is a schematic diagram of the main process of the method for locating task dependency bottlenecks according to an embodiment of the present invention. As an embodiment of the present invention, Figure 1 As shown, the method for locating task dependency bottlenecks may include:

[0069] Step 101: Generate a task dependency chain according to the dependency relationships between tasks at different levels.

[0070] First, tasks at different levels are connected in series to form a complete task dependency chain, and the different levels of tasks are recorded. If it is a parent task, the task level is a positive number, and if it is a subtask, the task level is recorded as a negative number.

[0071] like Figure 2 As shown in the figure, task A is the indirect parent task of task D. Then a piece of data is used to record that task A is the indirect parent task of task D, and the task level is 3. At the same time, there will be a piece of data to record that the indirect child task of task A is task D, and the task level is -3. In this way, no matter which task you start from, you can easily query the direct or indirect parent task and direct or indirect child task of the task. If you want to query a certain level, you only need to query the task level.

[0072] In an embodiment of the present invention, the dependencies of all tasks on the link can be calculated according to metadata to generate a task dependency chain, and the generated task dependency chain is stored in HDFS (full name Hadoop Distributed File System, which is a distributed file system. It adopts a master-slave architecture, has high fault tolerance, can be deployed on cheap machines, and can provide high-throughput data access, which is suitable for applications with large data sets). Therefore, when troubleshooting a problem, there is no need to search upward layer by layer, but all parent and child tasks of the target application can be queried at one time.

[0073] Step 102, query the parent task corresponding to the target task to be optimized from the task dependency chain, and record the target task and its corresponding parent task in a task array, wherein the parent task includes a direct parent task and an indirect parent task.

[0074] Since there are so many link tasks distributed on the platform, we cannot select them aimlessly. We need to optimize the tasks in a targeted manner. Therefore, we first need to determine the target tasks that need to be optimized urgently based on the business priority, and use the task number of the task as the input parameter. Optionally, we can use the task that provides services to the outside as the target task. Optionally, we can further filter out the tasks with higher priority from the tasks that provide services to the outside and use them as the target tasks. This will help to easily locate the bottleneck of task dependency, so as to optimize the tasks in a targeted manner. Figure 2 As shown, task D can be determined as the target task to be optimized.

[0075] After determining the target task to be optimized according to business needs, the parent task corresponding to the target task is queried from the task dependency chain, wherein the parent task includes a direct parent task and an indirect parent task. Figure 2 As shown in , through task D, we can find that the direct parent tasks are task B and task C, and the indirect parent task is task A. Therefore, all the parent tasks of task D are task A, task B and task C. Figure 3 As shown, the direct parent task that can be found through task F is task E, and the indirect parent tasks are task A, task B, task C, and task D. Therefore, all parent tasks of task F are A, task B, task C, task D, and task E.

[0076] Optionally, the target task and its corresponding parent task are recorded in a task array, including: adding an initialization task to all tasks without a parent task, and recording the initialization task, the target task, and the target task in the task array. Since the target task may correspond to multiple different task links, the starting tasks of each task link may not be the same. In order to facilitate the calculation of the path, the embodiment of the present invention adds an initialization task to all tasks without a parent task, and records the initialization task, the target task, and all parent tasks of the target task in a task array, the length of which is recorded as n.

[0077] Optionally, in order to facilitate the calculation of the path in subsequent steps, in the task array, the initialization task is located at the first position and the target task is located at the second position, that is, the task array is [initialization task, target task, all parent tasks of the target task]. It should be noted that the position of the parent task in the task array is not restricted.

[0078] like Figure 2 As shown in the figure, in the actual production environment, task A and task B do not have the same parent task. In order to facilitate the calculation of the path, an initialization task task0 is added to these two tasks, such as Figure 4 As shown, and recorded in the task array taskList, the value of the task array is [task0,taskD,taskA,taskC,taskB], or [task0,taskD,taskB,taskC,taskA], or [task0,taskD,taskC,taskB,taskA]. The embodiment of the present invention does not limit the order of the parent tasks in the task array.

[0079] Step 103: Generate an adjacency matrix according to the task array, and calculate each path corresponding to the target task based on the adjacency matrix.

[0080] The embodiment of the present invention uses an adjacency matrix to store the data of each task node of the task dependency chain, which facilitates the calculation of each path. The order of tasks corresponding to the rows and columns of the adjacency matrix is ​​the same as the order of tasks in the task array. The adjacency matrix is ​​a two-dimensional array that represents the adjacent relationship between each vertex of the graph.

[0081] Optionally, an adjacency matrix is ​​generated according to the task array, including: initializing an n*n adjacency matrix, where n is the length of the task array; for the xth task in the task array, determining whether the direct parent task of the xth task is the same as the yth task in the task array; if so, setting the value of the xth row and yth column in the adjacency matrix to 1; if not, setting the value of the xth row and yth column in the adjacency matrix to 0; if the xth task has no parent task, setting the value of the xth row and 1st column in the adjacency matrix to 1.

[0082] by Figure 4 Take the task dependency chain shown as an example. The value of the task array is [task0,taskD,taskA,taskC,taskB]. First, initialize an n*n adjacency matrix weight. The rows and columns of the matrix represent the tasks in the task array taskList and are consistent with the order of the tasks in the task array taskList. Then query the direct parent tasks in the task array taskList in turn. Assume that a task in the task array taskList is taskList[i]. Compare the direct parent task of task taskList[i] with other tasks in the task data taskList to determine whether the direct parent task of task taskList[i] is the same as the task taskList[j] in the task data taskList. If yes, set weight[i][j] to 1. If not, set weight[i][j] to 0. If task taskList[i] has no parent dependent task, add an initialization task task0 to it, so set weight[i][0] to 1. The following matrix is ​​obtained:

[0083] Task 0 Task D Task A Task C Task B Task 0 0 0 0 0 0 Task D 0 0 0 1 1 Task A 1 0 0 0 0 Task C 0 0 1 0 0 Task B 1 0 0 0 0

[0084] Optionally, calculating each path corresponding to the target task based on the adjacency matrix includes: using a depth-first search algorithm or a breadth-first search algorithm to calculate the adjacency matrix, thereby calculating each path corresponding to the target task. Depth-first search algorithm: DFS for short, is a type of graph algorithm, and the algorithm process mainly searches each possible branch path to the point where it cannot go any deeper. Breadth-first search algorithm: BFS for short, is a blind search method that moderately expands outward through the boundaries between found and unfound vertices. All paths corresponding to the target task can be accurately calculated using a depth-first search algorithm or a breadth-first search algorithm.

[0085] Optionally, a depth-first search algorithm may be used to calculate the path, and all the obtained paths are recorded in the path array pathList. Figure 2As shown, the depth-first search algorithm is used to calculate that the target task D corresponds to two paths, one is path 1: task A->task C->task D, and the other is path 2: task B->task D. Both paths are recorded in the path array pathList.

[0086] Step 104 , respectively generate character strings corresponding to the paths, and select the path with the smallest character string as the task dependency bottleneck.

[0087] After using the algorithm to calculate all the paths of the target task, the corresponding strings of each path are generated respectively, and then the strings are sorted to filter out the smallest string. The path corresponding to the string is the task dependency bottleneck, thereby accurately locating the task dependency bottleneck.

[0088] Optionally, step 104 may include: for any path, generating a character string corresponding to the path according to the parent tasks of each task in the path; sorting the character strings corresponding to each path in ascending order, and selecting the path with the smallest character string as the task dependency bottleneck. In an embodiment of the present invention, for any path, a character string is generated according to the parent tasks of each task in the path, thereby obtaining a character string corresponding to each path, and then sorting the character strings corresponding to each path in ascending order, and selecting the smallest character string according to the sorting result, and the path corresponding to the character string is the task dependency bottleneck.

[0089] Optionally, a character string corresponding to the path is generated according to the parent task of each task in the path, including: for any task in the path, the direct parent task of the task is recorded in the direct parent task array; if the direct parent task of the task is the same as the upstream task of the task, a character is generated according to the position of the direct parent task of the task in the direct parent task array, and the character is appended to the character string corresponding to the path; wherein, in the direct parent task array, each task is arranged in descending order according to the execution end time. Specifically, the direct parent task of each task in the task array taskList is queried in turn, and the direct parent task of each task is recorded in the direct parent task array parentTaskList, and sorted in order from late to early according to the execution end time. Figure 2As shown, taking task C as an example, its corresponding direct parent task is taskA, taking task D as an example, its corresponding direct parent task array is taskB and taskC, then the value of the direct parent task array parentTaskList is [[taskA], [taskB, taskC]]. Taking task C as an example, its corresponding parent task is taskA, which is the same as its upstream task taskA, so a character is generated according to the position of taskA in the direct parent task array [taskA] of task C, and the character is appended to the string.

[0090] Optionally, characters are generated according to the position of the direct parent task of the task in the direct parent task array, and the characters are appended to the character string corresponding to the path, including: appending the subscript corresponding to the position of the direct parent task of the task in the direct parent task array to the character string corresponding to the path. In order from left to right, the subscripts corresponding to the positions of the tasks in the array are 0, 1, 2, 3, etc., so when the direct parent task is the same as the upstream task, the subscript is appended to the character string to facilitate comparison of the string size. Since the direct parent tasks corresponding to the subscripts are sorted in order from late to early according to the execution end time, the larger the subscript means the earlier the execution end time and the lower the priority. The embodiment of the present invention can compare the priority by recording the subscripts, thereby accurately locating the task dependency bottleneck.

[0091] by Figure 4 Taking the task dependency chain shown in FIG. 1 as an example, step 104 may include the following steps:

[0092] First, initialize the path priority list pathPriorityList;

[0093] Then, query each path in the path array pathList. Each path contains multiple tasks, and there is a parent-child relationship between the upstream and downstream tasks. According to the parent-child relationship of the upstream and downstream tasks, query the direct parent task array fathertaskList corresponding to the downstream task. If the direct parent task in the direct parent task array parenttaskList is the same as the upstream task, then append the subscript corresponding to the position of the direct parent task in the direct parent task array fathertaskList to the string representing the priority of the path until the priority calculation of each path is completed and stored in the path priority list pathPriorityList.

[0094] Taking path 1 as an example, path 1 is: task A->task C->task D. First, query the parent task of task C. The parent task of task C is only task A, so the path priority string is currently 0. Then query the parent task of task D. The parent tasks of task D are task B and task C. Since they are arranged in descending order according to the execution end time, the order in the direct parent task array parentTaskList is [taskB,taskC]. The direct parent task C of task D is compared with the values ​​in the array [taskB,taskC] in turn, and it is concluded that the subscript of the direct parent task array [taskB,taskC] where task C is located is 1, so the string corresponding to path 1 is 0->1. The path and its corresponding string are recorded in the path priority list pathPriorityList.

[0095] Taking path 2 as an example, path 1 is: task B->task D. The parent tasks of task D are queried. The parent tasks of task D are task B and task C. Since they are arranged in descending order according to the execution end time, the order in the direct parent task array parentTaskList is [taskB,taskC]. The direct parent task B of task D is compared with the values ​​in the array [taskB,taskC] in turn, and it is found that the subscript of the direct parent task array [taskB,taskC] where task B is located is 0, so the string corresponding to path 2 is 0. The path and its corresponding string are recorded in the path priority list pathPriorityList.

[0096] Finally, sort the strings in the path priority list pathPriorityList in order from small to large, and output the path corresponding to the smallest string and its corresponding priority, which is the task dependency bottleneck corresponding to the target task.

[0097] In the embodiment of the present invention, the priority of a task is measured by the execution end time of the task, such as Figure 2 As shown in the figure, both Task B and Task C are the direct parent tasks of Task D. Task B ends later than Task C, so Task B has a higher priority than Task C, because the later execution end time has a greater probability of causing delays in Task D. Because the end time of tasks changes every day, the priorities of different paths also change.

[0098] It should be noted that multiple paths to be optimized and their corresponding priorities can be output as needed, and these paths to be optimized are sorted according to priority, so that the operation and maintenance personnel can optimize these paths to be optimized in a targeted manner according to the priority. Generally speaking, selecting a path to be optimized with a higher priority for optimization helps to quickly improve the timeliness of link tasks. Moreover, one or more tasks with a higher priority in the path to be optimized can be selected for optimization, or all paths in the path to be optimized can be selected for optimization, according to actual analysis of the actual situation, and the embodiment of the present invention is not limited.

[0099] Optionally, all paths corresponding to the target task and their corresponding priorities can also be output, and the path with the highest priority is the most critical task dependency bottleneck. Figure 2 As shown in the figure, there are two paths for the target task D, and the one with higher priority is task B->task D. Therefore, it can be seen that task B is the dependency bottleneck of the target task D. Subsequent relevant personnel can make targeted optimizations based on the characteristics of task B.

[0100] According to the various embodiments described above, it can be seen that the embodiments of the present invention generate an adjacency matrix based on the task array, calculate the paths corresponding to the target task according to the adjacency matrix, and then generate the strings corresponding to each path respectively, thereby screening out the path with the smallest string as the task dependency bottleneck technical means, which solves the technical problem that the task dependency bottleneck cannot be quickly located in the prior art. For tasks with multiple levels and multiple links, the embodiments of the present invention calculate all paths of the target task through the adjacency matrix, and then quickly locate the task dependency bottleneck through the priority of the path, thereby quickly improving the timeliness of the link task.

[0101] Figure 5 is a schematic diagram of the main process of a method for locating task dependency bottlenecks according to a reference embodiment of the present invention. As another embodiment of the present invention, Figure 4 As shown, the method for locating task dependency bottlenecks may include:

[0102] Step 501: Generate a task dependency chain according to the dependency relationships between tasks at different levels.

[0103] The dependencies of all tasks on the link can be calculated based on metadata, and tasks at different levels can be connected in series to form a complete task dependency chain. The different levels of tasks are recorded. If it is a parent task, the task level is a positive number, and if it is a child task, the task level is recorded as a negative number. In this way, no matter which task you start from, you can easily query the direct or indirect parent tasks and direct or indirect child tasks of the task. If you want to query a certain level, you only need to query the task level.

[0104] Step 502: Find out the parent task corresponding to the target task to be optimized from the task dependency chain.

[0105] The task of providing external services can be used as the target task, which helps to easily locate the bottleneck of task dependency and optimize the task in a targeted manner. After determining the target task to be optimized according to business needs, the parent task corresponding to the target task is queried from the task dependency chain, where the parent task includes direct parent tasks and indirect parent tasks.

[0106] Step 503: add an initialization task to all tasks without a parent task, and record the initialization task, the target task, and the target task in a task array.

[0107] Since the target task will correspond to multiple different task links, the starting tasks of each task link may not be the same. In order to facilitate the calculation of the path, the embodiment of the present invention adds an initialization task to all tasks without a parent task, and records the initialization task, the target task, and all parent tasks of the target task in a task array taskList. The length of the task array is recorded as n. In the task array, the initialization task is located in the first position, and the target task is located in the second position, that is, the task array is [initialization task, target task, all parent tasks of the target task]. It should be pointed out that the position of the parent task in the task array is not restricted.

[0108] Step 504, initialize an n*n adjacency matrix, where n is the length of the task array.

[0109] The embodiment of the present invention uses an adjacency matrix to store the data of each task node of the task dependency chain, so as to facilitate the calculation of each path. The order of tasks corresponding to the rows and columns of the adjacency matrix is ​​the same as the order of tasks in the task array.

[0110] Step 505: Generate an adjacency matrix based on the task array.

[0111] Optionally, step 505 may include: for the xth task in the task array, determining whether the direct parent task of the xth task is the same as the yth task in the task array; if so, setting the value of the xth row and yth column in the adjacency matrix to 1; if not, setting the value of the xth row and yth column in the adjacency matrix to 0; if the xth task has no parent task, setting the value of the xth row and 1st column in the adjacency matrix to 1. Specifically, query the direct parent tasks in the task array taskList in turn, assuming that a task in the task array taskList is taskList[i], compare the direct parent task of task taskList[i] with other tasks in the task data taskList, determine whether the direct parent task of task taskList[i] is the same as task taskList[j] in the task data taskList, if yes, set weight[i][j] to 1, if not, set weight[i][j] to 0. If task taskList[i] has no parent dependent task, add an initialization task task0 to it, and set weight[i][0] to 1.

[0112] Step 506: Use a depth-first search algorithm or a breadth-first search algorithm to calculate the adjacency matrix, so as to calculate each path corresponding to the target task.

[0113] All paths corresponding to the target task can be accurately calculated through the depth-first search algorithm or the breadth-first search algorithm, and all paths are recorded in the path array pathList.

[0114] Step 507: Initialize the path priority list pathPriorityList.

[0115] Step 508: for any path, generate a character string corresponding to the path according to the parent task of each task in the path, and record each path and its corresponding character string into a path priority list pathPriorityList.

[0116] Specifically, a character string corresponding to the path is generated according to the parent tasks of each task in the path, including: for any task in the path, the direct parent tasks of each task in the task array taskList are queried in turn, and the direct parent task of each task is recorded in the direct parent task array parentTaskList, and sorted in order from late to early according to the execution end time; if the direct parent task of the task is the same as the upstream task of the task, the subscript corresponding to the position of the direct parent task of the task in the direct parent task array is appended to the character string corresponding to the path, and each path and its corresponding character string is recorded in the path priority list pathPriorityList.

[0117] Step 509 , sort the character strings corresponding to the paths in the path priority list pathPriorityList in ascending order, output each path and its corresponding priority, and select the path with the smallest character string as the task dependency bottleneck.

[0118] Generally speaking, selecting a path with a higher priority to be optimized for optimization helps to quickly improve the timeliness of link tasks. Moreover, one or more tasks with a higher priority in the path to be optimized can be selected for optimization, or all paths in the path to be optimized can be selected for optimization, according to actual analysis of the actual situation, and the embodiment of the present invention does not limit it.

[0119] In addition, the specific implementation content of the method for locating task dependency bottlenecks in a reference embodiment of the present invention has been described in detail in the method for locating task dependency bottlenecks described above, so the repeated content will not be described here.

[0120] Figure 6 is a schematic diagram of main modules of an apparatus for locating task-dependent bottlenecks according to an embodiment of the present invention, such as Figure 6 As shown, the device 600 for locating task dependency bottlenecks includes a generation module 601, a query module 602, a calculation module 603 and a positioning module 604; wherein, the generation module 601 is used to generate a task dependency chain according to the dependency relationship between tasks at different levels; the query module 602 is used to query the parent task corresponding to the target task to be optimized from the task dependency chain, and record the target task and its corresponding parent task in a task array; wherein, the parent task includes a direct parent task and an indirect parent task; the calculation module 603 is used to generate an adjacency matrix based on the task array, and calculate the paths corresponding to the target task according to the adjacency matrix; the positioning module 604 is used to generate the character strings corresponding to the paths respectively, and screen out the path with the smallest character string as the task dependency bottleneck.

[0121] Optionally, the query module 602 is further used to:

[0122] An initialization task is added to all tasks without a parent task, and the initialization task, the target task and the target task are recorded in a task array.

[0123] Optionally, in the task array, the initialization task is located at the first position, and the target task is located at the second position;

[0124] The order of tasks corresponding to the rows and columns of the adjacency matrix is ​​the same as the order of tasks in the task array.

[0125] Optionally, the calculation module 603 is further used for:

[0126] Initialize an n*n adjacency matrix, where n is the length of the task array;

[0127] For the xth task in the task array, determine whether the direct parent task of the xth task is the same as the yth task in the task array; if so, set the value of the xth row and yth column in the adjacency matrix to 1; if not, set the value of the xth row and yth column in the adjacency matrix to 0; if the xth task has no parent task, set the value of the xth row and 1st column in the adjacency matrix to 1.

[0128] Optionally, the calculation module 603 is further used for:

[0129] The adjacency matrix is ​​calculated using a depth-first search algorithm or a breadth-first search algorithm, thereby calculating each path corresponding to the target task.

[0130] Optionally, the positioning module 604 is further used for:

[0131] For any path, generate a character string corresponding to the path according to the parent tasks of each task in the path;

[0132] The strings corresponding to the paths are sorted in order from small to large, and the path with the smallest string is selected as the task dependency bottleneck.

[0133] Optionally, the positioning module 604 is further configured to:

[0134] For any task in the path, the direct parent task of the task is recorded in the direct parent task array; if the direct parent task of the task is the same as the upstream task of the task, a character is generated according to the position of the direct parent task of the task in the direct parent task array, and the character is appended to the character string corresponding to the path;

[0135] Among them, in the direct parent task array, each task is arranged in descending order according to the execution end time.

[0136] Optionally, the positioning module 604 is further configured to:

[0137] The subscript corresponding to the position of the direct parent task of the task in the direct parent task array is appended to the character string corresponding to the path.

[0138] According to the various embodiments described above, it can be seen that the embodiments of the present invention generate an adjacency matrix based on the task array, calculate the paths corresponding to the target task according to the adjacency matrix, and then generate the strings corresponding to each path respectively, thereby screening out the path with the smallest string as the task dependency bottleneck technical means, which solves the technical problem that the task dependency bottleneck cannot be quickly located in the prior art. For tasks with multiple levels and multiple links, the embodiments of the present invention calculate all paths of the target task through the adjacency matrix, and then quickly locate the task dependency bottleneck through the priority of the path, thereby quickly improving the timeliness of the link task.

[0139] It should be noted that the specific implementation content of the device for locating task-dependent bottlenecks in the present invention has been described in detail in the method for locating task-dependent bottlenecks described above, so the repeated content will not be described here.

[0140] Figure 7 An exemplary system architecture 700 is shown to which the method for locating a task dependency bottleneck or the apparatus for locating a task dependency bottleneck according to an embodiment of the present invention can be applied.

[0141] like Figure 7 As shown, system architecture 700 may include terminal devices 701, 702, 703, network 704 and server 705. Network 704 is used to provide a medium for communication links between terminal devices 701, 702, 703 and server 705. Network 704 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0142] Users can use terminal devices 701, 702, and 703 to interact with server 705 through network 704 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 701, 702, and 703, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only examples).

[0143] The terminal devices 701 , 702 , and 703 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0144] Server 705 may be a server that provides various services, such as a backend management server (for example only) that provides support for shopping websites browsed by users using terminal devices 701, 702, and 703. The backend management server may analyze and process the received data such as item information query requests, and feed back the processing results to the terminal device.

[0145] It should be noted that the method for locating the task dependency bottleneck provided in the embodiment of the present invention is generally executed by the server 705 , and accordingly, the device for locating the task dependency bottleneck is generally set in the server 705 .

[0146] It should be understood that Figure 7 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0147] Reference below Figure 8 , which shows a schematic diagram of the structure of a computer system 800 of a terminal device suitable for implementing an embodiment of the present invention. Figure 8 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0148] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the system 800 are also stored. The CPU 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0149] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0150] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the central processing unit (CPU) 801, the above-mentioned functions defined in the system of the present invention are executed.

[0151] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0152] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0153] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes a generation module, a query module, a calculation module and a positioning module, wherein the names of these modules do not constitute a limitation on the modules themselves in some cases.

[0154] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or it may exist independently and not be assembled into the device. The above computer-readable medium carries one or more programs. When the above one or more programs are executed by a device, the device implements the following method: generating a task dependency chain according to the dependency relationship between tasks at different levels; querying the parent task corresponding to the target task to be optimized from the task dependency chain, and recording the target task and its corresponding parent task in a task array; wherein the parent task includes a direct parent task and an indirect parent task; generating an adjacency matrix based on the task array, and calculating each path corresponding to the target task according to the adjacency matrix; respectively generating the character strings corresponding to each path, and screening out the path with the smallest character string as the task dependency bottleneck.

[0155] According to the technical solution of the embodiment of the present invention, because an adjacency matrix is ​​generated based on a task array, each path corresponding to the target task is calculated according to the adjacency matrix, and then the character string corresponding to each path is respectively generated, thereby screening out the path with the smallest character string as the task dependency bottleneck. Therefore, the technical problem of the inability to quickly locate the dependency bottleneck of the task in the prior art is overcome. For tasks with multiple levels and multiple links, the embodiment of the present invention calculates all paths of the target task through the adjacency matrix, and then quickly locates the dependency bottleneck of the task through the priority of the path, thereby quickly improving the timeliness of the link task.

[0156] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for locating task dependency bottlenecks, characterized in that: include: Generate task dependency chains based on the dependencies between tasks at different levels; Query the parent task corresponding to the target task to be optimized from the task dependency chain, add an initialization task to all tasks without a parent task, and record the initialization task, the target task and its corresponding parent task in a task array; wherein the parent task includes a direct parent task and an indirect parent task; Generate an adjacency matrix based on the task array, and calculate each path corresponding to the target task according to the adjacency matrix; Generate a string corresponding to each path, and select the path with the smallest string as the task dependency bottleneck.

2. The method according to claim 1, characterized in that In the task array, the initialization task is located at the first position, and the target task is located at the second position; The order of tasks corresponding to the rows and columns of the adjacency matrix is ​​the same as the order of tasks in the task array.

3. The method according to claim 2, characterized in that Generate an adjacency matrix according to the task array, including: Initialize an n*n adjacency matrix, where n is the length of the task array; For the xth task in the task array, determine whether the direct parent task of the xth task is the same as the yth task in the task array; if so, set the value of the xth row and yth column in the adjacency matrix to 1; if not, set the value of the xth row and yth column in the adjacency matrix to 0; if the xth task has no parent task, set the value of the xth row and 1st column in the adjacency matrix to 1.

4. The method according to claim 1, characterized in that: Calculating each path corresponding to the target task based on the adjacency matrix includes: The adjacency matrix is ​​calculated using a depth-first search algorithm or a breadth-first search algorithm, thereby calculating each path corresponding to the target task.

5. The method according to claim 1, characterized in that Generate the strings corresponding to the paths respectively, and select the path with the smallest string as the task dependency bottleneck, including: For any path, generate a character string corresponding to the path according to the parent tasks of each task in the path; The strings corresponding to the paths are sorted in order from small to large, and the path with the smallest string is selected as the task dependency bottleneck.

6. The method according to claim 5, characterized in that Generate a character string corresponding to the path according to the parent task of each task in the path, including: For any task in the path, the direct parent task of the task is recorded in the direct parent task array; if the direct parent task of the task is the same as the upstream task of the task, a character is generated according to the position of the direct parent task of the task in the direct parent task array, and the character is appended to the character string corresponding to the path; Among them, in the direct parent task array, each task is arranged in descending order according to the execution end time.

7. The method according to claim 5, characterized in that Generating a character according to the position of the direct parent task of the task in the direct parent task array, and appending the character to the string corresponding to the path, including: Append the subscript corresponding to the position of the direct parent task of the task in the direct parent task array to the character string corresponding to the path.

8. A device for locating task dependency bottlenecks, characterized in that: include: A generation module is used to generate a task dependency chain based on the dependency relationships between tasks at different levels; A query module, used to query the parent task corresponding to the target task to be optimized from the task dependency chain, add an initialization task to all tasks without a parent task, and record the initialization task, the target task and its corresponding parent task in a task array; wherein the parent task includes a direct parent task and an indirect parent task; A calculation module, used to generate an adjacency matrix based on the task array, and calculate each path corresponding to the target task according to the adjacency matrix; The positioning module is used to generate the character strings corresponding to the paths respectively, and select the path with the smallest character string as the task dependency bottleneck.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images