A task delay analysis processing method and device

Abstract

The application discloses a task delay analysis processing method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: in response to triggering of task delay analysis, obtaining basic information of a task to be detected; determining whether the task to be detected is delayed according to the basic information of the task to be detected; in the case that the task to be detected is delayed, determining a use coefficient of a computing resource index of a cluster and / or a queue to which the task to be detected belongs; when the use coefficient of the computing resource index is less than a first threshold value, confirming that the computing resource is the cause of task delay; and when the use coefficient of the computing resource index is greater than or equal to the first threshold value, confirming that the computing resource is not the cause of task delay. The embodiment improves the speed and convenience of task delay analysis, and has high efficiency without manual analysis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular to a task delay analysis processing method and device. BACKGROUND

[0002] In the process of constructing enterprise-level data warehouse, from data extraction / collection task to final data-to-internal / external service task construction, the dependence is complex, and the core task of the basic detail layer is the most front link in the whole data link, so it is particularly important to ensure its stable and efficient operation. Under the condition of resource control and production cost, it is crucial to avoid core task delay and ensure stable operation of the task. In order to avoid task delay as much as possible, it is necessary to analyze the reasons for task delay, so as to reduce task delay according to the reasons.

[0003] In the process of implementing the present application, the inventors have found that at least the following problems exist in the prior art:

[0004] The task delay analysis method based on computing resources is complex, mainly relies on manual work, takes more time, cannot quickly and conveniently locate the cause of task delay, and is low in efficiency. SUMMARY

[0005] Therefore, the embodiments of the present application provide a task delay analysis processing method and device, which can locate the cause of task delay more quickly and conveniently compared with the prior art.

[0006] To achieve the above object, according to an aspect of the embodiments of the present application, a task delay analysis processing method is provided, comprising:

[0007] In response to triggering of task delay analysis, basic information of a to-be-detected task is acquired;

[0008] According to the basic information of the to-be-detected task, it is determined whether the to-be-detected task is delayed;

[0009] In the case that the to-be-detected task is delayed, a usage coefficient of a computing resource index of a cluster and / or a queue to which the to-be-detected task belongs is determined; when the usage coefficient of the computing resource index is less than a first threshold, it is confirmed that the computing resource is the cause of task delay; when the usage coefficient of the computing resource index is greater than or equal to the first threshold, it is confirmed that the computing resource is not the cause of task delay.

[0010] Optionally, it further comprises:

[0011] Before the basic information of the task to be detected is acquired, a time dimension of task delay analysis specified by a user is acquired; the time dimension of task delay analysis specified by the user is a current time or a specified time period; a type of task delay analysis is determined according to the time dimension; the type of task delay analysis includes real-time task delay analysis and offline task delay analysis.

[0012] Optionally, the determining whether the task to be detected is delayed according to the basic information of the task to be detected includes:

[0013] When the type of task delay analysis is real-time task delay analysis, a current running duration of the task to be detected is determined according to a running start time of the task to be detected and a current time; a ratio of the current running duration of the task to be detected to an average historical running duration of the task to be detected is determined; when the ratio is greater than or equal to a second threshold value, it is confirmed that the task to be detected is delayed; when the ratio is less than the second threshold value, it is confirmed that the task to be detected is not delayed.

[0014] Optionally, the determining whether the task to be detected is delayed according to the basic information of the task to be detected includes:

[0015] When the type of task delay analysis is offline task delay analysis, a running duration of the task to be detected is determined according to a running start time and a running end time of the task to be detected; a ratio of the running duration of the task to be detected to an average historical running duration of the task to be detected is determined; when the ratio is greater than or equal to a second threshold value, it is confirmed that the task to be detected is delayed; when the ratio is less than the second threshold value, it is confirmed that the task to be detected is not delayed.

[0016] Optionally, the determining the usage coefficient of the computing resource index of the cluster and / or the queue to which the task to be detected belongs includes:

[0017] When the type of task delay analysis is real-time task delay analysis, index values of a plurality of computing resources of the cluster and / or the queue to which the task to be detected belongs at a current time are acquired; for each of the plurality of computing resources, a usage coefficient of the computing resource index is determined according to the index value of the computing resource of the cluster and / or the queue to which the task to be detected belongs at the current time and average values of the computing resource index at a plurality of time periods in history.

[0018] Optionally, the determining the usage coefficient of the computing resource index of the cluster and / or the queue to which the task to be detected belongs includes:

[0019] In a case that the type of the task delay analysis is the offline task delay analysis, the index values of the computing resources of the cluster and / or the queue to which the to-be-detected task belongs in a specified time period are acquired; for each of the computing resources, a usage coefficient of the computing resource index is determined according to the computing resource index value of the cluster and / or the queue to which the to-be-detected task belongs in the specified time period and the average value of the computing resource index in a plurality of historical time periods.

[0020] Optionally, the method further comprises:

[0021] Optionally, the method further comprises:

[0022] Optionally, the method further comprises:

[0023] After the reason causing the task delay is determined according to the analysis result, an abnormal task is determined according to a set of tasks running in a current time and a set of tasks running in a plurality of historical time periods, and information of the abnormal task is output.

[0024] To achieve the above object, according to another aspect of embodiments of the present application, a task delay analysis device is provided, comprising:

[0025] The acquisition module is configured to acquire basic information of a to-be-detected task in response to triggering of task delay analysis.

[0026] The determination module is configured to determine whether the to-be-detected task is delayed according to the basic information of the to-be-detected task.

[0027] The analysis module is configured to determine a usage coefficient of a computing resource index of a cluster and / or a queue to which the to-be-detected task belongs in a case that the to-be-detected task is delayed; and confirm that the computing resource is the reason causing the task delay in a case that the usage coefficient of the computing resource index is less than a first threshold value; and confirm that the computing resource is not the reason causing the task delay in a case that the usage coefficient of the computing resource index is greater than or equal to the first threshold value.

[0028] To achieve the above object, according to still another aspect of embodiments of the present application, a task delay analysis electronic device is provided, comprising:

[0029] One or more processors;

[0030] A storage device configured to store one or more programs,

[0031] When the one or more programs are executed by the one or more processors, the one or more processors implement the task delay analysis processing method described above.

[0032] To achieve the above object, according to another aspect of the embodiments of the present application, there is provided a computer readable medium having stored thereon a computer program which, when executed by a processor, implements any of the above-mentioned task delay analysis processing methods.

[0033] One of the embodiments of the above-mentioned application has the following advantages or beneficial effects: by obtaining the basic information of the task to be detected, determining whether the task to be detected is delayed according to the basic information of the task to be detected, in the case that the task to be detected is delayed, analyzing the multiple computing resource indicators of the cluster and / or the queue to which the task to be detected belongs during the running of the task to be detected, to determine the cause of the task delay according to the analysis result, these processing steps realize automatic and rapid positioning of the delayed task, and automatic and rapid positioning of the cause of the task delay, overcome the problems of complex, slow and low efficiency of the task delay analysis method based on computing resources in the prior art, and further achieve the effect that the analysis of the task delay becomes more convenient and fast.

[0034] The further effects of the above-mentioned non-conventional optional mode will be described in the following in combination with the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings are used to better understand the present application, and do not constitute an improper limitation on the present application. Among them:

[0036] Figure 1 is a main flowchart of a task delay analysis processing method according to an embodiment of the present application;

[0037] Figure 2 is a flowchart of an optional task delay analysis processing method according to an embodiment of the present application;

[0038] Figure 3 is a flowchart of another optional task delay analysis processing method according to an embodiment of the present application;

[0039] Figure 4 is a flowchart of still another optional task delay analysis processing method according to an embodiment of the present application;

[0040] Figure 5 is a main module diagram of a task delay analysis processing device according to an embodiment of the present application;

[0041] Figure 6 is an exemplary system architecture diagram to which the embodiments of the present application can be applied;

[0042] Figure 7 is a structural diagram of a computer system of a mobile device or a server suitable for implementing the embodiments of the present application. Detailed Implementation

[0043] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0044] It should be noted that the embodiments of the present invention are currently mainly applied to task delay analysis scenarios, and are applicable to various systems or computer architectures, such as databases, servers, home or public computer systems, etc.

[0045] See Figure 1 The diagram shows the main flowchart of a task delay analysis and processing method provided by an embodiment of the present invention, which includes the following steps:

[0046] S101: In response to the triggering of task delay analysis, obtain basic information about the task to be detected.

[0047] S102: Determine whether the task to be detected is delayed based on the basic information of the task to be detected.

[0048] S103: When the task to be detected is delayed, determine the utilization coefficient of the computing resource index of the cluster and / or queue to which the task to be detected belongs; when the utilization coefficient of the computing resource index is less than a first threshold, confirm that the computing resource is the cause of the task delay; when the utilization coefficient of the computing resource index is greater than or equal to the first threshold, confirm that the computing resource is not the cause of the task delay.

[0049] In the above implementation, for step S101, the triggering of task delay analysis can be specified by the user or automatically specified by the computer program implemented by this method, either one or more currently running or historically running tasks. The tasks can be in the same or different clusters, and the clusters have one or more resource schedulers to manage the tasks; for example, the resource scheduler can be the Yarn resource scheduler. Furthermore, the tasks are categorized by importance level; for example, the levels can range from high to low as L0-L3, where L0 is the highest level, indicating that the task at this level is the most critical.

[0050] After the task is specified, the task information is obtained through a specific interface to obtain the task number pid, and then the subsequent step operation is facilitated. For example, the interface can be an interface connected to a certain big data platform, or an interface connected to a certain database. For example, the task information can include: the cluster to which the task belongs, the resource scheduler to which the task belongs, and the queue to which the task belongs, etc. In addition, when obtaining the task information, the program or the user can set the acquisition principle by himself, for example, the acquisition principle can be set as: only the highest level task is acquired.

[0051] For step S102, after obtaining the task information, the running duration of the to-be-detected task is determined according to the running start time and the task end time (if the task is currently running, the current time is taken) of the to-be-detected task. Then the ratio of the running duration of the to-be-detected task to the average historical running duration of the to-be-detected task is determined; when the ratio is greater than or equal to the second threshold value, it is confirmed that the to-be-detected task is delayed; when the ratio is less than the second threshold value, it is confirmed that the to-be-detected task is not delayed. For example: for a task that starts at 10:00:00 and ends at 11:00:00, the running duration of the task can be obtained by the difference between 11:00:00 and 10:00:00, which is 1 hour. According to the program query, the total duration of the task running in the previous three days is 1 hour, and the average duration is about 0.333 hours after being divided by 3. The ratio of the running duration 1 hour to the average running duration 0.333 hours in the previous three days is 3, and if the second threshold value is set to 1.3 by the user or the program automatically, it is determined that the to-be-detected task is delayed.

[0052] For step S103, if it is determined in step S102 that the to-be-detected task is delayed, the user or the program automatically determines which computing resource indicators to analyze, and obtains these indicators through the interface to read the background information or logs. The computing resource indicators include but are not limited to the memory and CPU of the cluster to which the task belongs, and / or the indicators of the queue to which the task belongs. For example, the memory indicators can include:

[0053] totalMB (total configured memory), OverAllocationMemoryMB (over-allocated memory), availbleMB (available memory), allocatedMB (allocated memory), totalOverAllocationMB (total over-allocated memory), and effectiveMemoryMB (effective memory).

[0054] For example, the CPU indicators can include:

[0055] totalVirtualCores (total virtual cores of the cluster), overAllocationVcores (over-allocated cores), availableVirtualCores (available virtual cores), allocatedVirtualCores (allocated virtual cores).

[0056] For example, the queue metrics can include:

[0057] Queue usage Memory, Queue usage vCore, Queue Apps Pending, Queue PendingContainers, Queue PendingMB, Queue Apps Running, Queue Apps Completed, Queue Apps Submitted.

[0058] After obtaining the computing resource metrics, for each of them, a usage coefficient M of the computing resource metric is determined by an operation according to the value of the computing resource metric during the current or last run of the task (exemplarily, the time representation thereof can be refined from years to seconds) and the average value of the computing resource metric over a plurality of time periods in history. When the usage coefficient M of the computing resource metric is less than a first threshold value set by a user or generated or preset by a program, it is confirmed that the computing resource is the cause of the delay of the task; when the usage coefficient of the computing resource metric is greater than or equal to the first threshold value, it is confirmed that the computing resource is not the cause of the delay of the task.

[0059] For example, assuming that the current time is March 2, 2021, 17:00:00, for a task that has now appeared a delay, which started running at 16:00:00 on March 2, 2021, and for a resource metric selected by the computer program for the analysis of the delay task, the totalMB is selected, the usage coefficient of the selected resource metric is MtotalMB, and can be determined by the following formula:

[0060] MtotalMB = (totalMB Time ) / (totalMB (day(Time-1)) + totalMB (day(Time-2)) + totalMB (day(Time-3)) )*1 / 3

[0061] Wherein, totalMB TimeRefers to the total configuration memory size used in the time period from 2021-03-02-16:00:00 to 2021-03-02-17:00:00, totalMB (day(Time-1)) Refers to the total configuration memory size used in the time period from 2021-03-01-16:00:00 to 2021-03-01-17:00:00, totalMB (day(Time-2)) Refers to the total configuration memory size used in the time period from 2021-02-28-16:00:00 to 2021-02-28-17:00:00, totalMB (day(Time-3)) Refers to the total configuration memory size used in the time period from 2021-02-27-16:00:00 to 2021-02-27-17:00:00. The operation of the formula is to obtain a ratio value by dividing the calculation resource index value of the current or last running period (the time representation can be refined from years to seconds) by the average value of the calculation resource index of a plurality of time periods in history. The obtained ratio value is the usage coefficient. Assuming that the first threshold is set to 1, and totalMB Time is 1024MB, totalMB (day(Time-1)) , totalMB (day(Time-2)) , totalMB (day(Time-3)) are all 2048MB, then the resource usage coefficient MtotalMB of the total configuration memory is 0.5, which is less than the first threshold 1, so it can be concluded that the delay reason of the analyzed task is that the total configuration memory size in the cluster to which it belongs is insufficient.

[0062] In the embodiments of the present application, the above steps are used to automatically and quickly locate the delayed task, and automatically and quickly locate the reason causing the task delay, thereby overcoming the problems of the prior art, such as complex, slow and low-efficiency task delay analysis method based on calculation resources, and further achieving the effect that the analysis of task delay becomes more convenient and fast.

[0063] Referring to Figure 2 , an optional task delay analysis processing method flowchart according to an embodiment of the present application is shown, which includes the following steps:

[0064] S201: In response to the triggering of task delay analysis, the time dimension of user-specified task delay analysis is obtained.

[0065] S202: According to the obtained time dimension, the basic information of the task to be detected is obtained.

[0066] S203: According to the basic information of the task to be detected, it is judged whether the task to be detected is delayed.

[0067] S204: In the case that the task to be detected appears delay, the multiple computing resource indexes of the cluster and / or the queue to which the task to be detected belongs during the running period are analyzed to determine the cause of the task delay according to the analysis result.

[0068] In steps S201 and S202, the user selects the time dimension specified by himself through the program interface. The time dimension includes the time period that has passed and the real-time current time, and the interface provides a selection entry for the user to select. For example, the current time is March 2, 2021 15:00:00, the user can select to input the specified time period on the interface, such as March 1, 2021 02:32:00 to March 1, 2021 03:00:00, or the user just selects to input the current time. After the user selects the time period, the backend program obtains the time dimension of the task delay analysis specified by the user, and then determines the tasks running in the selected time period or time node according to the time dimension selected by the user, and these tasks can also be selected by the user or the system automatically, to confirm which tasks are analyzed and which tasks are not analyzed.

[0069] Further, for a specific task, the type of task delay analysis in the subsequent steps is also confirmed according to the time dimension selected by the user, and the type of task delay analysis includes real-time task delay analysis and offline task delay analysis. The real-time task delay analysis refers to the delay analysis of the task running at the current time, and the offline task delay analysis refers to the delay analysis of the task running in the selected time period, and the start and end time of the time period are earlier than the current time.

[0070] In the embodiment of the application, by obtaining the time dimension of the task delay analysis specified by the user, the type of task delay analysis is determined according to the obtained time dimension, the compatibility of the offline task delay analysis function and the real-time task delay analysis function is realized, the user can select offline task delay analysis or real-time task delay analysis according to the demand, and the user experience in the task delay analysis process is improved.

[0071] In the embodiment of the application, how steps S203-S204 are performed can be referred to the description of steps S102-S103 shown in Figure 1 , and the description of other parts in steps S201-S202 can be referred to the description of step S101 shown in Figure 1 .

[0072] Referring to Figure 3 , another task delay analysis processing method flow diagram according to an embodiment of the application is shown, which includes the following steps:

[0073] S301: In response to the triggering of the task delay analysis, basic information of a task to be detected is acquired.

[0074] S302: Whether the task to be detected is delayed is determined according to the basic information of the task to be detected.

[0075] S303: In the case that the task to be detected is delayed, a computing resource index input by a user is received, and the computing resource index input by the user is verified.

[0076] S304: If the verification is passed, a plurality of computing resource indexes of a cluster and / or a queue to which the task to be detected belongs during a running period of the task to be detected are analyzed to determine a cause of the task delay according to an analysis result.

[0077] In the embodiments of the present application, how to perform steps S301-S302 can refer to the description of steps S101-S102 shown in Figure 1 , and details are not described herein.

[0078] In the embodiments of the present application, steps S303-S304 are based on step S103 shown in Figure 1 . When the user determines which computing resource indexes to analyze, the user needs to input a name of the index by using a keyboard or a touch screen or select the index according to options provided by a program. When the user inputs, the name of the index is inevitably input incorrectly, or the selected index is not suitable for the subsequent analysis in form. At this time, the verification of the computing resource index input by the user in step S303 avoids the subsequent steps from being blocked or errors from occurring due to the input error.

[0079] For example, in step S303, the verification can be performed by using a string matching function to determine whether the name of the index input by the user is included in the existing resource indexes, or by using a type parameter in the selected index to determine whether the type parameter in the selected index matches a type parameter in the index required to be analyzed in the task, so as to determine whether the selected index is suitable for the subsequent analysis in form. If the name is input correctly or is suitable in form, the verification is passed, and the part of the resource index analysis in the subsequent steps is performed. If the name is input incorrectly or is not suitable in form, the user is prompted that the index does not exist, and the user is required to input the index again.

[0080] For other parts of steps S303-S304, refer to the description of step S103 shown in Figure 1 .

[0081] Referring to Figure 4 , another optional task delay analysis processing method flowchart according to an embodiment of the present application is shown, comprising the following steps:

[0082] S401: In response to the triggering of task delay analysis, the basic information of the task to be detected is acquired.

[0083] S402: According to the basic information of the task to be detected, it is determined whether the task to be detected is delayed.

[0084] S403: In the case where the task to be detected is delayed, the multiple computing resource indicators of the cluster and / or the queue to which the task to be detected belongs during the running of the task are analyzed to determine the cause of the task delay according to the analysis result.

[0085] S404: After determining the cause of the task delay according to the analysis result, the information of the abnormal task is output.

[0086] In the embodiments of the present application, how steps S401-S403 are performed can be referred to the description of steps S101-S103 in the embodiment shown in Figure 1 , which will not be repeated here.

[0087] In the embodiments of the present application, after determining the cause of the task delay, step S404 needs to first acquire the task set U t at the current time through the interface and the background resource manager, and the task sets running in multiple time periods in the past, if multiple task sets running in time periods in the past are acquired, the multiple sets need to be de-duplicated to obtain a new set U k, . For example, assuming that the time when the current task starts is 2021-05-28 12:00:00, and the current time is 2021-05-28 13:00:00, the task sets running in multiple time periods can be the set of tasks running from 2021-05-27 12:00:00 to 2021-05-27 13:00:00, the set of tasks running from 2021-05-26 12:00:00 to 2021-05-26 13:00:00, and the set of tasks running from 2021-05-25 12:00:00 to 2021-05-25 13:00:00. U t and U k, are operated to obtain the task set U f , and the task set U fThe newly added abnormal task set is used to locate all the newly added abnormal tasks occupying resources, and the specific information of the abnormal tasks, such as the task name, the task execution time, the task occupied resources and the like, can be obtained through interface calling. The operation operation can be a difference set.

[0088] Further, based on the obtained task set, the metadata in the set, i.e., the individual task in the task set, can be accessed to call the task level of the task, and if the task level is lower than a preset level, for example, lower than the L1 level, the task can be located as a low-level abnormal task occupying resources, and the specific information of the abnormal task can be obtained through interface calling.

[0089] Further, for the located zero to multiple newly added abnormal tasks and / or low-level abnormal tasks, the abnormal tasks can be closed through calling the task manager or other manager execution operation, so that the abnormal tasks no longer occupy resources, and the core task is ensured to run normally.

[0090] Referring to Figure 5 , a main module schematic diagram of a task delay analysis processing device 500 provided by an embodiment of the present application is shown, which comprises:

[0091] An obtaining module 501 is configured to obtain basic information of a to-be-detected task in response to triggering of task delay analysis.

[0092] A judging module 502 is configured to judge whether the to-be-detected task appears delay according to the basic information of the to-be-detected task.

[0093] An analysis module 503 is configured to determine a usage coefficient of a computing resource index of a cluster and / or a queue to which the to-be-detected task belongs in a case where the to-be-detected task appears delay; confirm that the computing resource is the cause of task delay when the usage coefficient of the computing resource index is less than a first threshold value; and confirm that the computing resource is not the cause of task delay when the usage coefficient of the computing resource index is greater than or equal to the first threshold value.

[0094] Optionally, in the embodiment of the present application, the obtaining module 501 is further configured to:

[0095] obtain a time dimension of user-specified task delay analysis before obtaining the basic information of the to-be-detected task; the time dimension of user-specified task delay analysis is the current time or a specified time period; determine the type of task delay analysis according to the time dimension; the type of task delay analysis comprises real-time task delay analysis and offline task delay analysis.

[0096] Optionally, in the embodiment of the present application, the judging module 502 is configured to:

[0097] In a case that the type of the task delay analysis is the real-time task delay analysis, a current running duration of the to-be-detected task is determined according to a running start time of the to-be-detected task and a current time; a ratio of the current running duration of the to-be-detected task to an average historical running duration of the to-be-detected task is determined; in a case that the ratio is greater than or equal to a second threshold value, it is confirmed that the to-be-detected task has a delay; and in a case that the ratio is less than the second threshold value, it is confirmed that the to-be-detected task has no delay.

[0098] Optionally, in the apparatus, the judging module 502 is configured to:

[0099] In a case that the type of the task delay analysis is the offline task delay analysis, a running duration of the to-be-detected task is determined according to a running start time and a running end time of the to-be-detected task; a ratio of the running duration of the to-be-detected task to an average historical running duration of the to-be-detected task is determined; in a case that the ratio is greater than or equal to a second threshold value, it is confirmed that the to-be-detected task has a delay; and in a case that the ratio is less than the second threshold value, it is confirmed that the to-be-detected task has no delay.

[0100] Optionally, in the apparatus, the analyzing module 503 is configured to:

[0101] In a case that the type of the task delay analysis is the real-time task delay analysis, an index value of each of a plurality of computing resources of a cluster and / or a queue to which the to-be-detected task belongs at a current time is obtained; for each of the plurality of computing resources, a usage coefficient of the computing resource index is determined according to the index value of the computing resource of the cluster and / or the queue to which the to-be-detected task belongs at the current time and an average value of the computing resource index in a plurality of time periods in the past.

[0102] Optionally, in the apparatus, the analyzing module 503 is configured to:

[0103] In a case that the type of the task delay analysis is the offline task delay analysis, an index value of each of a plurality of computing resources of a cluster and / or a queue to which the to-be-detected task belongs in a specified time period is obtained; for each of the plurality of computing resources, a usage coefficient of the computing resource index is determined according to the index value of the computing resource of the cluster and / or the queue to which the to-be-detected task belongs in the specified time period and an average value of the computing resource index in a plurality of time periods in the past.

[0104] Optionally, in the apparatus, the analyzing module 503 is further configured to:

[0105] Before the usage coefficient of the computing resource index is determined, a computing resource index input by a user is received, the computing resource index input by the user is verified, and it is confirmed that the verification is passed.

[0106] Optionally, in the apparatus, the analysis module 503 is further configured to:

[0107] After determining the cause of the task delay according to the analysis result, an abnormal task is determined according to a set of tasks running in the current time and a set of tasks running in a plurality of time periods in the past, and information of the abnormal task is output.

[0108] In the embodiments of the present application, the above apparatus realizes automatic and rapid positioning of the delayed task and automatic and rapid positioning of the cause of the task delay, overcomes the problems of the prior art that the task delay analysis method based on computing resources is complex, slow and inefficient, and thus achieves the effect that the analysis of the task delay becomes more convenient and fast.

[0109] Figure 6 An exemplary system architecture 600 to which embodiments of the application can be applied is shown.

[0110] As shown in Figure 6 The system architecture 600 can include terminal devices 601, 602, 603, a network 604 and a server 605 (only examples), for example. The network 604 serves as a medium to provide communication links between the terminal devices 601, 602, 603 and the server 605. The network 604 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0111] A user can use the terminal devices 601, 602, 603 to interact with the server 605 through the network 604 to receive or send messages, etc. Various communication client applications can be installed on the terminal devices 601, 602, 603.

[0112] The terminal devices 601, 602, 603 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0113] The server 605 can be a server providing various services, such as a background management server supporting a shopping website browsed by a user using the terminal devices 601, 602, 603 (only an example).

[0114] It should be noted that the task delay analysis processing method provided by the embodiments of the present application is generally executed by the server 605, and correspondingly, the task delay analysis processing apparatus is generally provided in the server 605.

[0115] It should be understood that Figure 6The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0116] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing a terminal device of the present invention. Figure 7 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0117] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0118] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0119] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of this invention.

[0120] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0121] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code containing one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order than that shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0122] The modules described in the embodiments of the present application can be implemented in the form of software or in the form of hardware. The described modules can also be arranged in a processor, for example, a processor can be described as including an acquisition module, a judgment module and an analysis module. In some cases, the names of these modules do not constitute a limitation on the modules themselves, for example, the acquisition module can also be described as a "task information acquisition module".

[0123] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments, or can exist independently without being assembled into the device. The computer readable medium carries one or more programs, when the one or more programs are executed by the device, the device includes: acquiring basic information of a task to be detected in response to triggering of task delay analysis; judging whether the task to be detected is delayed according to the basic information of the task to be detected; determining a usage coefficient of a computing resource index of a cluster and / or a queue to which the task to be detected belongs in the case that the task to be detected is delayed; confirming that the computing resource is the cause of task delay when the usage coefficient of the computing resource index is less than a first threshold; and confirming that the computing resource is not the cause of task delay when the usage coefficient of the computing resource index is greater than or equal to the first threshold.

[0124] According to the technical scheme of the embodiments of the present application, the problem of complex, slow and low efficiency of the task delay analysis method based on computing resources in the prior art is overcome, and the effect that the analysis of task delay becomes relatively fast and convenient is achieved.

[0125] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can occur depending on design requirements and other factors. Any modification, equivalent replacement and improvement made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for analyzing and processing task delays, characterized in that, The method includes: In response to the triggering of task delay analysis, obtain basic information about the task to be detected; Determine whether the task to be detected is delayed based on the basic information of the task to be detected; If the task to be detected is delayed, determine the utilization coefficient of the computing resource index of the cluster and / or queue to which the task to be detected belongs; if the utilization coefficient of the computing resource index is less than a first threshold, confirm that the computing resource is the cause of the task delay; if the utilization coefficient of the computing resource index is greater than or equal to the first threshold, confirm that the computing resource is not the cause of the task delay. The step of determining whether the task to be detected is delayed based on the basic information of the task to be detected includes: When the task delay analysis type is real-time task delay analysis, the current runtime of the task to be detected is determined based on the start time and current time; the ratio of the current runtime of the task to be detected to the average historical runtime of the task to be detected is determined; if the ratio is greater than or equal to a second threshold, it is confirmed that the task to be detected has a delay; if the ratio is less than the second threshold, it is confirmed that the task to be detected has no delay. Alternatively, when the task delay analysis type is offline task delay analysis, the runtime of the task to be detected is determined based on the start time and end time; the ratio of the runtime of the task to be detected to the average historical runtime of the task to be detected is determined; if the ratio is greater than or equal to a second threshold, it is confirmed that the task to be detected has a delay; if the ratio is less than the second threshold, it is confirmed that the task to be detected has no delay. The utilization coefficients for determining the computing resource indicators of the cluster and / or queue to which the task to be detected belongs include: When the task delay analysis type is real-time task delay analysis, obtain the index values ​​of various computing resources of the cluster and / or queue to which the task to be detected belongs at the current time; for each of the various computing resources, determine the utilization coefficient of the computing resource index based on the computing resource index value of the cluster and / or queue to which the task to be detected belongs at the current time and the average value of the computing resource index over multiple historical time periods. When the task latency analysis type is offline task latency analysis, the index values ​​of various computing resources of the cluster and / or queue to which the task to be detected belongs are obtained within a specified time period; for each of the various computing resources, the utilization coefficient of the computing resource index is determined based on the computing resource index value of the cluster and / or queue to which the task to be detected belongs within the specified time period and the average value of the computing resource index over multiple historical time periods.

2. The method according to claim 1, characterized in that, The method further includes: Before obtaining the basic information of the task to be detected, the time dimension of the task delay analysis specified by the user is obtained; the time dimension of the task delay analysis specified by the user is the current time or a specified time period; the type of task delay analysis is determined according to the time dimension; the types of task delay analysis include: real-time task delay analysis and offline task delay analysis.

3. The method according to claim 1, characterized in that, The method further includes: Before determining the utilization coefficient of the computing resource indicators, the computing resource indicators input by the user are received, the computing resource indicators input by the user are verified, and the verification is confirmed to be successful.

4. The method of claim 1, characterized in that, The method further includes: After determining the cause of the task delay based on the analysis results, abnormal tasks are identified based on the set of tasks running in the current time and the set of tasks running in multiple time periods in history, and the information of the abnormal tasks is output.

5. A task delay analysis and processing device, characterized in that, include: The acquisition module, in response to the triggering of task delay analysis, acquires basic information about the task to be detected; The judgment module determines whether the task to be detected is delayed based on the basic information of the task to be detected. The analysis module determines the utilization coefficient of the computing resource indicators of the cluster and / or queue to which the task to be detected belongs when the task to be detected is delayed. When the utilization coefficient of the computing resource indicator is less than the first threshold, the computing resource is confirmed as the cause of the task delay. When the utilization coefficient of the computing resource indicator is greater than or equal to the first threshold, it is confirmed that the computing resource is not the cause of the task delay. When the task delay analysis type is real-time task delay analysis, the judgment module determines the current runtime of the task to be detected based on the start time of the task to be detected and the current time; and determines the ratio of the current runtime of the task to be detected to the average historical runtime of the task to be detected. When the ratio is greater than or equal to the second threshold, it is confirmed that the task to be detected is delayed; when the ratio is less than the second threshold, it is confirmed that the task to be detected is not delayed; or, when the task delay analysis type is offline task delay analysis, the runtime of the task to be detected is determined based on the start and end times of the task to be detected; the ratio of the runtime of the task to be detected to the average historical runtime of the task to be detected is determined; when the ratio is greater than or equal to the second threshold, it is confirmed that the task to be detected is delayed; when the ratio is less than the second threshold, it is confirmed that the task to be detected is not delayed. When the task delay analysis type is real-time task delay analysis, the analysis module obtains the index values ​​of various computing resources of the cluster and / or queue to which the task to be detected belongs at the current time; for each of the various computing resources, the module determines the utilization coefficient of the computing resource index based on the computing resource index value of the cluster and / or queue to which the task to be detected belongs at the current time and the average value of the computing resource index over multiple historical time periods. When the task latency analysis type is offline task latency analysis, the index values ​​of various computing resources of the cluster and / or queue to which the task to be detected belongs are obtained within a specified time period; for each of the various computing resources, the utilization coefficient of the computing resource index is determined based on the computing resource index value of the cluster and / or queue to which the task to be detected belongs within the specified time period and the average value of the computing resource index over multiple historical time periods.

6. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-4.

7. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-4.

Images