End time prediction method and apparatus, device, and storage medium

Abstract

The present disclosure relates to an end time prediction method, device, equipment and storage medium. The method comprises: when the current running state of the current monitoring task is a running state, reading a first start time of the current monitoring task from a first running feature of the current monitoring task, processing the first running feature and a second running feature of a related task of the current monitoring task by using a pre-trained running time prediction model to obtain a first running time of the current monitoring task, and then predicting an end time of the current monitoring task based on the first running time and the first start time. Thus, when the task running environment changes or the upstream task of the task is abnormal, the model is used to process the running feature to accurately determine the end time, and the end time can be dynamically determined when the monitoring time dynamically changes, thereby ensuring the reliability and flexibility of the end time prediction.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an end time prediction method, apparatus, device and storage medium. Background Technology

[0002] In big data development scenarios, task scheduling systems (such as Airflow) schedule many tasks. In order to maintain the tasks and ensure the normal output of task data, it is necessary to monitor or predict the end time of each task.

[0003] In related technologies, the historical execution time of a task is simply used to predict its end time. However, when the task's operating environment changes or its upstream tasks experience anomalies, the prediction accuracy is low, and real-time dynamic prediction of the end time cannot be achieved. Therefore, providing a method with higher accuracy and real-time dynamic prediction of the end time is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides an end time prediction method, apparatus, device, and storage medium.

[0005] In a first aspect, this disclosure provides an end-time prediction method, which includes:

[0006] Obtain the current running status of the current monitoring task at any monitoring time, wherein the current monitoring task is any one of the pre-built monitored task chains, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task;

[0007] If the current running state is in a running state, the first start time of the current monitoring task is read from the first running characteristic of the current monitoring task;

[0008] The first runtime feature and the second runtime feature of the related task are processed using a pre-trained runtime prediction model to obtain the first runtime of the current monitoring task;

[0009] Based on the first runtime and the first start time, the end time of the current monitoring task is predicted.

[0010] Secondly, this disclosure provides an end time prediction device, the device comprising:

[0011] The acquisition module is used to acquire the current running status of the current monitoring task at any monitoring time. The current monitoring task is any one of the tasks in the pre-built monitored task chain, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task.

[0012] The reading module is used to read the first start time of the current monitoring task from the first running characteristics of the current monitoring task if the current running state is a running state;

[0013] The determination module is used to process the first runtime feature and the second runtime feature of the related task using a pre-trained runtime prediction model to obtain the first runtime of the current monitoring task;

[0014] The prediction module is used to predict the end time of the current monitoring task based on the first runtime and the first start time.

[0015] Thirdly, embodiments of this disclosure also provide an electronic device, the device comprising:

[0016] processor;

[0017] Memory, used to store executable instructions;

[0018] The processor is used to read executable instructions from memory and execute the executable instructions to implement the method provided in the first aspect above.

[0019] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, wherein the storage medium stores the computer program, and when the computer program is executed by a processor, the processor causes the processor to implement the method provided in the first aspect above.

[0020] The technical solution provided in this disclosure has the following advantages compared with the prior art:

[0021] This disclosure discloses an end time prediction method, apparatus, device, and storage medium. The method obtains the current running state of a current monitoring task at any given monitoring time. The current monitoring task is any task in a pre-constructed monitored task chain, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task. If the current running state is in progress, the method reads the first start time of the current monitoring task from its first running characteristics. A pre-trained runtime prediction model is used to process the first running characteristics and the second running characteristics of the related tasks to obtain the first runtime of the current monitoring task. Based on the first runtime and the first start time, the method predicts the end time of the current monitoring task. Therefore, when the task's operating environment changes or its upstream tasks experience anomalies, the system can utilize the operational characteristics of multiple tasks with dependencies in the entire monitored task chain and combine them with a runtime prediction model to determine the first runtime of the current monitored task. Then, based on the first start time and the first runtime, the end time of the current monitored task can be determined, thereby improving the accuracy of the end time prediction. Furthermore, when the monitoring time changes, the first start time and the first runtime are redefined to obtain a new end time, achieving the effect of dynamically determining the end time of the current monitored task and ensuring the reliability and flexibility of the end time prediction. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0023] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating an end time prediction method provided in an embodiment of this disclosure;

[0025] Figure 2 A flowchart illustrating another end-time prediction method provided in this embodiment of the disclosure;

[0026] Figure 3 A schematic diagram of a tree-structured monitored task chain provided in an embodiment of this disclosure;

[0027] Figure 4 This is a logical schematic diagram of splicing feature generation provided in an embodiment of the present disclosure;

[0028] Figure 5A logical schematic diagram of a task scheduling system monitoring method provided in this embodiment of the disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of an end time prediction device provided in an embodiment of the present disclosure;

[0030] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0031] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0032] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0033] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] To accurately and dynamically predict the task's end time, the following combines... Figures 1 to 5The end-time prediction method provided in this disclosure is described below. In this disclosure, the end-time prediction method can be executed by an electronic device or a server. The electronic device may include devices with communication functions such as mobile phones, tablets, desktop computers, and laptops. The server may be a cloud server or a server cluster, or other devices with storage and computing functions. It should be noted that the following embodiments use an electronic device as the execution subject for illustrative explanation.

[0037] Figure 1 A flowchart illustrating an end-time prediction method provided in an embodiment of this disclosure is shown.

[0038] like Figure 1 As shown, the end time prediction method may include the following steps.

[0039] S110. Obtain the current running status of the current monitoring task at any monitoring time. The current monitoring task is any one of the pre-built monitored task chains, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task.

[0040] In this embodiment, when it is necessary to monitor the end time of a task in a task scheduling system (e.g., Airflow), a pre-built chain of monitored tasks is obtained, any task in the chain of monitored tasks is taken as the current monitored task, and the running status at any monitoring time is obtained as the current running status.

[0041] The monitored task chain is a complete task chain composed of various tasks in the task scheduling system. Specifically, the monitored task chain includes the currently monitored task and related tasks that depend on the currently monitored task. Optionally, related tasks can be upstream or downstream tasks of the currently monitored task.

[0042] In this embodiment, optionally, S110 specifically includes: reading the current identification information carried by the pre-configured target interface; matching the current identification information with the identification information corresponding to a plurality of pre-determined running states respectively, and determining the current running state from the plurality of pre-determined running states based on the matching result.

[0043] The target interface refers to a pre-configured virtual interface used to obtain the running status of a task. Identification information is used to characterize the running status of the task. Optionally, the running status of a task may include: waiting to run, running, and completed. The identification information corresponding to the waiting to run status is 0, the identification information corresponding to the running status is 1, and the identification information corresponding to the completed status is 2.

[0044] Specifically, starting from the last task in the monitored task chain, the current identification information is read and matched with the identification information corresponding to multiple pre-determined running states. Then, the running state of the first node task is determined based on the matching results. The above operation is repeated based on the dependency relationship until the running state of the current monitored task is read and the current running state is obtained.

[0045] S120. If the current running status is "running", read the first start time of the current monitoring task from the first running characteristic of the current monitoring task.

[0046] It is understandable that if the current identifier information is determined to be 1 in the above manner, then the current running state is determined to be running. Furthermore, during the current monitoring task, the running start time and other relevant characteristics can be stored in the database (e.g., a MySQL database), so that the first start time of the current monitoring task can be read directly from the database.

[0047] S130. Using a pre-trained runtime prediction model, process the first runtime feature and the second runtime feature of related tasks to obtain the first runtime of the current monitoring task.

[0048] In this embodiment, before predicting the end time, a runtime prediction model can be trained based on the historical characteristics of each task in the monitored task chain and the runtime of each task. After determining that the current monitored task is in a running state, the runtime prediction model can be directly used to process the first running characteristics of the current monitored task and the second running characteristics of related tasks to determine the first runtime of the current monitored task.

[0049] The first operational feature is the operational information of the current monitoring task from its start time to the current monitoring time, and the second operational feature is the operational information of the related task in the most recent round of monitoring. Optionally, both the first and second operational features may include: the amount of memory allocated to the task, the number of CPUs allocated to the task, the queue's used memory, the queue's used CPUs, the queue's maximum memory, the queue's maximum CPUs, the number of containers occupied by the queue, the number of waiting containers, the number of remaining containers, the queue environment, and allocated resources.

[0050] Therefore, when the task's operating environment changes or its upstream task malfunctions, the first runtime of the current monitored task can be determined by utilizing the operating characteristics of each task in the entire monitored task chain and combining them with a runtime prediction model.

[0051] S140. Based on the first runtime and the first start time, predict the end time of the current monitoring task.

[0052] In this embodiment, S140 specifically includes: adding the first running time and the first start time to obtain the end time of the current monitoring task.

[0053] Optionally, the end time of the current monitoring task can be calculated as follows:

[0054] T end =T start +T run

[0055] Among them, T start It is the first start time, T run It is the first runtime, T end This is the end time of the current monitoring task.

[0056] This disclosure provides an end-time prediction method. It obtains the current running state of a monitoring task at any given monitoring time. The current monitoring task is any task in a pre-constructed monitored task chain, and the monitored task chain also includes related tasks that have dependencies on the current monitoring task. If the current running state is in progress, the first start time of the current monitoring task is read from its first running feature. A pre-trained runtime prediction model is used to process the first running feature and the second running features of the related tasks to obtain the first runtime of the current monitoring task. Based on the first runtime and the first start time, the end time of the current monitoring task is predicted. Therefore, when the task's running environment changes or its upstream tasks experience anomalies, the running features of multiple dependent tasks in the entire monitored task chain, combined with the runtime prediction model, can be used to determine the first runtime of the current monitoring task. Then, based on the first start time and the first runtime, the end time of the current monitoring task can be determined, thereby improving the accuracy of end-time prediction. Furthermore, when the monitoring time changes, the first start time and the first runtime are re-determined to obtain a new end time, achieving the effect of dynamically determining the end time of the current monitoring task and ensuring the reliability and flexibility of end-time prediction.

[0057] Furthermore, to ensure the security of the scheduling system and to facilitate users in maintaining or adjusting the scheduling system as early as possible when an anomaly occurs, after executing S140, the method further includes: determining whether the end time of the current monitoring task is within the preset alarm time range; if the end time of the current monitoring task is not within the preset alarm time range, generating alarm information and sending alarm information to the user.

[0058] The alarm time range refers to the alarm timeline set for the task. It can be understood that if the end time of the current monitoring task is not within the preset alarm time range, it means that the end time of the current monitoring task has exceeded the alarm timeline, in which case an alarm message will be generated and promptly notified to the user.

[0059] Therefore, after determining the end time of the current monitoring task, the system uses an anomaly feedback mechanism to determine if the end time of the current monitoring task exceeds the alarm timeline. If it does, an alarm message is sent to the user immediately, thereby ensuring the security of the scheduling system and facilitating the user to maintain or adjust the scheduling system as early as possible.

[0060] In another embodiment of this disclosure, the current running state can also be a waiting running state or an ended running state, and the end time of the current monitoring task can be predicted using the corresponding prediction method.

[0061] In some embodiments of this disclosure, after performing S110, the method further includes:

[0062] S150. If the current running state is a waiting running state, and the relevant tasks include the upstream task of the current monitoring task, then the second start running time of the current monitoring task is determined based on the first running characteristic corresponding to the current monitoring task and the second running characteristic corresponding to the upstream task.

[0063] S160. If the current running state is a waiting running state, and the upstream task of the current monitoring task is not included in the relevant tasks, then the second start time of the current monitoring task is read from the first running feature corresponding to the current monitoring task.

[0064] S170. Based on the runtime of multiple time intervals in the first runtime feature, determine the second runtime of the current monitoring task;

[0065] S180. Add the second start time and the second run duration to obtain the end time of the current monitoring task.

[0066] Specifically, S150 includes: obtaining the set start time of the current monitoring task from the first operating feature, and obtaining the actual end time of the upstream task from the second operating feature corresponding to the upstream task; and taking the maximum value between the set start time and the actual end time as the second start operating time.

[0067] The set start time is the starting time specified for the current monitoring task, and the actual end time is the actual end time of the upstream task. The set start time is greater than or equal to the actual end time of the last running upstream task.

[0068] It is understandable that the current monitoring task can run immediately after the upstream task finishes running, or it can run some time after the upstream task finishes running, at the current monitoring task's set start time. Optionally, the maximum value between the set start time and the actual end time can be the set start time, or it can be the set start time or the actual end time of the last running upstream task.

[0069] Specifically, S170 includes: sorting the runtime of multiple time intervals according to a preset order, and determining the sorting position corresponding to the runtime of each time interval; selecting the runtime at the median position from the runtime of multiple time intervals as the second runtime.

[0070] Specifically, the past running time of the current monitoring task is divided into multiple time intervals, each of which can be any value such as 5 minutes or 10 minutes. Then, the running time in multiple time intervals is arranged in reverse or forward order, and the sorting position corresponding to the running time of each time interval is determined. Finally, the running time at the median position is selected as the second running time.

[0071] In other embodiments, S170 specifically includes: averaging the runtime of multiple time intervals and using the calculated average as the second runtime.

[0072] In some embodiments of this disclosure, after performing S110, the method further includes:

[0073] If the current running status is "run completed", read the end time of the current monitoring task from the first running feature.

[0074] It is understandable that during the current monitoring task's execution, its corresponding first running feature is stored in the database (e.g., a MySQL database) in real time. Therefore, when the current monitoring task is detected to be in a completed state, the end time of the current monitoring task is directly read from the first running feature already stored in the database.

[0075] Therefore, when the current running status is either in a waiting state or a completed state, the end time of the current monitoring task is adaptively determined based on the stored data and / or the set data.

[0076] In another embodiment of this disclosure, in order to effectively use the runtime prediction model to determine the first runtime of the current monitoring task, before predicting the current monitoring task, it is necessary to construct the monitored task chain and train the runtime prediction model based on the historical characteristics of each task in the monitored task chain.

[0077] Figure 2A flowchart illustrating another end-time prediction method provided in an embodiment of this disclosure is shown.

[0078] like Figure 2 As shown, the end time prediction method may include the following steps.

[0079] S210. Using historical monitoring tasks as the root node and upstream tasks of historical monitoring tasks as leaf nodes, and based on the dependency relationship between historical monitoring tasks and upstream tasks, construct a tree-structured chain of monitored tasks.

[0080] Historical monitoring tasks refer to tasks monitored during historical periods prior to the monitoring time.

[0081] Specifically, when historical monitoring tasks are obtained, upstream tasks that have dependencies on the historical monitoring tasks are retrieved from the database (e.g., a Hadoop database) based on the task identifier (e.g., identity ID). A monitored task chain is then formed based on the historical monitoring tasks, upstream tasks, and dependencies. Furthermore, the monitored task chain is layered, specifically using the historical monitoring task as the root node and its upstream tasks as leaf nodes, and is recursively constructed layer by layer according to these dependencies to form a tree-structured monitored task chain.

[0082] For ease of understanding, Figure 3 A schematic diagram of the monitored task chain in a tree structure is shown. (For example...) Figure 3 As shown, with historical monitoring task A as the root node and upstream tasks A_11, A_12, A_21, A_22, A_23, and A_31 as leaf nodes, and based on the dependency relationship, the connection relationship between different nodes is generated to obtain a tree-structured monitoring task chain.

[0083] S220. Extract the historical features of each task and the runtime of each task from the historical features corresponding to the monitored task chain in the tree structure.

[0084] In this embodiment, the historical features include horizontal features, vertical features, and cross features. S220 specifically includes: according to a first time window, obtaining multiple first-type features from the historical features corresponding to the monitored task chain in the tree structure as horizontal features; according to a second time window, obtaining multiple second-type features from the historical features corresponding to the monitored task chain in the tree structure as vertical features; and using a multilayer perceptron to perform feature cross processing on the horizontal and vertical features to obtain cross features.

[0085] The historical features corresponding to the monitored task chain in the tree structure are pre-stored in a database (MySQL database). Information is collected from the database once every first time window (e.g., 20 minutes) to obtain horizontal features. Information is collected from the database once every second time window (e.g., 2 days, 5 days, 7 days) to obtain vertical features. The horizontal and vertical features are then used as input data for a multilayer perceptron (MLP). The feature cross-processing capability of the MLP is used to perform feature cross-processing on the horizontal and vertical features to obtain cross features.

[0086] Optionally, the horizontal features may include information such as the amount of memory allocated to the task, the number of CPUs allocated to the task, the memory used by the queue, the number of CPUs used by the queue, the maximum memory of the queue, the maximum number of CPUs in the queue, the number of containers occupied by the running task, the number of waiting containers, and the number of remaining containers.

[0087] Optionally, vertical features may include information such as queue environment and allocated resources.

[0088] S230. Based on the historical characteristics and runtime of each task, the preset network is trained to obtain a runtime prediction model.

[0089] In this embodiment, optionally, S230 includes: splicing the horizontal features, vertical features, and cross features according to the number of rows corresponding to the horizontal features, vertical features, and cross features respectively, to generate spliced ​​features for each task; and training a preset network based on the spliced ​​features of each task and the runtime of each task to obtain a runtime prediction model.

[0090] Specifically, if the number of rows for horizontal features, vertical features, and cross features is the same, but the number of columns is different, then multiple columns corresponding to the horizontal features, vertical features, and cross features can be concatenated according to the number of rows of these three types of features to generate concatenated features for each task, thereby obtaining features for model training.

[0091] Optionally, the splicing features of each task can be determined in the following way:

[0092] Fea final =concat(Fea) row Fea col Fea cross )

[0093] Among them, Fea final It is the splicing feature of each task, Fea row It is a lateral feature, Fea row It is a vertical feature, Fea cross It is a cross feature.

[0094] For ease of understanding, see Figure 4 The diagram shown illustrates the logic of feature generation. First, after obtaining the horizontal and vertical features, the horizontal and vertical features are input into a multilayer perceptron to obtain cross features. Then, the horizontal, cross, and vertical features are concatenated to obtain the concatenated features for each task.

[0095] Furthermore, the pre-defined network is iteratively trained using the concatenated features of each task and the runtime of each task to obtain a runtime prediction model. Optionally, the pre-defined network may include, but is not limited to, the LightGBM model.

[0096] To improve the robustness of the runtime prediction model, 80% of the samples used for training can be used as training samples, and the remaining 20% ​​as validation samples. The model is then evaluated on both the validation and test sets according to evaluation metrics. The runtime prediction model is considered complete when the evaluation metrics meet preset conditions. Optionally, the evaluation metrics may include, but are not limited to, the coefficient of determination (R²). 2 The error types are: Mean Absolute Error (MAE), Mean Square Error (RMAE), Root Mean Square Error (MSE), and Mean Absolute Percentage Error (MAPE).

[0097] Therefore, before predicting the current monitoring task, a tree-structured chain of monitored tasks is constructed based on historical monitoring tasks, upstream tasks of historical monitoring tasks, and dependencies. Based on the historical features corresponding to the tree-structured chain of monitored tasks, a runtime prediction model is trained, and then the trained runtime prediction model is used to predict the first runtime of the current monitoring task.

[0098] S240. Obtain the current running status of the current monitoring task at any monitoring time, wherein the current monitoring task is any one of the pre-built monitored task chains, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task.

[0099] S250. If the current running status is "running", read the first start time of the current monitoring task from the first running characteristic of the current monitoring task.

[0100] S260. Using a pre-trained runtime prediction model, process the first runtime feature and the second runtime feature of related tasks to obtain the first runtime of the current monitoring task.

[0101] S270. Based on the first runtime and the first start time, predict the end time of the current monitoring task.

[0102] S240 to S270 are similar to S110 to S140, and will not be described in detail here.

[0103] In another embodiment of this disclosure, the above scheme is explained in its entirety. Figure 5 This illustration shows a logical diagram of a task scheduling system monitoring method provided in an embodiment of the present disclosure, such as... Figure 5 As shown, the monitoring method of this task scheduling system includes: the construction of the monitored task chain and model training stage, the end time prediction stage, and the early warning stage.

[0104] The monitored task chain construction and model training phases include:

[0105] S1. Construct a tree-structured chain of monitored tasks.

[0106] Specifically, a tree-structured chain of monitored tasks is constructed, with historical monitoring tasks as the root node and upstream tasks of historical monitoring tasks as leaf nodes, based on the dependency relationship between historical monitoring tasks and the upstream tasks.

[0107] S2, Training runtime prediction model.

[0108] Specifically, the historical features and runtime of each task are extracted from the historical features corresponding to the monitored task chain in the tree structure; based on the historical features and runtime of each task, the preset network is trained to obtain the runtime prediction model.

[0109] The end-time prediction phase includes:

[0110] S3. Determine the current operating status of the current monitoring task at any given monitoring time.

[0111] Specifically, the current identification information carried by the pre-configured target interface is read, the current identification information is matched with the identification information corresponding to multiple pre-determined running states, and the current running state is determined from the multiple pre-determined running states based on the matching results.

[0112] S4. The current running status is in operation. Read the first start time of the current monitoring task from the first running characteristic of the current monitoring task.

[0113] S5. Using a pre-trained runtime prediction model, process the first runtime feature and the second runtime feature of related tasks to obtain the first runtime of the current monitoring task, and predict the end time of the current monitoring task based on the first runtime and the first start time.

[0114] Specifically, the end time of the current monitoring task is obtained by adding the first runtime and the first start time.

[0115] S6. The current running status is waiting to run. Determine the second start time of the current monitoring task.

[0116] Specifically, if the relevant tasks include the upstream task of the current monitoring task, the set start time of the current monitoring task is obtained from the first running feature, and the actual end time of the upstream task is obtained from the second running feature corresponding to the upstream task. The maximum value between the set start time and the actual end time is taken as the second start running time. If the relevant tasks do not include the upstream task of the current monitoring task, the second start time of the current monitoring task is read from the first running feature corresponding to the current monitoring task.

[0117] S7. Based on the runtime of multiple time intervals in the first runtime feature, determine the second runtime of the current monitoring task, and based on the second runtime and the second start time, predict the end time of the current monitoring task.

[0118] Specifically, the second start time and the second runtime are added together to obtain the end time of the current monitoring task.

[0119] S8. The current running status is the running completed status. Read the end time of the current monitoring task from the first running feature.

[0120] The early warning phase includes:

[0121] S9. Determine whether the end time of the current monitoring task is within the preset alarm time range.

[0122] Specifically, if the end time of the current monitoring task is within the preset alarm time range, then execute S10; otherwise, end.

[0123] S10. Generate alarm information and send the alarm information to the user.

[0124] This disclosure also provides an end-time prediction apparatus for implementing the above-described end-time prediction method, which is described below in conjunction with... Figure 6The following explanation is provided. In this embodiment, the end time prediction device can be an electronic device or a server. The electronic device can include devices with communication functions such as mobile phones, tablets, desktop computers, and laptops. The server can be a cloud server or server cluster, or other devices with storage and computing functions.

[0125] Figure 6 A schematic diagram of the structure of an end time prediction device provided in an embodiment of this disclosure is shown.

[0126] like Figure 6 As shown, the end time prediction device 600 may include:

[0127] The acquisition module 610 is used to acquire the current running status of the current monitoring task at any monitoring time, wherein the current monitoring task is any one of the tasks in the pre-built monitored task chain, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task.

[0128] The reading module 620 is used to read the first start time of the current monitoring task from the first running characteristics of the current monitoring task if the current running state is a running state;

[0129] The determination module 630 is used to process the first running feature and the second running feature of the related task using a pre-trained runtime prediction model to obtain the first runtime of the current monitoring task;

[0130] The prediction module 640 is used to predict the end time of the current monitoring task based on the first runtime and the first start time.

[0131] This disclosure provides an end-time prediction device that obtains the current running state of a monitoring task at any given monitoring time. The current monitoring task is any task in a pre-constructed monitored task chain, and the monitored task chain also includes related tasks that are dependent on the current monitoring task. If the current running state is in progress, the first start time of the current monitoring task is read from its first running characteristics. A pre-trained runtime prediction model is used to process the first running characteristics and the second running characteristics of the related tasks to obtain the first runtime of the current monitoring task. Based on the first runtime and the first start time, the end time of the current monitoring task is predicted. Therefore, when the task's running environment changes or its upstream tasks experience anomalies, the first runtime of the current monitoring task can be determined using the running characteristics of multiple dependent tasks in the entire monitored task chain combined with the runtime prediction model. Then, the end time of the current monitoring task can be determined based on the first start time and the first runtime, thereby improving the accuracy of end-time prediction. Furthermore, when the monitoring time changes, the first start time and the first runtime are re-determined to obtain a new end time, achieving the effect of dynamically determining the end time of the current monitoring task and ensuring the reliability and flexibility of end-time prediction.

[0132] In some optional embodiments, the acquisition module 610 includes:

[0133] The identifier reading unit is used to read the current identifier information carried by the pre-configured target interface;

[0134] The identifier matching unit is used to match the current identifier information with the identifier information corresponding to a plurality of predetermined operating states, and to determine the current operating state from the plurality of predetermined operating states based on the matching result.

[0135] In some optional embodiments, the prediction module 640 is specifically used for:

[0136] The end time of the current monitoring task is obtained by adding the first runtime and the first start time.

[0137] In some alternative embodiments, the device further includes:

[0138] The first determining module is configured to determine the second start time of the current monitoring task based on the first running characteristic corresponding to the current monitoring task and the second running characteristic corresponding to the upstream task if the current running state is a waiting running state and the related tasks include the upstream task of the current monitoring task.

[0139] The second determining module is used to read the second start time of the current monitoring task from the first running feature corresponding to the current monitoring task if the current running state is a waiting running state and the related tasks do not include the upstream task of the current monitoring task;

[0140] The third determining module is used to determine the second runtime of the current monitoring task based on the runtime of multiple time intervals in the first running feature;

[0141] The fourth determining module is used to add the second start time and the second runtime to obtain the end time of the current monitoring task.

[0142] In some optional embodiments, the first determining module is specifically used for:

[0143] The set start time of the current monitoring task is obtained from the first operating feature, and the actual end time of the upstream task is obtained from the second operating feature corresponding to the upstream task.

[0144] The maximum value between the set start time and the actual end time is taken as the second start running time.

[0145] In some optional embodiments, the third determining module is specifically used to: sort the runtime of the plurality of time intervals according to a preset order, and determine the sorting position corresponding to the runtime of each time interval;

[0146] The runtime at the median position of the runtimes from the plurality of time intervals is selected as the second runtime.

[0147] In some alternative embodiments, the device further includes:

[0148] The end time reading module reads the end time of the current monitoring task from the first running feature if the current running status is a running completion status.

[0149] In some alternative embodiments, the device further includes:

[0150] The construction module is used to construct a tree-structured chain of monitored tasks, with the historical monitoring task as the root node, the upstream task of the historical monitoring task as the leaf node, and based on the dependency relationship between the historical monitoring task and the upstream task.

[0151] The extraction module is used to extract the historical features of each task and the runtime of each task from the historical features corresponding to the monitored task chain in the tree structure.

[0152] The training module is used to train a preset network based on the historical features of each task and the runtime of each task to obtain the runtime prediction model.

[0153] In some optional embodiments, the historical features include horizontal features, vertical features, and intersection features, and the extraction module includes:

[0154] The first acquisition unit is used to acquire multiple first-type features from the historical features corresponding to the monitored task chain in the tree structure according to the first time window, as the horizontal features;

[0155] The second acquisition unit is used to acquire multiple second-type features from the historical features corresponding to the monitored task chain in the tree structure according to the second time window, as the vertical features;

[0156] The third acquisition unit is used to perform feature cross-processing on the horizontal features and the vertical features using a multilayer perceptron to obtain the cross-features.

[0157] In some alternative embodiments, the training module includes:

[0158] The splicing unit is used to splice the horizontal features, vertical features, and intersection features according to the row number corresponding to the horizontal features, vertical features, and intersection features respectively, to generate spliced ​​features for each task;

[0159] The training unit is used to train the preset network based on the splicing features of each task and the runtime of each task to obtain the runtime prediction model.

[0160] In some alternative embodiments, the device further includes:

[0161] The judgment module is used to determine whether the end time of the current monitoring task is within the preset alarm time range;

[0162] The alarm module is used to generate an alarm message and send the alarm message to the user if the end time of the current monitoring task is not within the preset alarm time range.

[0163] It should be noted that, Figure 6 The end time prediction device 600 shown can perform... Figures 1 to 5 The various steps in the method embodiment shown are implemented. Figures 1 to 5 The processes and effects in the method embodiments shown are not described in detail here.

[0164] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform a method according to an embodiment of this disclosure.

[0165] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0166] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0167] refer to Figure 7 The present invention describes a structural block diagram of an electronic device 700 that can serve as a server or client of the present disclosure. This is an example of a hardware device that can be applied to various aspects of the present disclosure, and the electronic device 700 can be the aforementioned electronic device. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0168] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the electronic device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0169] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 708 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0170] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the end-time prediction method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the end-time prediction method by any other suitable means (e.g., by means of firmware).

[0171] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0172] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0173] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0174] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0175] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0176] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0177] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0178] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for predicting end time, characterized in that, include: Obtain the current running status of the current monitoring task at any monitoring time, wherein the current monitoring task is any one of the pre-built monitored task chains, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task; If the current running state is in a running state, the first start time of the current monitoring task is read from the first running characteristic of the current monitoring task; The first runtime feature and the second runtime feature of the related task are processed using a pre-trained runtime prediction model to obtain the first runtime of the current monitoring task; Based on the first runtime and the first start time, predict the end time of the current monitoring task; The runtime prediction model is trained on a preset network based on the historical features and runtime of each task. The historical features and runtime are extracted from the historical features corresponding to the tree-structured monitored task chain. The tree-structured monitored task chain is constructed based on the historical monitoring tasks, the upstream tasks of the historical monitoring tasks, and the dependencies.

2. The method according to claim 1, characterized in that, The step of obtaining the current running status of the current monitoring task at any monitoring time includes: Read the current identification information carried by the pre-configured target interface; The current identification information is matched with the identification information corresponding to a plurality of predetermined operating states, and the current operating state is determined from the plurality of predetermined operating states based on the matching results.

3. The method according to claim 1, characterized in that, The step of predicting the end time of the current monitoring task based on the first runtime and the first start time includes: The end time of the current monitoring task is obtained by adding the first runtime and the first start time.

4. The method according to claim 1, characterized in that, The method further includes: If the current running state is a waiting running state, and the related tasks include the upstream task of the current monitoring task, then the second start running time of the current monitoring task is determined based on the first running characteristic corresponding to the current monitoring task and the second running characteristic corresponding to the upstream task. If the current running state is a waiting running state, and the related tasks do not include the upstream task of the current monitoring task, then the second start time of the current monitoring task is read from the first running feature corresponding to the current monitoring task; Based on the runtime of multiple time intervals in the first runtime feature, the second runtime of the current monitoring task is determined; The end time of the current monitoring task is obtained by adding the second start time and the second runtime.

5. The method according to claim 4, characterized in that, The step of determining the second start time of the current monitoring task based on the first operational characteristic corresponding to the current monitoring task and the second operational characteristic corresponding to the upstream task includes: The set start time of the current monitoring task is obtained from the first operating feature, and the actual end time of the upstream task is obtained from the second operating feature corresponding to the upstream task. The maximum value between the set start time and the actual end time is taken as the second start running time.

6. The method according to claim 4, characterized in that, The step of determining the second runtime of the current monitoring task based on the runtime of multiple time intervals in the first runtime feature includes: The runtime of the multiple time intervals is sorted according to a preset order to determine the sorting position corresponding to the runtime of each time interval; The runtime at the median position of the runtimes from the plurality of time intervals is selected as the second runtime.

7. The method according to claim 1, characterized in that, Also includes: If the current running status is a running completed status, the end time of the current monitoring task is read from the first running feature.

8. The method according to claim 1, characterized in that, Before obtaining the current running status of the current monitoring task at any monitoring time, the method further includes: Using historical monitoring tasks as the root node and upstream tasks of the historical monitoring tasks as leaf nodes, and based on the dependency relationship between the historical monitoring tasks and the upstream tasks, a tree-structured chain of monitored tasks is constructed. Extract the historical features of each task and the runtime of each task from the historical features corresponding to the monitored task chain in the tree structure. Based on the historical characteristics and runtime of each task, a preset network is trained to obtain the runtime prediction model.

9. The method according to claim 8, characterized in that, The historical features include horizontal features, vertical features, and cross features. Extracting the historical features of each task from the historical features corresponding to the monitored task chain in the tree structure includes: According to the first time window, multiple first-class features are obtained from the historical features corresponding to the monitored task chain in the tree structure, and used as the horizontal features; According to the second time window, multiple second-type features are obtained from the historical features corresponding to the monitored task chain in the tree structure, and used as the vertical features; The cross features are obtained by performing feature cross-processing on the horizontal and vertical features using a multilayer perceptron.

10. The method according to claim 9, characterized in that, The step of training a preset network based on the historical characteristics and runtime of each task to obtain the runtime prediction model includes: According to the row number corresponding to the horizontal feature, the vertical feature, and the intersection feature, the horizontal feature, the vertical feature, and the intersection feature are concatenated to generate the concatenated features for each task; Based on the splicing features of each task and the runtime of each task, the preset network is trained to obtain the runtime prediction model.

11. The method according to any one of claims 1 to 10, characterized in that, Also includes: Determine whether the end time of the current monitoring task is within the preset alarm time range; If the end time of the current monitoring task is not within the preset alarm time range, an alarm message is generated and sent to the user.

12. An end time prediction device, characterized in that, include: The acquisition module is used to acquire the current running status of the current monitoring task at any monitoring time. The current monitoring task is any one of the tasks in the pre-built monitored task chain, and the monitored task chain also includes related tasks that have a dependency relationship with the current monitoring task. The reading module is used to read the first start time of the current monitoring task from the first running characteristics of the current monitoring task if the current running state is a running state; The determination module is used to process the first runtime feature and the second runtime feature of the related task using a pre-trained runtime prediction model to obtain the first runtime of the current monitoring task; The prediction module is used to predict the end time of the current monitoring task based on the first runtime and the first start time. The runtime prediction model is trained on a preset network based on the historical features and runtime of each task. The historical features and runtime are extracted from the historical features corresponding to the tree-structured monitored task chain. The tree-structured monitored task chain is constructed based on the historical monitoring tasks, the upstream tasks of the historical monitoring tasks, and the dependencies.

13. An electronic device, characterized in that, include: processor; Memory, used to store executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the method of any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The storage medium stores a computer program that, when executed by a processor, causes the processor to implement the method described in any one of claims 1-11.

Images