A batch task anomaly detection information generation method, device, equipment and medium

Abstract

The application discloses a batch task exception detection information generation method and device, equipment and a medium. The method comprises the following steps: obtaining a historical acquisition sequence of a target batch task corresponding to a historical time period, the historical acquisition sequence comprising at least one batch acquisition data, the batch acquisition data comprising a task start time, a task end time and a task execution duration; inputting the historical acquisition sequence into a task duration prediction model to predict the predicted execution data of a target time corresponding to the target batch task; obtaining real-time execution data of the target time of the target batch task; calculating a three-dimensional error weight according to the real-time execution data and the predicted execution data corresponding to the target batch task; and generating batch task exception detection information corresponding to the target batch task according to the task type corresponding to the target batch task and the three-dimensional error weight. The application can improve the accuracy of batch task exception detection information generation.

Description

Technical Field

[0001] This invention relates to the fields of financial technology and data processing technology, and in particular to a method, apparatus, equipment and medium for generating batch task anomaly detection information. Background Technology

[0002] With the rapid development of technology, the types and number of batch tasks are gradually increasing, with each batch task comprising multiple subtasks. To ensure the reliability of batch task operation, it is necessary to perform anomaly detection on the execution status of batch tasks, so as to promptly perform operation and maintenance on batch tasks based on the anomaly detection results.

[0003] Currently, abnormalities in the execution status of batch tasks are detected by manually setting a fixed runtime threshold for batch tasks.

[0004] However, batch tasks are diverse and numerous, and the accuracy of detecting abnormal execution status of batch tasks based on manually set fixed runtime thresholds is low. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for generating batch task anomaly detection information, so as to improve the accuracy of batch task anomaly detection information generation.

[0006] In a first aspect, embodiments of the present invention provide a method for generating batch task anomaly detection information, the method comprising:

[0007] Obtain the historical collection sequence of the target batch task corresponding to the historical time period. The historical collection sequence includes at least one batch collection data, which includes: task start time, task end time and task execution duration.

[0008] The historical data collection sequence is input into the task duration prediction model to predict the execution data of the target time corresponding to the target batch task. The predicted execution data includes: prediction start time, prediction end time and prediction execution duration.

[0009] Obtain real-time execution data for the target batch tasks at the target time. The real-time execution data includes: real-time start time, real-time end time, and real-time task duration.

[0010] The three-dimensional error weights are calculated based on the real-time execution data and predicted execution data corresponding to the target batch tasks.

[0011] Based on the task type and three-dimensional error weights corresponding to the target batch tasks, generate batch task anomaly detection information corresponding to the target batch tasks.

[0012] Secondly, embodiments of the present invention also provide a batch task anomaly detection information generation device, the device comprising:

[0013] The acquisition sequence module is used to acquire the historical acquisition sequence of the target batch task corresponding to the historical time period. The historical acquisition sequence includes at least one batch acquisition data, which includes: task start time, task end time and task execution duration.

[0014] The execution data prediction module is used to input historical collection sequences into the task duration prediction model to predict the predicted execution data for the target time of the target batch tasks. The predicted execution data includes: prediction start time, prediction end time, and prediction execution duration.

[0015] The execution data acquisition module is used to acquire real-time execution data of the target batch tasks for the target time. The real-time execution data includes: real-time start time, real-time end time, and real-time task duration.

[0016] The three-dimensional error calculation module is used to calculate the three-dimensional error weights based on the real-time execution data and predicted execution data corresponding to the target batch tasks.

[0017] The anomaly information generation module is used to generate batch task anomaly detection information corresponding to the target batch tasks based on the task type and three-dimensional error weights.

[0018] Thirdly, embodiments of the present invention also provide a batch task anomaly detection information generation device, the batch task anomaly detection information generation device comprising:

[0019] At least one processor; and

[0020] A memory that is communicatively connected to at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to execute the batch task anomaly detection information generation method of any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the batch task anomaly detection information generation method of any embodiment of the present invention.

[0023] The technical solution of this invention involves acquiring historical acquisition sequences of target batch tasks corresponding to historical time periods. These historical acquisition sequences include at least one batch acquisition data set, comprising: task start time, task end time, and task execution duration. The historical acquisition sequences are input into a task duration prediction model to predict the target execution data for the target batch tasks, including: prediction start time, prediction end time, and prediction execution duration. Real-time execution data for the target batch tasks is acquired, including: real-time start time, real-time end time, and real-time task duration. Based on the real-time execution data and predicted execution data corresponding to the target batch tasks, three-dimensional error weights are calculated. Based on the task type and three-dimensional error weights corresponding to the target batch tasks, batch task anomaly detection information is generated. By using multi-dimensional data to detect anomalies in the execution status of batch tasks, the accuracy of generating batch task anomaly detection information is improved.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a method for generating batch task anomaly detection information according to an embodiment of the present invention;

[0027] Figure 2 This is a flowchart of a method for generating batch task anomaly detection information according to an embodiment of the present invention;

[0028] Figure 3 This is a structural diagram of a batch task anomaly detection information generation device provided according to an embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of the structure of a batch task anomaly detection information generation device provided in an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] The acquisition, storage, and application of historical data collection sequences involved in the technical solutions of this invention comply with relevant laws and regulations and do not violate public order and good morals.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a method for generating batch task anomaly detection information according to Embodiment 1 of the present invention. This embodiment of the invention is applicable to the generation of batch task anomaly detection information. The method can be executed by a batch task anomaly detection information generation device, which can be implemented in hardware and / or software.

[0035] See Figure 1 The batch task anomaly detection information generation method shown includes:

[0036] S101. Obtain the historical collection sequence of the target batch task corresponding to the historical time period. The historical collection sequence includes at least one batch collection data, which includes: task start time, task end time and task execution duration.

[0037] The historical time period refers to the continuous time interval during which the target batch task has completed its execution before the current detection time. This period serves as the historical data collection interval for model learning of operational patterns. This time period can be pre-defined based on business needs, data accumulation, and task execution cycle, defining effective time boundaries for time-series data. The target batch task can be a single distributed batch framework (DBF) for anomaly detection and time-series prediction, representing the smallest business unit for monitoring and analysis. Anomaly detection methods can be applied to different batch task tasks, with each task independently conducting data collection and analysis. The historical collection sequence can be an ordered time-series data set formed by arranging the data from each execution of the target batch task within the historical time period in chronological order. The sequence has strong temporal correlation, and the order of elements cannot be disrupted, representing the long-term temporal change patterns of the batch task. Batch collection data can be the smallest data unit constituting the historical collection sequence, corresponding to a set of structured data collected during a single complete execution of the target batch task. Each data point uniquely corresponds to one execution action of the batch task. The task start time refers to the point in time when the single execution process of the target batch task officially begins. The task completion time can refer to the point in time when the entire execution process of the target batch tasks is completed and the job status changes to "completed". The task execution duration can be the time difference calculated from the task completion time and the task start time of a single task, representing the total time consumed by the target batch tasks in a single run.

[0038] Specifically, based on business configuration, historical time periods are pre-determined, clearly defining the start and end times for data collection and defining the time boundaries for data collection. After each complete run of the target batch task, the scanning program automatically retrieves three basic data points from the DBF distributed batch management platform: task start time, task end time, and task execution duration, forming a batch collection data entry. All batch collection data collected within the historical time period are arranged and stored sequentially according to the actual execution time of the tasks, forming a historical collection sequence; the temporal relationship of the data is preserved, the arrangement order is not disrupted, and the integrity of the temporal characteristics is guaranteed.

[0039] S102. Input the historical collection sequence into the task duration prediction model to predict the target execution data corresponding to the target batch task. The predicted execution data includes: prediction start time, prediction end time and prediction execution duration.

[0040] The task duration prediction model can be a time-series prediction model trained on sample data and with fixed parameters. It can be trained based on a Long Short-Term Memory (LSTM) network model, which has learned the historical execution patterns of the target batch tasks and can infer subsequent task performance indicators based on the input time-series data. The target time can be the execution time node corresponding to a single execution of the target batch task to be detected, i.e., the execution cycle of the batch task to be predicted. The predicted execution data can be a set of estimated performance indicators of the target batch task at the target time, inferred and output by the task duration prediction model based on historical execution patterns. The predicted start time can be the time point at which the target batch task is expected to start execution at the target time, as inferred by the model. The predicted end time can be the time point at which the target batch task is expected to complete the entire execution process at the target time, as inferred by the model. The predicted execution duration can be the total duration at which a single execution of the target batch task is expected to consume at the target time, as inferred by the model.

[0041] Specifically, the historically collected sequences with ordered temporal structures are encapsulated according to the model's preset input format and dimensional requirements to generate input sample data that the model can recognize. This encapsulated input sample data is then fed into the task duration prediction model. The model invokes the temporal association rules and runtime feature mapping relationships learned during the training phase to perform forward inference operations and deduce the task's execution state at the target time. After completing the calculation, the model synchronously outputs a set of three-dimensional structured data, sequentially containing the prediction start time, prediction end time, and prediction execution duration. This set of data represents the predicted execution data for the target batch tasks at the target time.

[0042] S103. Obtain the real-time execution data of the target batch tasks for the target time. The real-time execution data includes: real-time start time, real-time end time, and real-time task duration.

[0043] Real-time execution data refers to the set of actual operational metrics collected after a single run of the target batch tasks is completed within the target time. This structured data reflects the actual operational status of the tasks and is used for comparison and analysis with the predicted execution data output by the model. The real-time start time is the actual point in time when the target batch tasks officially begin execution within the target time. The real-time end time is the actual point in time when the target batch tasks complete all job processes and change their status to "run completed" within the target time. The real-time task duration is the time difference calculated from the real-time end time and real-time start time, representing the total actual time consumed by a single run of the target batch tasks within the target time.

[0044] Specifically, the scanning program monitors the task status of the DBF distributed batch management platform in real time. When it detects that a target batch task has switched from a running state to a completed state within the target time, it automatically triggers data collection. The scanning program retrieves the real-time start time and real-time end time from the task execution log and job status data table of the DBF distributed batch management platform. Based on the retrieved real-time start time and real-time end time, the real-time task duration is calculated through time difference calculation. The three indicators—real-time start time, real-time end time, and real-time task duration—are combined according to a preset data structure to form a complete real-time execution data. The assembled real-time execution data is temporarily stored and synchronously pushed to the downstream error calculation stage for pairing with the predicted execution data for the same target time; simultaneously, according to a unified standard, this data is supplemented into the historical time series data source.

[0045] S104. Calculate the three-dimensional error weights based on the real-time execution data and predicted execution data corresponding to the target batch tasks.

[0046] Among them, the three-dimensional error weight can be the deviation between the measured data and the predicted data calculated for the three independent data dimensions of start time, end time and execution duration. The single comprehensive quantitative value obtained by fusing the errors of the three dimensions uniformly represents the degree of deviation between the overall running status of the target batch task and the normal expected state.

[0047] Specifically, for the three dimensions of start time, end time, and execution duration, the deviation between the measured value and the predicted value is calculated one by one to obtain the three-dimensional raw error. That is, the difference between the real-time start time and the predicted start time is obtained as the first dimension error; the difference between the real-time end time and the predicted end time is obtained as the second dimension error; and the difference between the real-time task duration and the predicted execution duration is obtained as the third dimension error. These three sets of errors together constitute the three-dimensional error vector. Pre-configured weight parameters are retrieved, and corresponding data weights are assigned to the three dimensions of start time, end time, and execution duration. For example, for daily batch tasks, execution duration is usually assigned a higher weight, while non-core monitoring dimensions are assigned relatively lower weights. The raw error of each dimension is multiplied by the corresponding data weight in turn to obtain the weighted error of each dimension. All weighted errors are summed to obtain a single comprehensive value, i.e., the three-dimensional error weight.

[0048] S105. Generate batch task anomaly detection information corresponding to the target batch task based on the task type and three-dimensional error weights.

[0049] The task type can be a category categorized based on the business affiliation, operational priority, functional role, and timeliness requirements of the target batch tasks. This category is used to differentiate the importance level, risk impact scope, and control standards of different batch operations. Different task types correspond to different anomaly control rules. Batch task anomaly detection information can be standardized structured information generated based on comparison and judgment results. This information fully records the operational status, anomaly judgment conclusion, anomaly level, and deviation degree of the target batch tasks, and can be used for platform display, log retention, alarm push notifications, and operation and maintenance traceability.

[0050] Specifically, the system retrieves the task type corresponding to the target batch tasks, searches the system's preset configuration relationships, and matches the corresponding anomaly warning level for that task type. Higher task priority and wider impact result in a higher warning level; low-priority tasks are matched with a standard warning level. The system reads the general baseline threshold generated during model training and dynamically adjusts it based on the currently matched anomaly warning level: high warning levels tighten the threshold range to improve anomaly detection sensitivity; low warning levels moderately widen the threshold range to reduce the probability of invalid alarms. This adjustment yields the target threshold suitable for the current task. The system compares the three-dimensional error weights to be judged with the target threshold: if the three-dimensional error weights are less than or equal to the target threshold, the target batch task is judged to be running normally; if the three-dimensional error weights are greater than the target threshold, the target batch task is judged to be running abnormally. Finally, the system assembles complete batch task anomaly detection information according to a preset data format, combining fields such as task basic identifier, task type, three-dimensional error weights, threshold comparison results, anomaly level, and deviation magnitude. Based on the matched anomaly warning level, the corresponding warning strategy is invoked to complete the information push: in the normal state, only logs are retained; in the abnormal state, differentiated alarms are executed according to the level, including different forms such as red prompts on the platform interface, operation and maintenance message pushes, SMS notifications, and audible and visual alarms.

[0051] The technical solution of this invention involves acquiring historical acquisition sequences of target batch tasks corresponding to historical time periods. These historical acquisition sequences include at least one batch acquisition data set, comprising: task start time, task end time, and task execution duration. The historical acquisition sequences are input into a task duration prediction model to predict the target execution data for the target batch tasks, including: prediction start time, prediction end time, and prediction execution duration. Real-time execution data for the target batch tasks is acquired, including: real-time start time, real-time end time, and real-time task duration. Based on the real-time execution data and predicted execution data corresponding to the target batch tasks, three-dimensional error weights are calculated. Based on the task type and three-dimensional error weights corresponding to the target batch tasks, batch task anomaly detection information is generated. By using multi-dimensional data to detect anomalies in the execution status of batch tasks, the accuracy of generating batch task anomaly detection information is improved.

[0052] Example 2

[0053] Figure 2 This is a flowchart illustrating a method for generating batch task anomaly detection information according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the batch task anomaly detection information generation operation.

[0054] Furthermore, the step of "calculating the three-dimensional error weights based on the real-time execution data and predicted execution data corresponding to the target batch tasks" is refined into "for each data item in the real-time execution data corresponding to the target batch tasks, the data error is obtained based on the difference between the data item and its corresponding data item in the predicted execution data; the data weight corresponding to the data item is determined based on the task time limit requirements of the target batch tasks; and the three-dimensional error weights are calculated by weighted summation of the data errors and data weights corresponding to each data item," in order to improve the operation of generating batch task anomaly detection information.

[0055] It should be noted that for parts not described in detail in the embodiments of the present invention, please refer to the descriptions in other embodiments.

[0056] See Figure 2 The batch task anomaly detection information generation method shown includes:

[0057] S201. Obtain the historical collection sequence of the target batch task corresponding to the historical time period. The historical collection sequence includes at least one batch collection data, which includes: task start time, task end time and task execution duration.

[0058] S202. Input the historical collection sequence into the task duration prediction model to predict the target execution data corresponding to the target batch task. The predicted execution data includes: prediction start time, prediction end time and prediction execution duration.

[0059] S203. Obtain the real-time execution data of the target batch tasks for the target time. The real-time execution data includes: real-time start time, real-time end time, and real-time task duration.

[0060] S204. For each data item in the real-time execution data corresponding to the target batch task, obtain the data error based on the difference between the data item and its corresponding data item in the predicted execution data.

[0061] The data items can be the smallest independent indicator units that constitute real-time execution data and predicted execution data. Specifically, they are divided into three categories: start time data items, end time data items, and task duration data items, with each category corresponding to the other. Data error can be the deviation obtained by interpolating a single type of data item, representing the degree of deviation between the actual running state and the expected running state on a single indicator dimension.

[0062] Specifically, data items are paired one-to-one according to the meaning of the indicators, establishing three sets of correspondences: real-time start time and predicted start time form the first pairing; real-time end time and predicted end time form the second pairing; and real-time task duration and predicted execution duration form the third pairing. For each pairing, a numerical difference operation is used to calculate the difference between the measured data item and the corresponding predicted data item; the absolute value of the result is taken as the basic deviation value. The absolute value result obtained from the operation for each pairing is defined as the data error corresponding to that dimension, resulting in the start time data error, end time data error, and task duration data error. The three sets of data errors are then integrated into a structured error set.

[0063] S205. Determine the data weights corresponding to the data items based on the time limit requirements of the target batch tasks.

[0064] The task time limit requirements can be a set of pre-defined business timeliness constraints for the target batch tasks, including the timeliness priority of each operational indicator, the degree of impact of timeout, and the level of control strictness. This serves as the business basis for distinguishing the importance of different data items. Different types of batch tasks have different time limit constraint strengths and core focus indicators. Data weight can be a quantified coefficient configured for an individual data item, used to characterize the importance of that data item in the overall anomaly detection system. The larger the weight value, the higher the impact of the corresponding data item on the batch task's operational status and anomaly detection results.

[0065] Specifically, based on the unique identifier of the target batch task, the system's built-in business rule library is retrieved to read the complete task time limit requirements information corresponding to the task, clarifying the timeliness control level and business impact weight level of the three types of data items. The three data items—start time, end time, and task execution duration—are each bound to the corresponding timeliness constraint rules in the task time limit requirements information. For example, for end-of-day settlement batch tasks, task execution duration is the core control indicator; for batch tasks relying on upstream file integration, task start time is the key control indicator. Based on the strictness of timeliness control and the magnitude of business impact corresponding to each data item, pre-bound quantitative coefficients are retrieved from the preset weight parameter library as the data weight corresponding to that data item. The stricter the control requirements and the larger the scope of abnormal impact of a data item, the higher the matched weight value. Each data item and its corresponding data weight are associated and encapsulated to form a weight parameter set.

[0066] S206. Calculate the three-dimensional error weight by weighted summation based on the data error and data weight corresponding to each data item.

[0067] Specifically, following the one-to-one correspondence between data items, the error of each data item is multiplied by its corresponding weight to obtain the weighted error value for each dimension. For example, the error of the task start time is multiplied by its corresponding weight to obtain the first-dimensional weighted error; the error of the task end time is multiplied by its corresponding weight to obtain the second-dimensional weighted error; and the error of the task execution duration is multiplied by its corresponding weight to obtain the third-dimensional weighted error. The weighted error values ​​calculated from these three dimensions are then summed to obtain the final value, which is the three-dimensional error weight.

[0068] S207. Generate batch task anomaly detection information corresponding to the target batch task based on the task type and three-dimensional error weights.

[0069] This invention, through its embodiments, obtains data errors by analyzing the differences between each data item in the real-time execution data corresponding to a target batch task and its corresponding data item in the predicted execution data. It then determines the data weights corresponding to each data item based on the task time limit requirements of the target batch task. Finally, it calculates a three-dimensional error weight by weighted summation of the data errors and weights corresponding to each data item. By extracting indicator deviations across dimensions, configuring differentiated weights based on business timeliness rules, and generating a comprehensive judgment index through weighted summation, this invention accurately quantifies the deviation of each operational indicator of the batch task from the normal state. Furthermore, it incorporates the business management priority of batch operations, achieving differentiated and refined anomaly identification and improving the accuracy of anomaly detection in distributed batch tasks.

[0070] Optionally, based on the task type and three-dimensional error weights corresponding to the target batch tasks, batch task anomaly detection information is generated, including: determining the anomaly warning level based on the task type corresponding to the target batch tasks; adjusting the obtained baseline threshold according to the anomaly warning level to obtain the target threshold; comparing the target threshold with the three-dimensional error weights to obtain the threshold comparison result; generating batch task anomaly detection information corresponding to the target batch tasks based on the threshold comparison result, and issuing an alert according to the alert method corresponding to the anomaly warning level.

[0071] Specifically, the system reads the type identifier of the target batch tasks, retrieves the preset configuration relationship, and matches the corresponding anomaly warning level for the task. Tasks with higher business priority and greater impact from faults receive higher warning levels. A general baseline threshold is retrieved, and adjustments are made according to the correction rules corresponding to the current anomaly warning level: high warning levels tighten the threshold to improve anomaly detection sensitivity; low warning levels moderately loosen the threshold to reduce invalid alarms. After adjustment, the target threshold for judgment is obtained. The three-dimensional error weight is compared with the target threshold: if the three-dimensional error weight is less than or equal to the target threshold, the task is considered to be running normally; if the three-dimensional error weight is greater than the target threshold, the task is considered to be running abnormally, forming the final threshold comparison result. Combining fields such as task number, task type, target time, three-dimensional error weight, threshold comparison result, and anomaly warning level, complete batch task anomaly detection information is assembled according to a preset format and synchronously archived to the system log. Based on the matched anomaly warning level, a preset warning strategy is invoked: in normal states, only logs are retained, and no alarms are triggered; in abnormal states, reminder information is pushed according to the method corresponding to the warning level, completing the alarm action.

[0072] Anomaly warning levels are determined based on the task types corresponding to the target batch tasks. Based on these warning levels, the obtained baseline thresholds are adjusted to obtain the target thresholds. The target thresholds are then compared with the three-dimensional error weights to obtain the threshold comparison results. Based on these results, batch task anomaly detection information corresponding to the target batch tasks is generated, and warnings are issued according to the warning methods corresponding to the anomaly warning levels. Warning levels are differentiated based on task type, and thresholds are dynamically adjusted according to the levels to achieve different judgment criteria for different tasks, adapting to the anomaly detection requirements of different batch tasks and effectively balancing false alarms and missed alarms.

[0073] Optionally, the task duration prediction model is obtained through the following steps: Obtaining sample task execution sequences corresponding to the target batch tasks, each sample task execution sequence including at least one execution sample data, with the execution sample data including the sample execution start time, sample execution end time, and sample execution duration; Based on the execution results of each execution sample data in the sample task execution sequence, filtering each execution sample data to obtain a training sample set, which includes at least one normal execution data; Filtering the sample execution sequence according to preset filtering conditions to obtain a validation sample set and a test sample set, the preset filtering conditions including: filtering time period and filtering ratio; Inputting the training sample set into the task duration prediction model for training to obtain the trained task duration prediction model; Determining the validation set loss value based on the validation sample set and the task duration prediction model; Based on the validation set loss value, determining whether to re-execute the process of inputting the training sample set into the task duration prediction model for training to obtain the trained task duration prediction model, until the training termination condition is met.

[0074] The execution results can be labels indicating the task execution status of a single execution sample, categorized into normal execution and abnormal execution. The training sample set can be a dataset selected from the sample task execution sequence, containing only normal execution data, used for parameter iteration and feature learning of the task duration prediction model. Normal execution data can be historical execution data of the target batch tasks running stably under normal business conditions, without timeouts, interruptions, or failures. Preset filtering conditions can be pre-defined data partitioning rules, including two constraints: a filtering time period and a filtering ratio, used to separate datasets for different purposes from the overall sample sequence. The filtering time period can be a time interval defined based on temporal characteristics. The filtering ratio can be a pre-defined numerical ratio used to control the data volume proportions of the training sample set, validation sample set, and test sample set. The validation sample set can be a dataset partitioned from the sample task execution sequence according to preset conditions, used to evaluate the model's generalization ability and determine its fit during training. The test sample set can be a dataset partitioned from the sample task execution sequence according to preset conditions, used to comprehensively evaluate the overall performance after model training. The validation set loss value can be the deviation value calculated by the loss function after inputting the validation sample set into the current model, reflecting the model's fit to unknown normal data. The training termination condition can be a pre-defined rule for stopping iterative training, such as using the validation set loss value no longer decreasing as the termination criterion.

[0075] Specifically, the entire historical execution record of the target batch tasks is collected. Each record is encapsulated as execution sample data containing the start time, end time, and duration of the execution. All execution sample data are arranged in chronological order according to the actual execution time of the tasks, forming a complete sample task execution sequence. The execution result label corresponding to each execution sample data is read, and normal execution data is filtered according to a selection ratio to form a training sample set. Based on the preset selection time period and selection ratio, the overall sample task execution sequence is segmented without disrupting the original time sequence. A validation sample set and a test sample set are then created according to a predetermined ratio. The training sample set is continuously input into the initial task duration prediction model, and iterative training is performed using supervised learning. The model's internal weights and parameters are continuously updated, allowing the model to learn the temporal characteristics and changing patterns of the normal operation of the batch tasks, resulting in the model after each round of training. The validation sample set is input into the model completed in the current round of training, and the corresponding validation set loss value is calculated using a loss function to quantify the model's fitting effect. Compare the loss values ​​of the validation set obtained from multiple rounds of training: if the loss value continues to decrease, continue iteratively training the model using the training sample set; if the loss value no longer decreases after multiple rounds, the training termination condition is met, training is terminated, and the task duration prediction model is finally determined. For example, a scanning program can be used to collect basic information about the start time, end time, and execution duration of at least one batch of tasks and store it in a batch execution sequence. The batch execution sequence is then divided into training, validation, and test sets according to time sequence. Based on the time dependency of the time-series data, it is split chronologically, including: a training set (70%-80%): using only "known normal" data to ensure the model only learns normal patterns; a validation set (10%-15%): used to adjust model hyperparameters (such as learning rate and number of hidden layer nodes); and a test set (10%-15%): containing labeled normal and abnormal data to evaluate the detection effect. It should be noted that the task duration prediction model may include two layers of LSTM units, with the number of nodes in each layer selected according to data complexity (e.g., 64, 128, adjusted by validation set error). The activation function is tanh, and overfitting is suppressed through a Dropout layer (dropout rate = 0.2-0.5). It can accept time-series samples of shape (W, 3) (W is the time window length, 3 is the variable dimension), and the fully connected output layer outputs predicted values ​​of shape (1, 3), with the sigmoid function chosen as the activation function. The model uses mean squared error (MSE) to measure the deviation between the predicted and actual values ​​(the smaller the deviation, the closer the model is to the normal pattern). Adam (learning rate = 0.001-0.0001) is also used to accelerate convergence through adaptive learning rate and avoid local optima.Training should be stopped when the validation set loss stops decreasing for 5-10 consecutive epochs to avoid overfitting. The batch size can be 32 or 64 (adjusted according to the memory size), and the number of epochs can be dynamically determined through the early stopping mechanism (usually 20-50 epochs).

[0076] The task duration prediction model obtains the following results through the following steps: First, it acquires the sample task execution sequence corresponding to the target batch tasks. This sequence includes at least one execution sample data point, which includes the start time, end time, and duration of the execution. Second, based on the execution results of each execution sample data point in the sequence, it filters the data to obtain a training sample set, which includes at least one normal execution data point. Third, it filters the execution sequence according to preset criteria, including a filtering time period and a filtering ratio, to obtain a validation sample set and a test sample set. The task duration prediction model is trained by combining input samples to obtain the trained task duration prediction model. Based on the validation sample set and the task duration prediction model, the validation set loss value is determined. Based on the validation set loss value, it is determined whether to re-execute the task duration prediction model by inputting the training sample set into the task duration prediction model to obtain the trained task duration prediction model, until the training termination condition is met. Only normal execution data is selected for the training set to ensure that the model only learns the standard operation rules of batch tasks and avoids abnormal data interfering with feature learning. Multiple datasets are divided according to time and proportion, and training, parameter tuning, and performance testing are carried out by division of labor, which conforms to the time series data processing specifications and ensures the model's generalization ability.

[0077] Optionally, after determining whether to retrain the task duration prediction model by inputting the training sample set into the model based on the validation set loss value, until the training termination condition is met, the method further includes: inputting the normal execution data into the trained task duration prediction model to obtain normal sample prediction data for each normal execution data; calculating the sample error based on the normal execution data and the normal sample prediction data; calculating the standard deviation and mean error based on the sample error corresponding to each normal execution data; and calculating the baseline threshold based on the standard deviation and mean error.

[0078] Specifically, the entire set of normal execution data is traversed, and each data point is input into the trained task duration prediction model. The model performs forward inference and outputs the corresponding normal sample prediction data for each data point. Each normal execution data point is compared with its corresponding normal sample prediction data. Through multi-dimensional bias fusion calculation, the sample error for each data point is obtained, forming a complete set of normal sample errors. Statistical calculations are performed on all sample errors to obtain the mean and standard deviation of all sample errors. Following a preset statistical formula, the mean error, standard deviation, and batch task coefficients are used to calculate the baseline threshold used for anomaly detection. The preset statistical formula can be: Baseline Threshold = Mean Error + Batch Task Coefficient × Standard Deviation of Error

[0079] By inputting the normal execution data into the trained task duration prediction model for each normal execution data point, normal sample prediction data is obtained. Based on the normal execution data and normal sample prediction data, the sample error is calculated. Based on the sample error corresponding to each normal execution data point, the standard deviation and mean error are calculated. Based on the standard deviation and mean error, a baseline threshold is calculated. The threshold is determined by statistical methods based on the mean and standard deviation of the error, replacing the traditional method of manually setting a fixed threshold. The threshold adapts to the fluctuations in task operation, and the threshold determination is more accurate.

[0080] Optionally, after determining whether to retrain the task duration prediction model by inputting the training sample set into the model based on the validation set loss value, until the training termination condition is met, the method further includes: inputting the test sample data into the trained task duration prediction model to obtain test sample prediction data; calculating the test sample error value based on the test sample prediction data and the test sample data; determining the anomaly detection result corresponding to the test sample data based on the test sample error value and the benchmark threshold; obtaining statistical information based on the anomaly detection result corresponding to each test sample data, including precision, recall, and statistical weights; and determining whether to retrain the task duration prediction model by inputting the training sample set into the model based on the statistical information, until the training termination condition is met.

[0081] Specifically, the process iterates through all test sample data within the test sample set, inputting each data point into the trained task duration prediction model. The model performs forward inference and outputs the corresponding test sample prediction data sequentially. For each set of test sample data and test sample prediction data, the test sample error value for each sample is calculated according to a unified deviation calculation rule. The test sample error value for each sample is compared with a benchmark threshold: if the error value is not greater than the benchmark threshold, the model is considered to be running normally; if the error value is greater than the benchmark threshold, the model is considered to be running abnormally, and the abnormal detection result for each sample is recorded. Combining the human-generated real labels for each test sample data with the abnormal detection results output by the model, precision, recall, and statistical weights are calculated uniformly to form overall model performance statistics. The statistical weights can be determined by the harmonic average of precision and recall. The statistical information is compared with a preset pass / fail standard: if all indicators meet the requirements, the model is officially finalized and ready for practical use; if the indicators do not meet the standards, the model training process is returned, the model is retrained using the training sample set, and the above verification steps are repeated until the statistical information reaches the preset standard.

[0082] By inputting the test sample data from the test sample set into the trained task duration prediction model, test sample prediction data is obtained. The test sample error value is calculated based on the predicted data and the actual test sample data. The anomaly detection result corresponding to the test sample data is determined based on the test sample error value and a benchmark threshold. Statistical information, including precision, recall, and statistical weights, is obtained based on the anomaly detection results for each test sample data. Based on this statistical information, it is determined whether to retrain the task duration prediction model by inputting the training sample set into the model, obtaining a trained task duration prediction model. This process continues until the training termination condition is met. An iterative optimization mechanism is established to automatically trigger retraining for models that do not meet performance standards, continuously optimizing prediction and anomaly detection effects, effectively reducing the probability of false positives and false negatives, and ensuring the reliability of batch task monitoring.

[0083] Optionally, before obtaining the sample task execution sequence corresponding to the target batch task, where the sample task execution sequence includes at least one execution sample data, and the execution sample data includes the sample execution start time, sample execution end time, and sample execution duration, the method further includes: obtaining at least one historical execution sequence corresponding to the target batch task, where the historical execution sequence includes at least one historical execution data; determining the filtering time window and time step based on the historical running cycle data corresponding to the target batch task; and dividing the historical execution sequence according to the filtering time window to obtain at least one sample task execution sequence, so that the task duration prediction model can predict the batch task prediction data corresponding to the target time of the time step based on the sample task execution sequence.

[0084] Specifically, all historical execution records of the target batch tasks are collected, and each execution record is encapsulated as a historical execution data point. All historical execution data are then sorted sequentially according to the actual execution time of the tasks, and combined to obtain a complete historical execution sequence. Historical execution cycle data of the target batch tasks are retrieved, and the optimal filtering time window is determined through cross-validation, taking into account constraints such as task execution fluctuation patterns, total data volume, and model computational performance. Simultaneously, a fixed time step is set according to business forecasting requirements, clarifying the time interval for model prediction. Using a sliding truncation method, the original long-sequence historical execution sequence is continuously segmented according to the determined filtering time window, resulting in multiple fixed-length sub-sequences, i.e., sample task execution sequences. Each sample task execution sequence serves as model input, predicting the task execution data at the target time indicated by the predicted time step.

[0085] By acquiring at least one historical execution sequence corresponding to the target batch task, the historical execution sequence includes at least one historical execution data; based on the historical running cycle data corresponding to the target batch task, the filtering time window and time step are determined; the historical execution sequence is divided according to the filtering time window to obtain at least one sample task execution sequence, so that the task duration prediction model can predict the batch task prediction data corresponding to the target time based on the sample task execution sequence. The ultra-long original time series data is divided into fixed-length samples to adapt to the input requirements of the Long Short-Term Memory Network. The window size is determined in combination with the actual running cycle of the task to fully capture the periodic fluctuation characteristics of the task and improve the prediction accuracy.

[0086] Example 3

[0087] Figure 3 This is a schematic diagram of a batch task anomaly detection information generation device provided in Embodiment 3 of the present invention. This embodiment of the present invention is applicable to the generation of batch task anomaly detection information. The device can execute a batch task anomaly detection information generation method and can be implemented in hardware and / or software.

[0088] See Figure 3 The batch task anomaly detection information generation device shown includes: a data acquisition sequence module 301, an execution data prediction module 302, an execution data acquisition module 303, a three-dimensional error calculation module 304, and an anomaly information generation module 305, wherein...

[0089] The acquisition sequence module 301 is used to acquire the historical acquisition sequence of the target batch task corresponding to the historical time period. The historical acquisition sequence includes at least one batch acquisition data, which includes: task start time, task end time and task execution duration.

[0090] The execution data prediction module 302 is used to input the historical collection sequence into the task duration prediction model to predict the predicted execution data of the target time corresponding to the target batch task. The predicted execution data includes: prediction start time, prediction end time and prediction execution duration.

[0091] The execution data acquisition module 303 is used to acquire real-time execution data of the target batch tasks for the target time. The real-time execution data includes: real-time start time, real-time end time, and real-time task duration.

[0092] The three-dimensional error calculation module 304 is used to calculate the three-dimensional error weights based on the real-time execution data and predicted execution data corresponding to the target batch tasks.

[0093] The anomaly information generation module 305 is used to generate batch task anomaly detection information corresponding to the target batch task based on the task type and three-dimensional error weights of the target batch task.

[0094] The technical solution of this invention involves acquiring historical acquisition sequences of target batch tasks corresponding to historical time periods. These historical acquisition sequences include at least one batch acquisition data set, comprising: task start time, task end time, and task execution duration. The historical acquisition sequences are input into a task duration prediction model to predict the target execution data for the target batch tasks, including: prediction start time, prediction end time, and prediction execution duration. Real-time execution data for the target batch tasks is acquired, including: real-time start time, real-time end time, and real-time task duration. Based on the real-time execution data and predicted execution data corresponding to the target batch tasks, three-dimensional error weights are calculated. Based on the task type and three-dimensional error weights corresponding to the target batch tasks, batch task anomaly detection information is generated. By using multi-dimensional data to detect anomalies in the execution status of batch tasks, the accuracy of generating batch task anomaly detection information is improved.

[0095] Optional, the three-dimensional error calculation module 304 is specifically used for:

[0096] For each data item in the real-time execution data corresponding to the target batch task, the data error is obtained based on the difference between the data item and its corresponding data item in the predicted execution data.

[0097] Based on the time limit requirements of the target batch tasks, determine the data weights corresponding to the data items;

[0098] The three-dimensional error weights are calculated by weighting and summing the data errors and data weights corresponding to each data item.

[0099] Optionally, the exception information generation module 305 includes:

[0100] Determine the anomaly warning level based on the task type corresponding to the target batch tasks;

[0101] Based on the abnormal warning level, the obtained baseline threshold is adjusted to obtain the target threshold;

[0102] The target threshold is compared with the three-dimensional error weights to obtain the threshold comparison result;

[0103] Based on the threshold comparison results, generate batch task anomaly detection information corresponding to the target batch tasks, and issue warnings according to the warning methods corresponding to the anomaly warning levels.

[0104] Optionally, the task duration prediction model is obtained through the following steps:

[0105] The sample acquisition module is used to acquire the sample task execution sequence corresponding to the target batch task. The sample task execution sequence includes at least one execution sample data, which includes the sample execution start time, sample execution end time, and sample execution duration.

[0106] The sample filtering module is used to filter each execution sample data according to the execution result of each execution sample data in the sample task execution sequence to obtain a training sample set, which includes at least one normal execution data.

[0107] The conditional filtering module is used to filter the sample execution sequence according to preset filtering conditions to obtain the verification sample set and the test sample set. The preset filtering conditions include: the filtering time period and the filtering ratio.

[0108] The model training module is used to input the training sample set into the task duration prediction model for training, and obtain the trained task duration prediction model.

[0109] The loss value calculation module is used to determine the validation set loss value based on the validation sample set and the task duration prediction model.

[0110] The loss value judgment module is used to determine whether to re-train the task duration prediction model by inputting the training sample set into the model based on the loss value of the validation set, so as to obtain the trained task duration prediction model until the training termination condition is met.

[0111] Optionally, the baseline threshold is obtained through the following steps:

[0112] Based on the loss value of the validation set, it is determined whether to re-train the task duration prediction model by inputting the training sample set into the training model to obtain the trained task duration prediction model. After the training ends, the normal execution data is input into the trained task duration prediction model to obtain the normal sample prediction data.

[0113] The sample error is calculated based on normal execution data and normal sample prediction data;

[0114] Based on the sample error corresponding to each normal execution data, the standard deviation and mean error are calculated.

[0115] The baseline threshold is calculated based on the standard deviation and mean error.

[0116] Optionally, the task duration prediction model is also obtained through the following steps:

[0117] Based on the loss value of the validation set, it is determined whether to re-train the task duration prediction model by inputting the training sample set into the training sample set to obtain the trained task duration prediction model. After the training ends, the test sample data is input into the trained task duration prediction model for each test sample data in the test sample set to obtain the test sample prediction data.

[0118] The test sample error value is calculated based on the predicted test sample data and the test sample data.

[0119] Based on the test sample error value and the benchmark threshold corresponding to the test sample data, the anomaly detection result corresponding to the test sample data is determined;

[0120] Based on the anomaly detection results corresponding to each test sample data, statistical information is obtained, including: precision, recall, and statistical weights.

[0121] Based on the statistical information, determine whether to retrain the task duration prediction model by inputting the training sample set into the model, and obtain the trained task duration prediction model until the training ends.

[0122] Optionally, the sample task execution sequence is obtained through the following steps:

[0123] Before obtaining the sample task execution sequence corresponding to the target batch task, the sample task execution sequence includes at least one execution sample data, and the execution sample data includes the sample execution start time, sample execution end time and sample execution duration, at least one historical execution sequence corresponding to the target batch task is obtained, and the historical execution sequence includes at least one historical execution data.

[0124] Based on the historical running cycle data corresponding to the target batch tasks, determine the filtering time window and time step.

[0125] The historical execution sequence is divided according to the filtering time window to obtain at least one sample task execution sequence, so that the task duration prediction model can predict batch task prediction data corresponding to the target time of the time step based on the sample task execution sequence.

[0126] The batch task anomaly detection information generation device provided in this embodiment of the invention can execute the batch task anomaly detection information generation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the batch task anomaly detection information generation method.

[0127] Example 4

[0128] Figure 4 A schematic diagram of the structure of a batch task anomaly detection information generation device 400 that can be used to implement embodiments of the present invention is shown.

[0129] like Figure 4 As shown, the batch task anomaly detection information generation device 400 includes at least one processor 401 and a memory, such as a read-only memory (ROM) 402 and a random access memory (RAM) 403, communicatively connected to the at least one processor 401. The memory stores computer programs executable by the at least one processor. The processor 401 can perform various appropriate actions and processes based on the computer program stored in the ROM 402 or loaded into the RAM 403 from the storage unit 408. The RAM 403 can also store various programs and data required for the operation of the batch task anomaly detection information generation device 400. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0130] Multiple components in the batch task anomaly detection information generation device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, optical disk, etc.; and a communication unit 409, such as a network card, modem, wireless transceiver, etc. The communication unit 409 allows the batch task anomaly detection information generation device 400 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0131] Processor 401 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 401 performs the various methods and processes described above, such as batch task anomaly detection information generation methods.

[0132] In some embodiments, the batch task anomaly detection information generation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program may be loaded and / or installed onto the batch task anomaly detection information generation device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by processor 401, one or more steps of the batch task anomaly detection information generation method described above may be performed. Alternatively, in other embodiments, processor 401 may be configured to execute the batch task anomaly detection information generation method by any other suitable means (e.g., by means of firmware).

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0136] To provide user interaction, the systems and techniques described herein can be implemented on a batch task anomaly detection information generation device, which includes: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the batch task anomaly detection information generation device. Other types of devices can also be used to provide user interaction; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0138] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability.

[0139] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0140] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for generating batch task anomaly detection information, characterized in that, The method includes: Obtain the historical collection sequence of the target batch task corresponding to the historical time period. The historical collection sequence includes at least one batch collection data, and the batch collection data includes: task start time, task end time and task execution duration. The historical collection sequence is input into the task duration prediction model to predict the target execution data corresponding to the target batch task. The predicted execution data includes: prediction start time, prediction end time and prediction execution duration. Obtain real-time execution data for the target batch tasks, the real-time execution data including: real-time start time, real-time end time, and real-time task duration; The three-dimensional error weights are calculated based on the real-time execution data and predicted execution data corresponding to the target batch tasks. Based on the task type corresponding to the target batch task and the three-dimensional error weight, batch task anomaly detection information corresponding to the target batch task is generated.

2. The method according to claim 1, characterized in that, The step of calculating the three-dimensional error weights based on the real-time execution data and predicted execution data corresponding to the target batch tasks includes: For each data item in the real-time execution data corresponding to the target batch task, the data error is obtained based on the difference between the data item and its corresponding data item in the predicted execution data; Based on the time limit requirements of the target batch task, determine the data weight corresponding to the data item; The three-dimensional error weights are calculated by weighted summation of the data errors and data weights corresponding to each data item.

3. The method according to claim 1, characterized in that, The step of generating batch task anomaly detection information corresponding to the target batch task based on the task type corresponding to the target batch task and the three-dimensional error weight includes: The anomaly warning level is determined based on the task type corresponding to the target batch tasks; Based on the aforementioned abnormal warning level, the obtained baseline threshold is adjusted to obtain the target threshold; The target threshold is compared with the three-dimensional error weight to obtain the threshold comparison result; Based on the threshold comparison results, batch task anomaly detection information corresponding to the target batch task is generated, and an alarm is issued according to the alarm method corresponding to the anomaly alarm level.

4. The method according to claim 1, characterized in that, The task duration prediction model is obtained through the following steps: Obtain the sample task execution sequence corresponding to the target batch task. The sample task execution sequence includes at least one execution sample data, and the execution sample data includes the sample execution start time, the sample execution end time, and the sample execution duration. Based on the execution results of each execution sample data in the sample task execution sequence, each execution sample data is filtered to obtain a training sample set, which includes at least one normal execution data. In the sample execution sequence, a set of verification samples and a set of test samples are obtained by filtering according to preset filtering conditions. The preset filtering conditions include: a filtering time period and a filtering ratio. The training sample set is input into the task duration prediction model for training to obtain the trained task duration prediction model. Based on the validation sample set and the task duration prediction model, determine the validation set loss value; Based on the loss value of the validation set, determine whether to re-train the task duration prediction model by inputting the training sample set into the model, and obtain the trained task duration prediction model, until the training termination condition is met.

5. The method according to claim 4, characterized in that, After determining whether to retrain the task duration prediction model by inputting the training sample set into the training model based on the loss value of the validation set, and obtaining the trained task duration prediction model, until the training termination condition is met, the method further includes: For each of the normal execution data, the normal execution data is input into the trained task duration prediction model to obtain normal sample prediction data; The sample error is calculated based on the normal execution data and the normal sample prediction data. Based on the sample errors corresponding to each of the normal execution data, the standard deviation and mean error are calculated. The baseline threshold is calculated based on the standard deviation of the error and the mean of the error.

6. The method according to claim 5, characterized in that, After determining whether to retrain the task duration prediction model by inputting the training sample set into the training model based on the loss value of the validation set, and obtaining the trained task duration prediction model, until the training termination condition is met, the method further includes: For each test sample data in the test sample set, the test sample data is input into the trained task duration prediction model to obtain test sample prediction data; The test sample error value is calculated based on the predicted test sample data and the test sample data. Based on the test sample error value and the benchmark threshold corresponding to the test sample data, the anomaly detection result corresponding to the test sample data is determined; Based on the anomaly detection results corresponding to each of the test sample data, statistical information is obtained, including: precision, recall, and statistical weights. Based on the statistical information, determine whether to re-train the task duration prediction model by inputting the training sample set into the model, and obtain the trained task duration prediction model, until the training termination condition is met.

7. The method according to claim 4, characterized in that, Before obtaining the sample task execution sequence corresponding to the target batch tasks, wherein the sample task execution sequence includes at least one execution sample data, and the execution sample data includes the sample execution start time, sample execution end time, and sample execution duration, the following is also included: Obtain at least one historical execution sequence corresponding to the target batch task, wherein the historical execution sequence includes at least one historical execution data; Based on the historical running cycle data corresponding to the target batch tasks, determine the filtering time window and time step. The historical execution sequence is divided according to the filtering time window to obtain at least one sample task execution sequence, so that the task duration prediction model can predict batch task prediction data corresponding to the target time of the time step based on the sample task execution sequence.

8. A batch task anomaly detection information generation device, characterized in that, The device includes: The acquisition sequence module is used to acquire the historical acquisition sequence of the target batch task corresponding to the historical time period. The historical acquisition sequence includes at least one batch acquisition data, and the batch acquisition data includes: task start time, task end time and task execution duration. The execution data prediction module is used to input the historical collection sequence into the task duration prediction model to predict the predicted execution data of the target time corresponding to the target batch task. The predicted execution data includes: prediction start time, prediction end time and prediction execution duration. The execution data acquisition module is used to acquire real-time execution data of the target batch task for the target time. The real-time execution data includes: real-time start time, real-time end time, and real-time task duration. The three-dimensional error calculation module is used to calculate the three-dimensional error weights based on the real-time execution data and predicted execution data corresponding to the target batch tasks. An anomaly information generation module is used to generate batch task anomaly detection information corresponding to the target batch task based on the task type corresponding to the target batch task and the three-dimensional error weight.

9. A batch task anomaly detection information generation device, characterized in that, The batch task anomaly detection information generation device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the batch task anomaly detection information generation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are used to cause a processor to execute the batch task anomaly detection information generation method according to any one of claims 1-7.

Images