Batch Process Forecasting for Real-Time Schedule Delay Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional batch processing systems lack the ability to predict dynamic behavior and provide early warnings of potential interruptions and deadline delays, leading to uncertainties and adverse impacts on business operations.
Innovation Solution
A system and method for predicting batch processes by generating batch graphs from user inputs, applying process flow mining techniques to derive metadata, and using forecasting models to generate revised schedules that adapt to real-time changes and anomalies, with proactive notifications for unexpected delays.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional batch schedulers and monitoring tools are used, then batch processes can be executed and monitored, but the system cannot predict dynamic behavior or provide early warnings of potential interruptions and deadline delays
Solution Approach 1:
The system performs preliminary actions by collecting run history data and workload data before batch execution, building batch graphs and training forecasting models in advance. This enables the system to predict future batch behavior and provide early warnings before actual execution issues occur, rather than merely monitoring after the fact.
Solution Approach 2:
The system implements feedback mechanisms by continuously collecting actual run history data and comparing it with predicted values from forecasting models. This feedback loop allows the system to refine predictions, identify deviations from expected behavior, and provide accurate early warnings about potential interruptions and delays.
2Loss of time
If descriptive analysis and statistical extrapolations are provided, then batch process information is available, but early warning of potential interruptions and deadline delays cannot be given
Solution Approach 1:
The system replaces traditional mechanical statistical extrapolation methods with intelligent forecasting models including regression models and time-series analysis. These advanced analytical techniques substitute simple statistical calculations with more sophisticated prediction algorithms that can capture complex patterns in batch job execution, workload variations, and inter-job dependencies.
Solution Approach 2:
The system changes parameters by transitioning from static batch scheduling to dynamic prediction that incorporates multiple varying parameters including run history data, workload data, batch graph structures, and forecasting model outputs. This allows the system to adapt predictions to changing conditions and provide timely warnings with improved precision.
3Productivity
If batch schedulers focus on defining and automating execution, then batch processes can be automated, but dynamic behavior prediction and early warning capabilities are lacking
Solution Approach 1:
The system achieves multi-functionality by combining traditional batch scheduling automation with intelligent prediction capabilities. The same platform that defines and automates batch execution now also collects behavioral data, builds predictive models, and provides early warnings, eliminating the need for separate monitoring and prediction systems.
Solution Approach 2:
The system implements self-service by enabling batch processes to essentially predict their own behavior through the collected run history data and forecasting models. The system automatically generates predictions and warnings without requiring external intervention, allowing the batch processing infrastructure to monitor and predict its own performance characteristics.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This disclosure relates generally to method and system for predicting batch processes. Conventional batch schedulers provide a single point of control for defining and monitoring background executions in a distributed network. The method of the present disclosure obtains a set of batch jobs from one or more users to generate a set of batch graphs by deriving a metadata. Further, a set of batch models is generated for the set of batch graphs. The set of batch models includes at least one of a forecasting model, a job-job regression model, and a job-workload regression model. Further, a batch job schedule is generated for the set of batch graphs to predict a revised batch job schedule with a real time feed and the set of batch models. Additionally, a proactive notification is sent to each user alarming one or more unexpected delays indicating the revised batch job schedule.