Data-Driven Checkpoint Selector for Database Jobs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large analytical jobs in big data platforms face challenges such as high failure rates, substantial latency in solver-based techniques, and suboptimal query plans due to exponential error propagation, leading to increased costs and resource consumption.
Innovation Solution
A data-driven technique is employed to select checkpoints in recurrent database jobs, optimizing checkpoint placement using machine learning to minimize intermediate data and recovery time, and split large jobs into smaller ones based on temporal indicators and attribute values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional solver-based techniques are used to determine checkpoint frequency, then fault tolerance is improved, but substantial latency is introduced
Solution Approach 1:
The system pre-calculates checkpoint frequencies during an offline training phase using historical job data, creating a trained model that stores optimal checkpoint strategies. During actual job execution, the system simply queries this pre-computed model rather than performing real-time optimization calculations, thereby eliminating the latency issue while maintaining fault tolerance benefits
Solution Approach 2:
The patent replaces the conventional solver-based mathematical optimization approach with a machine learning model that has been trained offline. This substitution transforms the real-time computational problem into a simple model query operation, dramatically reducing latency while preserving the ability to determine optimal checkpoint frequencies for fault tolerance
2Reliability
If intermediate outputs are preserved for all sub-queries, then recovery from task failures is improved, but resource consumption increases
Solution Approach 1:
The system applies different checkpointing strategies to different operators within a query plan based on their individual characteristics. The trained model evaluates each operator's failure probability and impact on recovery time, selectively applying checkpointing only where necessary rather than uniformly across all operators, thus improving recovery capability while minimizing resource consumption
Solution Approach 2:
The system dynamically adjusts checkpoint frequency and placement based on parameters such as operator failure probability, intermediate data size, and recovery time impact. The trained model uses these parameters to determine optimal checkpoint strategies for each operator, balancing recovery improvement against resource consumption
3Device complexity
If query optimizer estimates are used for large jobs, then query planning is simplified, but error propagation increases exponentially
Solution Approach 1:
The system replaces traditional query optimizer cardinality estimation with machine learning-based predictions. The trained model learns accurate cardinality estimates from historical execution data, providing precise predictions without the exponential error propagation characteristic of conventional optimizers, while maintaining query planning simplicity
4Productivity
If large analytical jobs are executed, then data processing capability is improved, but failure rates increase
Solution Approach 1:
The system automatically divides large analytical jobs into smaller sub-jobs or stages with intermediate checkpoints. The trained model identifies optimal split points based on failure probability and recovery time considerations, allowing the system to process large datasets while maintaining lower failure rates through manageable segmentation and the ability to restart from checkpoint points
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Techniques are described herein that are capable of selecting checkpoints of a database job. For instance, at compile time, temporal indicators associated with the query plans of the database job are determined. Each temporal indicator indicates first and second subsets of stages of the respective query plan. Values of attributes of each stage in at least each first subset are predicted using a machine learning technique. At the compile time, candidate stage(s) for each query plan are identified based on the respective candidate stage being a child of stage(s) in the corresponding second subset or not being a child of another stage in the respective query plan. The candidate stage(s) for each query plan are selectively chosen as respective checkpoint(s) based on whether the values of the attributes of each stage in at least the first subset of the stages of the respective query plan satisfy one or more criteria.