Extensible Skeletons for Parallel In-Memory Database Execution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In-memory databases face challenges in leveraging parallelism effectively, as developers must explicitly program and synchronize custom code, leading to suboptimal parallel execution and performance opportunities being missed.
Innovation Solution
An execution framework that allows developers to write sequential computational logic, which is then transformed into parallel execution plans using extensible algorithmic skeletons, enabling the system to automatically optimize and execute custom business logic in parallel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If developers explicitly program parallelism and synchronization in custom code, then parallel execution capability is achieved, but programming complexity and difficulty increase
Solution Approach 1:
The patent introduces an intermediary layer (the execution framework with skeletons) between the developer's sequential code and the parallel execution environment. This intermediary automatically handles the parallelization process, allowing developers to write simple sequential code while the system translates it to efficient parallel execution without requiring developers to directly manage parallelism or synchronization complexity
Solution Approach 2:
The execution framework performs self-service by automatically analyzing sequential code and generating appropriate parallel execution plans without requiring explicit developer intervention. The system autonomously determines how to parallelize the code, manage synchronization, and optimize performance, thereby resolving the contradiction between achieving parallel execution and maintaining programming simplicity
2Reliability
If developers explicitly define synchronization points, then correct parallel execution is ensured, but development time and effort increase
Solution Approach 1:
The execution framework autonomously identifies critical sections and insertion points for synchronization within sequential code, automatically generating the necessary synchronization constructs without requiring developers to manually define them. This self-service approach ensures correct parallel execution while eliminating the time-consuming manual synchronization definition process
Solution Approach 2:
The patent replaces the manual mechanical process of identifying and inserting synchronization points with an automated analysis system. The framework uses static analysis and runtime information to automatically determine where synchronization is needed, substituting the time-consuming manual process with an automated mechanism that ensures correctness without developer intervention
3Ease of operation
If the system executes custom code sequentially, then programming simplicity is maintained, but performance opportunities are lost
Solution Approach 1:
The execution framework acts as an intermediary that transparently transforms sequential code into parallel execution without requiring changes to the developer's code. This mediator layer automatically analyzes the sequential code, identifies parallelization opportunities, and generates optimized parallel execution plans, thereby maintaining programming simplicity while capturing performance opportunities
Solution Approach 2:
The system performs preliminary analysis of sequential code during compilation or load time to identify parallelization opportunities before execution. By pre-processing the code to determine optimal parallelization strategies, the system prepares execution plans that can be efficiently executed in parallel without requiring developers to rewrite code or manually analyze performance opportunities
4Ease of operation
If the degree of parallelism is defined by the programmer, then control over execution is achieved, but optimal parallelism may not be realized at runtime
Solution Approach 1:
The execution framework dynamically determines the degree of parallelism at runtime based on available system resources, workload characteristics, and code analysis, rather than relying on static programmer definitions. This dynamic approach allows the system to adapt to changing conditions and realize optimal parallelism automatically, while still providing developers with control through the ability to specify preferences or constraints when needed
Data Source
AI summary
An execution framework allows developers to write sequential computational logic, constrained for the runtime system to efficiently parallelize execution of custom business logic. The framework can be leveraged to overcome limitations in executing low level procedural code, by empowering the system runtime environment to parallelize this code. Embodiments employ algorithmic skeletons in the realm of optimizing/executing data flow graphs of database management systems. By providing an extensible set of algorithmic skeletons the developer of custom logic can select the skeleton appropriate for new custom logic, and then fill in the corresponding computation logic according to the structural template of the skeleton. The skeleton provides a set of constraints known to the execution environment, that can be leveraged by the optimizer and the execution environment to generate parallel optimized execution plans containing custom logic, without the developer having to explicitly describe parallelization of the logic.