Dynamic Physical Node Allocation for Execution Plans
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for allocating physical nodes for processes in execution plans often result in insufficient resources during execution, leading to delays or failures, as nodes are pre-allocated before the execution plan is prepared, and overloading occurs when too many processes run on a single node.
Innovation Solution
The approach involves generating an execution plan with dummy node names, then dynamically allocating actual physical nodes after plan preparation, allowing processes to run on multiple nodes if necessary, ensuring sufficient resources are available.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If physical nodes are pre-allocated before execution plan generation, then node assignment is determined in advance, but resource insufficiency and node overload occur during execution
Solution Approach 1:
The patent implements dynamic node allocation where the system transitions from static pre-allocated nodes to dynamically selected nodes based on real-time resource availability. The job executor queries the resource management tool at execution time to obtain currently available nodes, ensuring that nodes have sufficient resources when processes are actually launched. This dynamic approach resolves the contradiction by adapting node selection to actual system state rather than relying on outdated pre-allocations.
Solution Approach 2:
The system establishes a feedback loop where the job executor continuously monitors resource availability through the resource management tool before and during execution. The resource requirement information is fed back to the job executor, which then adjusts node selection based on current resource status. This feedback mechanism ensures that node allocation reflects actual system conditions, preventing both resource insufficiency and overload while maintaining execution reliability.
2Productivity
If multiple processes are assigned to a single node, then resource utilization increases, but node overload occurs leading to execution delays
Solution Approach 1:
The patent employs dynamic node allocation where the job executor queries the resource management tool at execution time to determine current node availability and resource capacity. Instead of statically assigning multiple processes to nodes regardless of state, the system dynamically selects nodes that currently have sufficient resources. This prevents node overload and subsequent execution delays while maintaining high resource utilization by efficiently matching processes to available capacity.
Solution Approach 2:
The system changes the parameter of node selection from fixed pre-allocated node identifiers to dynamically determined node selections based on real-time resource metrics. The job executor evaluates current resource availability, node capacity, and process requirements to optimize the distribution of processes across nodes. This parameter change enables the system to adapt process-to-node mapping based on actual system state, preventing overload while maximizing resource utilization.
3Ease of operation
If physical nodes are allocated in advance via configuration file, then node selection is simplified, but the allocation cannot adapt to changing resource availability
Solution Approach 1:
The patent introduces the resource management tool as an intermediary between the job executor and physical nodes. Instead of directly hardcoding node names in configuration files, the system uses the resource management tool to mediate node selection. The tool maintains knowledge of resource availability and translates high-level job requirements into specific node assignments at execution time. This intermediary approach preserves ease of operation by abstracting away node management complexity while simultaneously enabling adaptability through real-time resource status awareness.
Solution Approach 2:
The system transitions from static configuration file node allocation to dynamic node selection orchestrated by the resource management tool. The configuration file specifies resource requirements rather than fixed node assignments, allowing the system to adapt to changing resource availability. The job executor queries the resource management tool to obtain current node availability information and dynamically selects appropriate nodes, combining the simplicity of configuration-based specification with the adaptability of runtime decision-making.
Data Source
AI summary
Provided are a computer program product, system, and method for allocating physical nodes for processes in an execution plan. An execution plan is generated indicating a plurality of processes. A resource requirement is generated indicating requested physical nodes and an assignment of the processes to execute on the requested physical nodes. A determination is made from the resource requirement of a resource allocation of physical nodes for the requested physical nodes and the processes. The execution plan is updated to generate an updated execution plan indicating the physical nodes on which the processes will execute according to the received resource allocation.