Dynamic Library Linking for Hybrid Compute Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Massively parallel computer systems face challenges in optimizing run-time characteristics to meet system and user requirements, leading to suboptimal performance due to the lack of dynamic library selection and configuration based on processor architectures.
Innovation Solution
A job scheduler evaluates job profiles and user inputs to determine the most suitable libraries for an application, configuring the environment to link with optimized libraries during run-time, thereby improving system performance by utilizing either general or special purpose processors based on availability and historical data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a fixed library configuration is used for all computing jobs, then system complexity is reduced and ease of operation is improved, but system performance and resource utilization deteriorate due to inability to optimize for specific processor architectures
Solution Approach 1:
The patent implements dynamic library selection by evaluating job profiles and determining optimal libraries at runtime based on processor architecture and job requirements, transforming the static library configuration into a dynamic adaptive system that optimizes performance for each computing job
Solution Approach 2:
The patent performs preliminary evaluation of job profiles and pre-determines optimal library configurations before executing computing jobs, allowing the system to prepare and configure the appropriate runtime environment in advance, thus improving execution efficiency without adding complexity during job execution
2Speed
If dynamic library selection is implemented based on processor architecture, then computing job execution speed is improved, but system complexity increases due to multiple library configurations
Solution Approach 1:
The patent enables the computing job to self-determine its optimal library configuration by evaluating its own job profile and requirements, allowing the system to automatically select appropriate libraries without manual intervention or complex external configuration management
Solution Approach 2:
The patent changes the runtime environment parameters dynamically by modifying library path configurations and environment variables based on the evaluated job profile and detected processor architecture, enabling the same system to adapt to different computational requirements through parameter adjustment rather than structural complexity
3Productivity
If compute nodes are allocated without considering job-specific optimizations, then resource allocation simplicity is maintained, but compute node utilization efficiency deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where job profiles and execution results are evaluated to determine optimal library configurations, and this information feeds back into the resource allocation process, allowing the system to continuously improve compute node utilization based on actual performance data and job characteristics
Data Source
AI summary
Embodiments of the invention provide techniques that improve resource management on a massively parallel computing system having a plurality of hybrid compute nodes. For example, a job scheduler may be provided which determines a library to link to an application based on system and user requirements. In one embodiment, the libraries may provide optimizations for job execution time, and also provide optimizations directed towards a specific processor architecture. Once the library is determined, the job scheduler may configure the environment of the application so that the application links with the optimized library during run-time. Doing so may improve overall system performance of the massively parallel computing system.