Parallel Computing Control Apparatus for Accurate Data Transfer Period Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In parallel computing systems, the inefficiency in using nodes arises from fragmented idle computing resources due to varying job execution times and inaccurate estimation of data transfer periods, leading to suboptimal scheduling and frequent job interruptions.
Innovation Solution
A parallel computing control apparatus that determines a relay node on the path between a source and destination node, obtains an index value indicating resource usage, and calculates an estimated transfer period based on this index value and memory usage, improving the accuracy of transfer period estimation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data transfer period is estimated using static information such as hardware performance, then the estimation process is simple, but the accuracy of transfer period estimation is low
Solution Approach 1:
The patent implements feedback by measuring actual transfer periods during runtime and using these measurements to update and refine the estimation model. The system continuously monitors real transfer performance and feeds this information back to improve future predictions, transforming static estimation into a dynamic, self-improving process that increases accuracy without proportionally increasing complexity.
Solution Approach 2:
The patent performs preliminary actions by pre-calculating and storing transfer period estimates based on historical data and system characteristics before actual data transfer operations. This allows the system to have ready-made estimation values available, reducing the need for complex real-time calculations while maintaining reasonable accuracy.
2Productivity
If job scheduling uses fragmented idle computing resources, then resource utilization efficiency improves, but job execution stability deteriorates due to frequent interruptions
Solution Approach 1:
The patent applies dynamics by making the job scheduling system adaptable and flexible rather than rigid. It dynamically adjusts scheduling decisions based on real-time resource availability, transfer period estimates, and job characteristics. This allows the system to optimize for resource utilization while maintaining job execution stability by responding to changing conditions rather than following fixed schedules.
Solution Approach 2:
The system performs preliminary actions by pre-estimating transfer periods and evaluating resource availability before making scheduling decisions. This advance planning allows jobs to be scheduled in a way that minimizes interruptions while still utilizing fragmented idle resources effectively, as the system can anticipate potential conflicts and avoid them proactively.
3Adaptability or versatility
If data is transferred between nodes during job execution, then resource allocation flexibility improves, but transfer period prediction accuracy worsens
Solution Approach 1:
The patent uses feedback to continuously refine transfer period predictions based on actual transfer performance observed during runtime. As data transfers occur between nodes during job execution, the system measures real transfer times and uses this feedback to update its prediction models, improving accuracy despite the dynamic and flexible nature of resource allocation.
Solution Approach 2:
The system applies parameter changes by adjusting estimation parameters based on observed system behavior and transfer characteristics. As resource allocation patterns change during execution, the system modifies its prediction parameters to reflect current conditions, maintaining accuracy even as flexibility increases.
Data Source
AI summary
A parallel computing control apparatus determines, among a plurality of nodes, relay nodes located on a path from a first node allocated to a job to a second node. The parallel computing control apparatus obtains an index value indicating a use state of resources in the relay nodes. The parallel computing control apparatus calculates an estimated value of a transfer period taken to transfer data on the job from the first node to the second node, on the basis of the index value and the memory usage of the job.