Distributed Computing Resource Allocation via Network Distance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed computing systems, the hierarchical network structure leads to increased data flow between network switches, causing a bottleneck in performance and insufficient network bandwidth, especially when compute nodes, storage devices, and drive casings are across different network segments.
Innovation Solution
A distributed computing system is configured to minimize network distance by selecting storage devices and drive casings based on the number of network switches between compute nodes, storage devices, and drive casings, using a managing unit to allocate storage regions and optimize resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a hierarchical network structure is used to connect compute nodes, storage devices, and drive casings across different network segments, then the system can achieve wide-range connectivity and resource sharing, but the amount of data flowing between network switches increases, causing performance bottlenecks and insufficient network bandwidth
Solution Approach 1:
The patent applies local quality by allocating storage devices and drive casings to compute nodes based on their network distance, prioritizing local connections within the same network segment. This creates optimized local data paths that reduce cross-segment traffic and improve overall network performance while maintaining system-wide connectivity
2Adaptability or versatility
If storage resources are shared between network segments via multiple network switches, then resource utilization increases, but the number of network switches in the data path increases, reducing network bandwidth and increasing latency
Solution Approach 1:
The patent implements preliminary action by pre-calculating and storing network distance information between compute nodes, storage devices, and drive casings in a management database. This allows the system to make optimal allocation decisions in advance, selecting storage resources that minimize network hops and latency before actual data transfer occurs
3Ease of operation
If a managing unit allocates storage regions without considering network distance, then resource allocation simplicity is maintained, but network performance bottlenecks occur due to excessive data flow between network switches
Solution Approach 1:
The patent implements feedback by continuously monitoring network distance metrics and using this information to dynamically optimize storage resource allocation. The management unit receives feedback about network topology and automatically adjusts allocation decisions to minimize data flow through bottleneck network switches, maintaining both simplicity and performance
Data Source
AI summary
In a distributed computing system, a bottleneck for performance of a network is avoided and a high-performance scalable resource management function is achieved. The distributed computing system includes a plurality of components connected to each other via a network. Each of the components includes a plurality of compute nodes, a plurality of drive casings, and a plurality of storage devices. The network includes a plurality of network switches and is configured in layers. When a storage region is to be allocated to a compute node among the compute nodes, a managing unit selects, from the storage devices, a storage device related to the storage region to be allocated or selects, from the drives, a drive casing related to the storage region to be allocated, based on a network distance between two of the compute node, the storage device, and the drive casing.