Distributed File System Data Placement Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed file systems face challenges in determining the optimal placement destination for object data to speed up processing that uses multiple pieces of data simultaneously, especially when data is stored newly, as existing methods rely on access characteristics and similarity in data content, which may not always accurately predict future usage patterns.
Innovation Solution
An information processing device that includes a storage location storage unit, generation information storage unit, relevance information storage unit, relevant data acquisition unit, and placement destination determination unit, which acquires and analyzes generation information and relevance data to determine the most suitable storage location for object data based on the storage location of similar and relevant data already stored in the system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data placement is determined based on access characteristics and data content similarity, then data can be stored in distributed file system, but the placement destination cannot be optimized for future simultaneous processing
Solution Approach 1:
The patent applies preliminary action by determining data placement destinations in advance based on generation information and relevance relationships before the data is actually accessed or processed. The master device analyzes the relevance between object data and other data that may be used simultaneously, and pre-determines optimal placement destinations across multiple racks, thereby optimizing future processing speed without waiting for actual access patterns to emerge.
2Reliability
If multiple copies of object data are stored in different racks, then fault tolerance is improved, but the cost of writing and updating increases
Solution Approach 1:
The patent applies local quality by strategically placing multiple copies of object data in different racks based on their relevance to other data. Instead of uniform distribution, the system identifies which racks contain related data and places copies locally in those racks, thereby maintaining fault tolerance while reducing the cost of writing and updating by concentrating related data access in specific locations.
3Speed
If data is placed in the same rack to speed up processing, then access speed is improved, but the network bandwidth requirement increases
Solution Approach 1:
The patent applies preliminary action by pre-determining optimal rack placements for object data and its copies based on relevance analysis. The master device identifies which racks contain data that is likely to be accessed simultaneously with the object data, and proactively places copies in those racks before access occurs. This eliminates the need for dynamic data movement during processing and reduces overall network bandwidth consumption while maintaining fast access speeds.
Data Source
AI summary
An information processing device can determine, as a placement destination of object data, a storage location which is the most suitable for speeding up future processing which uses a plurality of pieces of data including the object data simultaneously, even when the object data is stored newly before use of it.