Parallel Data Storage via File Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage technologies face inefficiencies and high latency when handling large data files, leading to potential irretrievable loss of data due to mismatched transmission and storage rates.
Innovation Solution
A computing system and method that divides large data files into evenly sized portions, each stored simultaneously across multiple digital data repositories connected via a digital bus organized into logical or physical channels, ensuring optimal throughput and minimizing latency by using a common sector size and RAID-like architecture.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is stored sequentially on a single repository, then data structure simplicity is maintained, but storage latency increases and throughput decreases
Solution Approach 1:
The patent divides large data files into multiple equal-sized portions and distributes them across multiple digital data repositories simultaneously. This segmentation enables parallel storage operations, significantly increasing throughput and reducing latency compared to sequential storage on a single repository.
Solution Approach 2:
The patent transitions from single-dimension sequential storage to multi-dimensional parallel storage by organizing data across multiple repositories in a distributed manner, utilizing the additional dimension of parallelism to improve storage performance.
2Speed
If transmission rate exceeds storage rate, then data transmission efficiency is improved, but data loss increases due to irretrievable loss
Solution Approach 1:
The patent performs preliminary data preparation by dividing files into properly sized portions that match storage capacity characteristics before transmission begins. This ensures that storage operations can keep pace with transmission, preventing data loss while maintaining high transmission rates.
3Productivity
If multiple data repositories are used simultaneously, then storage rate and efficiency increase, but system complexity increases
Solution Approach 1:
The patent assigns specific portions of data to specific repositories based on predetermined criteria, creating a consistent mapping pattern. This local quality approach simplifies the overall system management while enabling parallel operations across multiple repositories.
Solution Approach 2:
The patent uses repositories with common sector sizes and organizes data portions into uniform units, creating homogeneity across the distributed storage system. This standardization reduces complexity in data management and access operations.
4Productivity
If file size is not evenly divisible by number of repositories, then data structure flexibility is maintained, but storage efficiency decreases
Solution Approach 1:
The patent modifies the data structure by adding padding or adjusting file organization to make the total file size evenly divisible by the number of repositories. This parameter change enables perfect parallel distribution across all repositories, maximizing storage efficiency while maintaining data integrity.
Data Source
AI summary
A computing system has a central processing unit (CPU), a plurality of n digital data repositories R1-Rn, having a common sector size, a file system, and a digital bus connecting the CPU and the plurality of n digital data repositories R1-Rn. The file system receives an original file to be stored of FS bits in size, creates a new file of NFS bits, comprising all the data structure of the original file, NFS evenly divisible by n and by the common sector size, divides the NFS by n, creating n portions P1-Pn, and stores one portion to each disk in the plural array simultaneously.