Indexed Dual-File Data Reading for Random Block Modifications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Block storage systems face inefficiencies in data storage and reading due to frequent random modifications, particularly for small data lengths, leading to read/write amplification and complex operations.
Innovation Solution
A data processing method that distinguishes between first and second storage files based on data length, using different storage modes, and employs index information to read sub-data from these files separately, optimizing the reading process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If block storage system uses gradual writing of modified data into a single storage file, then data modification is implemented, but data storage efficiency and reading efficiency are affected due to multi-layer structure
Solution Approach 1:
The patent divides the storage system into multiple storage files (first storage file and second storage file) based on data length characteristics. Small data is stored in the first storage file while large data is stored in the second storage file, segmenting the storage structure to avoid multi-layer complexity and improve efficiency.
2Productivity
If block storage system stores all data in a single storage file, then data modification is simplified, but reading efficiency decreases due to need to search through entire file for random modifications
Solution Approach 1:
The patent segments data into different storage files based on length thresholds. By separating small data (first storage file) from large data (second storage file), the system reduces search time during random modifications by only searching relevant files, thereby improving reading efficiency.
Solution Approach 2:
The patent applies different storage strategies to different data segments. The first storage file uses a structure optimized for small data with frequent modifications, while the second storage file handles large data differently. This local optimization improves overall reading efficiency by matching storage structure to data characteristics.
3Productivity
If block storage system uses uniform storage mode for all data, then storage structure is simple, but read/write amplification increases for small data with frequent random modifications
Solution Approach 1:
The patent applies different storage modes to different data segments based on their characteristics. Small data uses a storage mode optimized for frequent modifications, while large data uses a different mode. This local differentiation reduces read/write amplification by avoiding unnecessary operations on data that don't require them.
Data Source
AI summary
A data processing method, an electronic device, and a medium are provided. The method includes: in response to receiving a data reading request, determining a first storage file and a second storage file according to an identifier of data to be read in the request, where a storage mode of the first storage file is different from that of the second storage file, and a data length of the second storage file is greater than that of the first storage file; determining an index result according to a position information of the data to be read and first index information of the first storage file; reading first sub-data from the first storage file when the index result indicates that the first storage file includes the first sub-data; and reading second sub-data from the second storage file according to the position information and index information of the second storage file.