Lexically Sorted Text Storage for Faster Cloud Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing systems face inefficiencies in storing and retrieving large datasets, particularly with textual data, due to resource-intensive search processes and storage challenges, especially in non-relational data structures like columnar data structures.
Innovation Solution
A method involving a primary file system that organizes and compresses textual data in lexical order, dividing it into subblocks and page blocks for efficient storage and searching, along with a secondary data structure for managing updates and modifications to prevent memory inefficiencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored in cloud computing systems, then storage capacity is increased, but storage space becomes scarce and retrieval becomes time-consuming
Solution Approach 1:
The patent divides the dataset into multiple subblocks, with each subblock containing a predetermined number of data entries. This segmentation enables efficient random access to specific portions of the data without requiring sequential scanning of the entire dataset, thereby reducing retrieval time while maintaining large storage capacity
Solution Approach 2:
The patent performs preliminary sorting of data entries in lexical order before storing them in subblocks. This pre-organization of data enables faster search and retrieval operations by allowing binary search algorithms to be applied within subblocks, significantly reducing the time required to locate specific data entries
2Quantity of substance
If large datasets are stored, then data volume is increased, but searching and retrieving desired data becomes resource-intensive
Solution Approach 1:
By dividing the large dataset into multiple smaller subblocks, the system can perform searches in parallel across subblocks or quickly jump to specific subblocks based on lexical ordering, reducing the computational resources required for searching compared to scanning the entire dataset sequentially
Solution Approach 2:
The patent transforms the data structure by organizing entries in lexical order within subblocks, changing the organizational parameter from arbitrary storage to sorted arrangement. This enables efficient binary search algorithms to be applied, significantly reducing the time and computational resources needed to retrieve data from large datasets
3Ease of manufacture
If columnar data structures with text are used, then data organization is improved, but analytics become very resource intensive
Solution Approach 1:
The patent divides the columnar text data into multiple subblocks, each containing a manageable number of data entries. This segmentation allows analytics operations to be performed on smaller subsets of data independently, reducing the overall computational resources required compared to processing the entire columnar structure at once
Solution Approach 2:
The patent performs preliminary sorting of text data in lexical order before analytics operations. This pre-organization enables more efficient text-based analytics operations and allows for early filtering and indexing, reducing the resources needed during actual analytics execution
Data Source
AI summary
A method of and system of efficient storage of data entries containing textual data is disclosed. The method may include accessing a plurality of data entries in a dataset, arranging the plurality of data entries in the dataset in a lexical order, placing a predetermined number of the plurality of data entries in each of a plurality of subblocks, performing data compression on the plurality of data entries in each of the plurality of subblocks to reduce redundancy in the plurality of data entries and create compressed data entries, placing one or more subblocks in each of a plurality of page blocks, and storing each of the plurality of page blocks in a storage device to provide efficient searching and improved functionality for the dataset.