Compressed Document Search with Snippets, Shells, and Assembly
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data storage and maintenance costs increase with growing data volumes, and compression techniques reduce data usability, leading to inefficient searching and accessing, which impacts organizational efficiency.
Innovation Solution
A data compression store and search system (DCS) identifies redundancies in documents, generating snippets and shells that allow for efficient storage and retrieval, using document assembler to reconstruct original documents from compressed data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compression is used to reduce data storage, then storage capacity requirements decrease, but data searching and accessing becomes slow and cumbersome
Solution Approach 1:
The patent segments data into fixed-width field formats with defined structures, allowing the system to process and search specific portions of compressed data without decompressing entire records. This segmentation enables efficient querying by targeting only relevant data segments while maintaining compression benefits.
Solution Approach 2:
The patent introduces an intermediary indexing layer that maintains searchable metadata about compressed data structures. This intermediary layer allows the system to locate and access specific compressed data elements efficiently without requiring full decompression, bridging the gap between compression efficiency and searchability.
2Quantity of substance
If more data is stored, then data volume increases, but storage capacity, computing resources, and personnel requirements increase
Solution Approach 1:
The patent changes the structural parameters of stored data by implementing fixed-width fields and standardized formats. This parameter change enables more efficient storage density while maintaining query capability, allowing the system to store more data with reduced infrastructure requirements compared to traditional variable-length formats.
3Quantity of substance
If compression reduces data storage, then storage costs decrease, but data usability is reduced
Solution Approach 1:
The patent performs preliminary organization of data into standardized fixed-width formats during the compression process. This preliminary action ensures that compressed data maintains a structured layout that facilitates efficient searching and accessing, preventing the loss of data usability that typically accompanies compression.
Data Source
AI summary
Various embodiments for a data compression store and search system are described herein. An embodiment operates by receiving a query to search a plurality of documents. It is determined that the plurality of documents are stored in a compressed format comprising a plurality of shells and a plurality of snippets. A first snippet that includes the query identifier is identified, and a first shell including a reference to the first snippet is identified. A readable version of a first document of the plurality of documents is assembled from the first shell. The assembled readable version of the first document is provided responsive to the query.