Encoded Storage Access Interface for Random Block Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Information processing and storage systems face challenges in efficiently managing the vast amounts of digital data, particularly in cloud data centers, where existing encoding techniques fail to effectively reduce data volume and storage costs.
Innovation Solution
The technology employs a block-based encoding and decoding method using multiple processing cores, where encoded blocks include headers and payloads that represent event parameters, allowing for random access and integration with existing storage access interfaces, enabling efficient data compression and decompression with minimal modifications to existing storage methods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If block-based encoding is implemented to reduce storage volume, then storage costs are reduced, but system complexity increases due to modified data structures and access patterns
Solution Approach 1:
The patent divides data into fixed-size blocks for independent encoding and processing. Each block is handled as a discrete unit with its own encoding/decoding operations, allowing parallel processing across multiple cores while maintaining manageable complexity through modular block-level operations rather than complex whole-data processing
Solution Approach 2:
The patent introduces metadata structures including block size arrays and modified dataset identifiers that act as intermediaries between the encoding system and existing storage interfaces. These metadata layers enable random access and interface compatibility without requiring fundamental changes to underlying storage mechanisms, thus reducing overall system complexity
2Productivity
If multiple processing cores are used to speed up encoding and decoding, then processing speed increases, but coordination overhead and system complexity increase
Solution Approach 1:
Data is divided into independent blocks that can be processed by different cores simultaneously. Each core handles specific blocks independently, eliminating the need for complex inter-core coordination during encoding/decoding operations, thus achieving parallel processing speedup with minimal overhead
Solution Approach 2:
Each processing core independently performs encoding or decoding operations on its assigned blocks without requiring synchronization or coordination with other cores. The block-based structure enables self-contained operations that automatically scale with the number of available cores, maximizing productivity while minimizing coordination complexity
3Ease of operation
If random access is enabled in encoded data streams, then data retrieval flexibility improves, but encoding structure complexity increases
Solution Approach 1:
The data stream is segmented into fixed-size blocks with consistent boundaries. Random access is achieved by calculating block indices from byte offsets, allowing direct navigation to any block without decoding preceding blocks. This segmented structure provides random access capability while maintaining relatively simple encoding compared to variable-length or hierarchical structures
Solution Approach 2:
Block size information is pre-calculated and stored in metadata arrays during the encoding phase. This preliminary organization of block size data enables efficient random access during decoding without requiring complex runtime analysis or re-encoding, thus improving retrieval flexibility while keeping the encoding structure manageable
4Adaptability or versatility
If existing storage interfaces are maintained for compatibility, then ease of integration improves, but ability to leverage encoding optimizations is reduced
Solution Approach 1:
Metadata structures including block size arrays and modified dataset identifiers serve as intermediaries between existing storage interfaces and the encoded block structure. These metadata layers allow standard storage operations to work with encoded data without requiring interface changes, thus maintaining compatibility while preserving encoding optimization benefits through the metadata abstraction layer
Data Source
AI summary
A combination of a block-oriented encoder and decoder with a modified dataset identifier that is associated with an encoded block size are used to perform block-based encoding and decoding operations. The encoding process may generate optional metadata that includes an array of encoded block sizes to support random access into the stream or group of encoded blocks during the decoding process. The modified dataset identifier associates the original dataset identifier with the block size used by the encoder.