Compact Key-Value Structure With 64-Bit Direct Access Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer systems face inefficiencies in data storage and retrieval due to the limitations of memory hierarchies, where higher performance memories have lower capacity and are more expensive, leading to suboptimal data access and storage strategies.
Innovation Solution
A compact in-memory key-value data structure is generated, utilizing a Direct Access Map (DAM) that aligns data with a 64-bit word size, encoding data using 7-bit ASCII for strings and optimizing storage to minimize memory usage, allowing efficient retrieval through singular data access operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data is stored in high-performance memory (cache/registers), then data access speed is improved, but memory capacity is reduced and cost increases
Solution Approach 1:
The patent segments data into fixed-size records that align with the natural word size of the processor (e.g., 64-bit words). Each record is a self-contained unit that can be efficiently stored and retrieved. This segmentation allows the memory system to maximize the use of high-performance cache memory by storing complete records in contiguous memory locations, thereby improving data access speed while optimizing the limited capacity of high-performance memory.
Solution Approach 2:
The patent changes the parameter of data representation by encoding all data (including variable-length strings and nested structures) into fixed-width binary records with a standardized format. This parameter change enables efficient packing of data into memory words, maximizing memory capacity utilization and improving access speed by allowing direct word-aligned access without complex addressing logic.
2Quantity of substance
If data is stored in a compact format aligned with word size, then memory efficiency is improved, but data structure complexity increases
Solution Approach 1:
The patent applies preliminary action by pre-processing data during the storage phase to convert it into the fixed-size record format. A record builder component systematically encodes data elements (strings, numbers, nested structures) into standardized binary representations with fixed field widths. This preliminary transformation simplifies subsequent data retrieval operations, as records can be directly accessed and manipulated without complex parsing or formatting logic during runtime.
Solution Approach 2:
The patent introduces an intermediary record builder component that mediates between the application's data structures and the underlying memory storage. This intermediary translates diverse data types into a unified fixed-size record format, handling the complexity of data encoding during storage while presenting a simplified interface for data retrieval. The record builder acts as a buffer that absorbs structural complexity during data ingestion, allowing efficient straightforward access during read operations.
Data Source
AI summary
The disclosed embodiments relate to systems and methods for generating a compact in memory key-value data structure/database which may or may not be hierarchical, i.e., include nested key-value data structures, which can be utilized for other applications. The key-value data to be encoded and stored in the generated database may be extracted from another database, such as a relational database, or other source. It will be appreciated that the disclosed embodiments may be used to encode and store any data, not just key-value data sets, which would benefit from the compact long-aligned form generated thereby. The resultant compact data structure may be efficiently accessed to directly retrieve particular data or search for particular data and such access may be enabled using a defined notation/syntax and/or one or more intermediary processes or programs.