Dictionary Table Reference Store for Column Store Database Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Column store databases face inefficiencies in storing and analyzing variable-length data types, leading to high storage requirements and slow access patterns due to empty-space overhead and complex optimization techniques.
Innovation Solution
The implementation of a dictionary table to store unique variable character field values, with reference keys pointing to the original data table, and a reference store column to replace the original column, utilizing a hash table for efficient data access, allowing for fixed-width storage and compression to minimize storage needs and speed up search access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If variable-length data types are stored directly in column store database, then data representation flexibility is improved, but storage efficiency deteriorates due to empty-space overhead
Solution Approach 1:
The patent creates a dictionary table that stores unique variable-length values once, and copies references to these values in the main data table. This allows the system to maintain flexible variable-length data representation while storing only fixed-length references in the column store, eliminating empty-space overhead.
Solution Approach 2:
The patent introduces a dictionary table as an intermediary between the variable-length data and the column store. The dictionary table maps variable-length values to fixed-length references, which then point to the actual data in the dictionary. This intermediary layer enables efficient storage while preserving data flexibility.
2Ease of operation
If complex optimization techniques are used for variable-length data, then data access capability is improved, but system complexity deteriorates
Solution Approach 1:
The patent performs preliminary action by pre-processing variable-length data into a dictionary of unique values with fixed-length references before storage. This pre-processing step creates a standardized structure that simplifies subsequent data access operations, eliminating the need for complex runtime optimization techniques.
Solution Approach 2:
The patent changes the parameter representation from variable-length to fixed-length by storing references instead of actual variable-length values in the column store. This parameter transformation maintains data access capability while simplifying the storage structure and reducing system complexity.
3Quantity of substance
If fixed-width storage is implemented for variable character fields, then storage efficiency is improved, but data type fidelity deteriorates
Solution Approach 1:
The patent copies variable-length values into a dictionary table where they are stored with their original formatting and characteristics preserved. The main data table then stores only fixed-length references to these copied values, achieving both fixed-width storage efficiency and data type fidelity.
Solution Approach 2:
The patent separates the data into two dimensions: the dictionary table stores the actual variable-length values with full fidelity, while the column store stores fixed-length references. This dimensional separation allows the system to achieve fixed-width storage benefits without sacrificing data type fidelity.
Data Source
AI summary
Data structures can provide for a column store of a database table. Methods can use the data structures for efficiently responding to a query. Unique field values of a column of a database table can be identified. The unique values can be stored in a dictionary table along with reference keys that point to a row of the database table. A reference store column can replace the original column, where the reference store column stores index values of the dictionary table. A hash table can be used in accessing the database. A hash function can provide a hash value of a query term, and the hash value can be used to access a hash table to obtain a stored value of an index value of the dictionary table. The index value can be used to access the dictionary table to obtain reference keys corresponding to rows of the database table.