Common Dictionary Module for Column Store Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In column-store databases, redundancy in dictionaries across multiple tables leads to increased memory footprint and processing overhead due to duplicate data, as each table maintains its own dictionary despite sharing the same information.
Innovation Solution
Implementing a common dictionary module that allows multiple tables to share a single dictionary, reducing memory usage and processing overhead by defining a common dictionary type that can be used across columns, with updates managed in a single location.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If each table maintains its own dictionary, then data access is straightforward and simple, but memory footprint increases due to redundant storage of the same information across multiple dictionaries
Solution Approach 1:
The patent merges multiple separate dictionaries that contain duplicate information into a single shared dictionary. Instead of having separate dictionaries for each table column, the system creates one common dictionary that multiple columns can reference, thereby eliminating redundant storage while maintaining straightforward data access through the unified dictionary structure.
Solution Approach 2:
The patent implements a universal dictionary that serves multiple functions and multiple table columns simultaneously. This single dictionary structure is designed to be shared across different tables and columns, allowing the same dictionary to fulfill the role of multiple individual dictionaries and reduce overall memory consumption without compromising data access efficiency.
2Adaptability or versatility
If multiple dictionaries store the same information, then each table has independent data management, but processing time increases due to redundant operations on duplicate data
Solution Approach 1:
The patent combines multiple independent dictionary operations into a single unified dictionary management system. By merging the dictionaries, the system eliminates redundant processing operations that would otherwise be performed on duplicate data across multiple dictionaries, thereby reducing processing time while maintaining the ability to manage data independently through the shared structure.
Solution Approach 2:
The patent extracts the redundant duplicate information from multiple separate dictionaries and consolidates it into a single dictionary instance. This extraction of common data elements allows the system to perform processing operations once on the shared dictionary rather than repeatedly on multiple copies, significantly reducing processing time while preserving data management flexibility.
3Productivity
If dictionaries are updated in multiple locations, then each table can be updated independently, but errors increase due to potential inconsistencies across duplicate dictionaries
Solution Approach 1:
The patent merges multiple dictionary update operations into a single unified update mechanism. By consolidating the dictionaries into one shared structure, the system ensures that updates are performed in a single location, eliminating the risk of inconsistencies that arise from updating multiple separate dictionaries independently. This maintains data consistency while preserving update independence through the shared dictionary interface.
Solution Approach 2:
The patent implements a feedback mechanism in the shared dictionary system where updates are centrally managed and automatically reflected across all referencing columns. The unified dictionary structure provides inherent feedback that ensures all parts of the system see the same updated data, preventing inconsistencies and errors that would occur with independent updates to multiple separate dictionaries.
Data Source
AI summary
A method and system including a common dictionary module; a common dictionary processor in communication with the common dictionary module and operative to execute processor-executable process steps to cause the system to: define a common dictionary type; assign one or more characteristics to the common dictionary type; receive one or more data values; compress the dictionary type; and generate a common dictionary. Numerous other aspects are provided.