Column Set Grouping for Sparse Database Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in efficiently managing and querying large numbers of columns, especially when dealing with sparse columns that have a substantial percentage of null values, leading to resource-intensive operations and complex schema management.
Innovation Solution
The introduction of a grouping component that aggregates atomic scalar values into column sets, allowing for a logical representation and compact storage, enabling efficient querying and manipulation of column groups as a single entity, similar to a scalar column, while maintaining access to individual columns.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large number of columns are selected and processed in traditional database systems, then comprehensive data retrieval is achieved, but resource consumption increases significantly
Solution Approach 1:
The patent merges multiple scalar columns into a single column set object that can be treated as one entity. This allows comprehensive data retrieval across multiple columns while reducing resource consumption by processing the column set as a single unit rather than individually processing each column.
Solution Approach 2:
The patent segments columns into logical groups called column sets that can be independently managed and processed. This segmentation allows the system to retrieve only the necessary column sets for a given query, reducing overall resource consumption while maintaining comprehensive retrieval capability when needed.
2Reliability
If sparse columns with substantial null values are managed individually, then data integrity is maintained, but storage and processing efficiency deteriorates
Solution Approach 1:
The patent combines multiple sparse columns into a single column set object that preserves data integrity while improving efficiency. The column set maintains the semantic meaning and integrity of individual columns while allowing bulk operations that significantly improve storage and processing efficiency for sparse data.
3Adaptability or versatility
If heterogeneous semi-structured data is stored in SQL Server tables with varying schemas, then data flexibility is achieved, but schema management complexity increases
Solution Approach 1:
The patent creates a universal column set structure that can accommodate heterogeneous semi-structured data with varying schemas. The column set object provides a unified interface for managing diverse columns, reducing schema management complexity while maintaining the flexibility to store and query heterogeneous data types.
4Difficulty of detecting and measuring
If columns are treated as individual entities in traditional databases, then precise data access is enabled, but operation efficiency deteriorates when dealing with column groups
Solution Approach 1:
The patent merges multiple columns into a column set object that can be accessed and operated upon as a single entity. This merging enables efficient bulk operations on column groups while maintaining the ability to access individual columns when precise data access is required, thus improving operation efficiency without sacrificing access precision.
Data Source
AI summary
Systems and methods that group atomic scalar values recognized by a database such as columns into sets (e.g., column sets). A grouping component associated with the SQL server creates a logical representation for column groupings, which are accessible by a single I/O and can be co-located (e.g., substantially close or compact) in terms of storage location. Interesting column sets (e.g., non-null) can also be selected for a data representation thereof as a single entity to other applications.