Column Store Index for Fast Analytics on Dynamic Schema Data

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing database systems, particularly MongoDB, face challenges in efficiently handling analytical queries due to their unstructured schema, leading to sub-par performance and the need for workarounds like change streams or ETL tools to replicate data for analytical processing, which complicates application architecture and increases latency.

Innovation Solution

Implementing a column store index that extracts and sorts keys from documents in the database, allowing for faster analytical queries by storing data in a column-oriented format, which supports efficient compression and query execution without requiring data remodeling.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If unstructured schema databases are used to provide flexibility and adaptability, then adaptability and compatibility are improved, but analytical query performance deteriorates

Engineering Contradiction:

ImproveadaptabilityVSAvoidanalytical query performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent segments data storage into two distinct formats: row-oriented storage for maintaining dynamic schema flexibility and column-oriented storage for optimizing analytical query performance. This segmentation allows the system to simultaneously support both adaptability and analytical performance by directing different types of operations to appropriate storage formats.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a column store index as an intermediary structure that bridges the gap between unstructured schema databases and analytical query requirements. This intermediary maintains the flexibility of dynamic schemas while providing the structured access patterns needed for efficient analytical processing, eliminating the need for data replication to separate analytical systems.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Productivity

If workarounds like change streams or ETL tools are used to replicate data for analytical processing, then analytical query performance is improved, but application architecture complexity and latency increase

Engineering Contradiction:

Improveanalytical query performanceVSAvoidapplication architecture complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent merges analytical processing capabilities directly into the primary database system by implementing column store indices within the same database engine. This consolidation eliminates the need for separate analytical systems, change streams, or ETL tools, thereby reducing architectural complexity and data latency while maintaining high analytical query performance.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent enables the database system to serve both transactional and analytical workloads simultaneously through its dual-format storage architecture. The system performs analytical processing on fresh data directly without requiring external tools or replicated data, achieving self-sufficiency and eliminating additional infrastructure complexity.

Inventive Principle:

Principle #25Self-service

3Productivity

If workarounds like change streams or ETL tools are used to replicate data for analytical processing, then analytical query performance is improved, but data latency increases

Engineering Contradiction:

Improveanalytical query performanceVSAvoiddata latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent performs preliminary organization of data into column-oriented structures as data is ingested, maintaining sorted and compressed column store indices alongside the primary row-oriented storage. This preliminary structuring enables immediate analytical queries on fresh data without requiring subsequent replication or transformation steps, thereby eliminating data latency.

Inventive Principle:

Principle #10Preliminary action

4Productivity

If column store index is implemented to improve analytical query performance, then query speed is improved, but storage requirements increase

Engineering Contradiction:

Improvequery speedVSAvoidstorage requirements

Core Design Contradiction:

ProductivityVSVolume of stationary object

Solution Approach 1:

The patent changes the storage parameters by implementing column-oriented storage with compression techniques, which achieves both high query speed and efficient space utilization. By organizing data vertically and applying compression algorithms, the system reduces the storage overhead of column store indices while maintaining fast analytical query performance.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12536145B2Systems and methods for column store indices

Publication Date: 2026.01.27 MONGODB INC

AI summary

A database system comprising a database having a dynamic schema and at least one processor configured to: generate at least one column store index associated with the database, at least by: extracting keys from at least one document in the database, and sorting the keys; store the at least one column store index at least by storing the keys in the database; and access the at least one column store index based on at least one access request.