Timeseries Bucketing and Indexing for Document Database Processing

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing non-relational database formats, such as MongoDB, are not conducive for storing timeseries data efficiently, as they are based on documents rather than time, requiring conversion to SQL-like formats for processing.

Innovation Solution

A system and method for storing timeseries data in a non-relational database using a data structure defined by documents, organized in buckets indexed by B-trees, with geo-indexing and secondary indexes, and supporting functionalities like aggregation pipelines, sampling, and windowing operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If timeseries data is stored in traditional non-relational database formats (document-based), then the database can store unstructured data flexibly, but it is not conducive for efficient timeseries data storage and processing

Engineering Contradiction:

Improveflexibility in storing unstructured dataVSAvoidefficiency of timeseries data processing

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent segments timeseries data into discrete events with specific time values, organizing them in a structured format within the document-based database. Each event is divided into comparable components (time, metadata, values) that can be efficiently indexed and queried, resolving the contradiction between flexible storage and processing efficiency.

Inventive Principle:

Principle #1Segmentation

2Productivity

If timeseries data is converted to SQL-like formats for processing, then SQL-like functions can be performed efficiently, but the database format complexity increases and conversion overhead is introduced

Engineering Contradiction:

Improveefficiency of SQL-like function processingVSAvoiddatabase format complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent introduces an intermediary aggregation pipeline that operates directly on the document-based timeseries data structure. This pipeline provides SQL-like processing capabilities without requiring format conversion, acting as a mediator between the flexible document storage and structured query requirements, thereby maintaining storage simplicity while enabling efficient processing.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Adaptability or versatility

If traditional document-based storage is used, then storage flexibility is maintained, but indexing and query performance for timeseries data is suboptimal

Engineering Contradiction:

Improvestorage flexibilityVSAvoidindexing and query performance

Core Design Contradiction:

Adaptability or versatilityVSSpeed

Solution Approach 1:

The patent implements preliminary indexing strategies where time values and metadata are pre-processed and organized into efficient index structures during data ingestion. This preliminary organization enables fast retrieval and querying of timeseries data while maintaining the flexible document-based storage format, resolving the contradiction between storage flexibility and query performance.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20250217377A1Systems and method for processing timeseries data

Publication Date: 2025.07.03 MONGODB INC

AI summary

In some implementations, events measured at various points in time may be organized in a data structure that defines an event represented by a document. In particular, events can be organized in columns of documents referred to as buckets. These buckets may be indexed using B-trees by addressing metadata values or value ranges. Buckets may be defined by periods of time. Documents may also be geoindexed and stored in one or more locations in a distributed computer network. One or more secondary indexes may be created based on time and/or metadata values within documents.