Cycle-based Continuous Query Model for Unbounded Data Streams

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current Data Stream Management Systems (DSMS) lack the functionality of structured query language (SQL) and Database Management Systems, particularly when dealing with unbounded and infinite streaming data, leading to higher overhead and inefficiencies in data processing and analysis.

Innovation Solution

The implementation of a Cycle-based Continuous Query (CCQ) model that enables SQL queries to be executed cycle-by-cycle on data streams in a chunk-by-chunk manner, leveraging the PostgreSQL engine and user-defined functions to buffer data across query execution cycles, allowing for continuous analytics and efficient handling of multiple streams.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Speed

If current generation DSMS processes streaming data independently of DBMS, then real-time data processing capability is improved, but system overhead increases and SQL functionality is lost

Engineering Contradiction:

Improvereal-time data processing capabilityVSAvoidsystem overhead

Core Design Contradiction:

SpeedVSDevice complexity

Solution Approach 1:

The patent merges DSMS and DBMS into a unified system where the query engine can simultaneously handle both traditional database operations and continuous stream processing. This integration eliminates the overhead of separate systems while maintaining real-time processing capabilities through the CCQ model that executes SQL queries on streaming data within the unified database engine.

Inventive Principle:

Principle #5Merging (Combining)

2Adaptability or versatility

If SQL queries are executed on unbounded streaming data, then continuous analytics capability is improved, but query definability becomes problematic

Engineering Contradiction:

Improvecontinuous analytics capabilityVSAvoidquery definability

Core Design Contradiction:

Adaptability or versatilityVSDifficulty of detecting and measuring

Solution Approach 1:

The patent segments unbounded streaming data into bounded data windows or batches that can be processed by traditional SQL queries. The CCQ model divides continuous streams into manageable chunks, allowing SQL's bounded query semantics to apply to each segment while maintaining continuous processing capability across all segments.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces dynamic data windows that can slide, expand, or contract based on query requirements. These dynamic windows allow SQL queries to operate on varying portions of the unbounded stream, making query definability flexible while maintaining continuous analytics capability through adaptive window management.

Inventive Principle:

Principle #15Dynamics

3Adaptability or versatility

If data streams are processed in a non-dataflow fashion with one-time SQL queries, then database technology leverage is improved, but processing efficiency for continuous streams decreases

Engineering Contradiction:

Improvedatabase technology leverageVSAvoidprocessing efficiency for continuous streams

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent implements continuous query execution where SQL queries run continuously on incoming data streams rather than executing one-time queries on buffered data. The CCQ model maintains persistent query instances that process data as it arrives, ensuring continuous useful action while leveraging database technology through the unified engine architecture.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS8880493B2Multi-streams analytics

Publication Date: 2014.11.04 MICRO FOCUS LLC

AI summary

Multi-stream analytics is disclosed. An example method of multi-stream analytics with a query engine includes punctuating unbounded streaming data into data chunks, each of the data chunks representing a bounded data set in the unbounded streaming data. The method also includes processing one of the data chunks. The method also includes rewinding a query instance for processing another of the data chunks.