Summarized Data Storage for Streaming Analytics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Traditional data storage systems face challenges in managing high-volume streaming data from connected devices, leading to increased storage costs, latency, and bandwidth issues, as they require persistent data retention which is not necessary for analytics applications where data value diminishes over time.

Innovation Solution

A summarized data storage management system that constructs and stores a compact, summarized version of data streams, supporting data approximation and providing an approximate query interface, which reduces storage space, latency, and bandwidth consumption while maintaining essential statistical properties for analytics applications.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional data storage systems store all streaming data for persistence, then data retention is ensured, but storage costs and bandwidth consumption increase significantly

Engineering Contradiction:

Improvedata retentionVSAvoidstorage consumption

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential statistical properties (sum, count, minimum, maximum) from the raw streaming data and stores these aggregated values instead of the complete data set. This extraction principle reduces storage requirements while preserving the essential information needed for analytics applications.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent transforms the data representation by changing from storing raw data values to storing aggregated statistical parameters. This parameter transformation allows the system to maintain data utility for analytics while significantly reducing the quantity of stored information.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If traditional data storage systems retain all historical data, then complete data availability is maintained, but latency increases for analytics operations

Engineering Contradiction:

Improvedata availabilityVSAvoidlatency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent performs preliminary aggregation of data statistics as data streams in, maintaining pre-computed sums, counts, minima, and maxima in memory. When analytics queries are executed, the system can immediately retrieve these pre-computed values without scanning through historical raw data, thereby reducing latency while maintaining data availability.

Inventive Principle:

Principle #10Preliminary action

3Measurement precision

If traditional data storage systems store complete data streams, then full data fidelity is preserved, but bandwidth consumption increases

Engineering Contradiction:

Improvedata fidelityVSAvoidbandwidth consumption

Core Design Contradiction:

Measurement precisionVSLoss of energy

Solution Approach 1:

Inventive Principle:

Principle #2Taking out (Extraction)

4Adaptability or versatility

If traditional data storage systems maintain all raw data, then comprehensive analytics are possible, but device complexity and storage costs increase

Engineering Contradiction:

Improveanalytics capabilityVSAvoidstorage system complexity

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

Inventive Principle:

Principle #35Parameter changes

Solution Approach 2:

The patent uses lightweight, simple data structures for storing aggregated statistics that are much less complex than storing complete raw data sets. These simplified storage structures reduce device complexity while maintaining essential analytics capabilities.

Inventive Principle:

Principle #27Cheap short-living objects (Disposable)

Data Source

PatentEP3497593B1Summarized data storage management system for streaming data

Publication Date: 2021.08.18 SAMSUNG ELECTRONICS CO LTD

Figure 1
Figure 2
Figure 3

AI summary

One embodiment provides a method comprising storing a first set of explicitly enumerated data values in a first window data structure associated with a first time span. The method further comprises storing one or more sets of summary statistics in a second set of window data structures. Each window data structure of the second set of window data structures is associated with a time span older than the first time span. The one or more sets of summary statistics are based on a second set of data values. The method further comprises storing a third set of explicitly enumerated data values in a third set of window data structures, wherein each data value of the third set of explicitly enumerated data values is annotated.