Database Activity Load Approximation via Exponential Smoothing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for analyzing data streams in databases face challenges with high storage and computation resource usage, especially when dealing with large scales, and require maintaining multiple statistical models for different time resolutions, which is costly and resource-intensive.
Innovation Solution
A system that uses a processor to generate an event stream from database server activity, converts it into a time series using exponential smoothing, and sends it to a streaming analytics engine for real-time analysis, allowing for efficient approximation of activity loads across multiple time windows without continuous recalculations, thus saving resources and enabling real-time anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If time window methods are used to aggregate user activities, then activity load statistics can be obtained, but high storage capacity and computation resources are consumed
Solution Approach 1:
The patent transforms the aggregation approach by changing the fundamental parameter from fixed time windows to exponential decay weighting. Instead of binning events into discrete time intervals, each event is weighted by exp(-λ(t-current_time)), where λ controls the decay rate. This parameter change enables continuous time resolution analysis while using constant memory regardless of time range, resolving the contradiction between measurement precision and resource consumption.
Solution Approach 2:
The patent creates a simplified computational model that copies the essential statistical properties of traditional time-window aggregation but implements it through exponential smoothing. This alternative representation maintains the ability to compute activity load statistics while using a fixed-size data structure (current smoothed value and time since last event) instead of storing entire time windows, thereby reducing storage and computation resources.
2Adaptability or versatility
If multiple statistical models per time-frame are maintained, then multiple time resolutions are available, but the cost is very expensive in time and disk space
Solution Approach 1:
The patent creates a universal exponential smoothing model that can adapt to multiple time resolutions through a single parameter λ (decay rate). By adjusting λ, the same model structure provides different effective time windows without requiring separate statistical models for each time frame. This multi-functional approach enables the system to analyze activity loads at various time resolutions (short-term, medium-term, long-term) using one unified model, dramatically reducing disk space and processing time compared to maintaining multiple independent models.
Solution Approach 2:
The patent introduces dynamic adaptability by allowing the smoothing parameter λ to be adjusted based on the desired time resolution. Instead of static models fixed to specific time frames, the exponential smoothing model dynamically adapts its effective window size through parameter adjustment. This dynamic characteristic enables flexible switching between different time resolutions without recreating models, saving both storage space and processing time while maintaining versatility.
3Reliability
If continuous recalculations are performed using windows of time, then updated activity load is obtained, but processing resources are consumed
Solution Approach 1:
The patent implements continuous updating of activity load statistics through exponential smoothing that processes events as they arrive without requiring periodic recalculations over fixed windows. The smoothed value is continuously updated using the formula: smoothed_value = smoothed_value + event_weight * (1 - exp(-λ * event_age)), where event_weight is 1 for new events and 0 for duplicate events. This continuous action provides reliable updated activity load with minimal processing resources, as each event requires only a constant-time calculation rather than recalculating over the entire time window.
Solution Approach 2:
The patent employs a computationally inexpensive updating mechanism that processes each event with a simple constant-time calculation. Instead of expensive periodic recalculations that reprocess all events in a time window, the exponential smoothing approach uses a lightweight update formula that requires only the current smoothed value, event timestamp, and decay parameter. This disposable, event-by-event updating approach maintains reliable activity load information while consuming minimal processing resources.
Data Source
AI summary
An example system includes a processor to monitor activity on a database server to generate an events stream. The processor can convert the events stream into a time series that approximates activity load at the database server using an exponential smoothing. The processor can also send the time series to a streaming analytics engine.