OLAP Star Schema for Asynchronous Test Data Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for collecting and correlating real-time data from network testing systems like Spirent TestCenter fail to provide seamless real-time and end-of-test results, as they lack effective synchronization and storage of data-plane and control-plane statistics, limiting post-test analysis and correlation of measurements.
Innovation Solution
Implementing an online analytical processing (OLAP) style star schema in a relational database to store both real-time and end-of-test results, using interval queries instead of timestamp-based correlations to aggregate data into intervals, and enabling correlation of test configurations, learned values, data-plane, and control-plane statistics for enhanced scalability and query efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If timestamp-based correlation is used to correlate real-time data from multiple sources, then data can be associated with specific time points, but the correlation is unreliable because measurement processes are asynchronous and timestamps rarely match exactly
Solution Approach 1:
The patent introduces an intermediary correlation mechanism that does not rely on direct timestamp matching. Instead, it uses a multi-step process involving data buffering, interval-based grouping, and statistical correlation methods. The system buffers data from asynchronous sources, groups data into time intervals, and then performs correlation analysis within those intervals, thereby mediating between the precise but mismatched timestamps and the need for reliable correlation.
2Ease of manufacture
If conventional relational database schemas are used to store test results, then data can be organized by configuration structure, but analytic queries require complicated join operations and are expensive to execute
Solution Approach 1:
The patent applies preliminary action by pre-computing and pre-organizing data into a columnar storage format optimized for analytical queries. Instead of organizing data for easy insertion (transactional optimization), the system transforms data into a format that enables efficient analytical processing. Materialized views and pre-aggregated statistics are created in advance, so that complex analytical queries can be executed efficiently without performing multiple expensive join operations at query time.
3Quantity of substance
If real-time data is displayed on user interface and then discarded, then memory usage is minimized, but the data cannot be queried or analyzed after the test
Solution Approach 1:
The patent applies local quality by implementing a hierarchical data retention strategy where different portions of the data are stored with different retention characteristics. Frequently accessed recent data is kept in high-performance storage with full detail, while historical data is archived in more capacity-efficient storage. The system also implements selective retention, keeping detailed data only for periods where analysis is most valuable, while summarizing or aggregating data for longer-term retention, thereby optimizing the balance between data availability and resource consumption.
Data Source
AI summary
The technology disclosed relates to a method for storing and time-correlating real-time and queryable test results of a test of a device under test (DUT). The method includes initiating the test applied to the DUT to collect real-time data from a multitude of data streams for multiple aspects of the DUT, the collected data including counters and fact-type values, the collected data having imperfectly synchronized time bases and the collected data being collected from different sources asynchronously at different times, specifying a recording time interval for recording the data collected among multiple databases, recording data according to the specified recording time interval, such that each piece of the recorded data is associated with a particular time interval, and at a conclusion of the test, correlating the recorded data with the test configuration data about a test state in the respective time intervals.


