Performance Percentile Determination Using Data List Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large volumes of data generated by network systems pose challenges in real-time data analysis due to the impracticality of precomputing analysis for all possible subsets, necessitating efficient execution and critical real-time capabilities for corrective actions.
Innovation Solution
A system for determining performance percentiles efficiently processes data by reducing dataset size through data list mapping, grouping similar values, and using the empirical distribution function to compute percentiles, allowing for real-time analysis and anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If precomputation is used for data analysis, then analysis time is reduced, but the system cannot handle all possible subsets and real-time requirements are not met
Solution Approach 1:
The patent segments the data analysis process into two phases: precomputation of statistical summaries (mean, variance, percentiles) for different data subsets, and real-time query processing using these precomputed statistics. This allows the system to precompute analysis for common subsets while maintaining the ability to handle diverse real-time queries efficiently without recomputing everything from scratch.
Solution Approach 2:
The system performs preliminary computation of statistical metrics for various data subsets and stores them in advance. When real-time queries are received, the system retrieves relevant precomputed statistics and combines them with current data points, eliminating the need to recompute entire distributions from scratch and thus meeting real-time requirements.
2Reliability
If real-time data analysis is performed on large datasets, then timely corrective actions can be taken, but computational efficiency decreases
Solution Approach 1:
The patent extracts key statistical characteristics (mean, variance, percentiles) from the full data distribution and stores them as precomputed summaries. During real-time analysis, the system uses only these extracted statistics rather than processing the entire large dataset, significantly reducing computational load while maintaining analytical accuracy for detecting anomalies and trends.
Solution Approach 2:
The system creates simplified representations (copies) of the data distribution in the form of precomputed statistical summaries. These summaries capture the essential characteristics of the data without requiring access to the full original dataset, enabling efficient real-time analysis by working with the compressed statistical information rather than the complete data.
3Productivity
If data is grouped and aggregated for analysis, then analysis speed increases, but data granularity and precision are reduced
Solution Approach 1:
The patent applies different levels of aggregation at different scopes: coarse-grained precomputed statistics for overall distribution characteristics, and fine-grained individual data point processing for specific queries. This allows the system to maintain high analysis speed through aggregation while preserving data granularity where needed by processing only the necessary subset of data points in detail.
Data Source
AI summary
A system for determining percentiles for a performance data includes an interface configured to receive a request to determine a percentile value for the performance data and a processor configured to determine relevant data based at least in part on the request, determine an ordered data list map, determine the percentile value based at least in part on the ordered data list map, and provide the percentile value for the performance data.