Percentile Determination via Data List Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large volumes of data generated by network systems pose challenges in real-time data analysis due to the impracticality of precomputing analysis for all possible subsets, necessitating efficient execution and critical real-time capabilities for corrective actions.
Innovation Solution
A system for determining percentiles efficiently processes data by reducing dataset size through data list mapping, grouping similar values, and using the empirical distribution function to compute percentiles, allowing for real-time analysis and anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data analysis is precomputed for all possible subsets, then measurement precision is improved, but device complexity and loss of time worsen due to the impracticality of processing all subsets
Solution Approach 1:
The patent segments the large dataset into multiple smaller partitions or blocks that can be processed independently. Each partition is analyzed separately to compute percentile values, which are then aggregated to produce the final result. This segmentation reduces the computational complexity from processing all data at once to processing manageable subsets while maintaining statistical accuracy.
Solution Approach 2:
The patent performs preliminary actions by pre-computing and storing percentile values for individual data partitions before the actual query is executed. When a percentile query is received, the system retrieves pre-computed values from storage and combines them, avoiding the need to reprocess all raw data. This preliminary computation reduces query response time while maintaining accuracy.
2Productivity
If data analysis is performed on very large datasets in real-time, then productivity is improved, but measurement precision worsens due to computational limitations
Solution Approach 1:
The patent introduces a new dimension by organizing data in a hierarchical structure with multiple levels of aggregation. Instead of computing percentiles directly from the entire dataset, the system computes percentiles at intermediate aggregation levels (e.g., partition level, then combined level). This dimensional approach allows real-time processing while maintaining statistical precision through multi-level aggregation.
Solution Approach 2:
The patent uses pre-computed percentile values from individual partitions as intermediary results. These intermediary percentile values serve as mediators that are combined to produce the final percentile value for the entire dataset. This intermediary approach enables real-time computation while preserving accuracy through proper aggregation of intermediate results.
3Adaptability or versatility
If all possible data subsets are precomputed, then adaptability is improved, but loss of time and device complexity worsen
Solution Approach 1:
The patent segments the data into reusable partitions that can be independently analyzed for different subsets. When a new data subset query is received, the system identifies relevant pre-computed partitions and combines them, avoiding reprocessing of unrelated data. This segmentation provides adaptability for various subset queries while reducing computation time through selective aggregation.
Data Source
AI summary
A system for determining percentiles includes an interface and a processor. The interface is configured to receive a request to determine a percentile value. The processor is configured to determine relevant data based at least in part on the request, determine an ordered data list map, determine the percentile value based at least in part on the ordered data list map, and provide the percentile value.