Local Extrema Data Sampling for Big Data Visualization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing volume of big data poses challenges in identifying suitable visualization techniques for effective data analysis, as existing methods struggle to efficiently sample and render large datasets for meaningful evaluation.
Innovation Solution
A data point selection method that samples data points by defining dimensions, sorting, and comparing values to retain peak or trough points separated by a minimum distance, and the first and last points within partitions, thereby reducing dataset size while preserving key data trends.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data sampling methods are used to reduce dataset size, then data rendering times and computational efficiency are improved, but the accuracy of data analysis and visualization may deteriorate
Solution Approach 1:
The patent segments the data sampling process into multiple stages: first dividing the sorted data into partitions based on the first dimension, then selectively sampling from each partition. This segmentation allows the system to process large datasets in manageable chunks while preserving key trends through systematic selection of representative data points from each partition.
Solution Approach 2:
The patent applies local quality by selecting data points based on their local characteristics within each partition. Specifically, it selects the first and last data points of each partition, as well as data points that are local extrema (peaks or troughs) within their respective partitions. This ensures that each selected data point represents the local structure of its partition, maintaining overall data accuracy while reducing total sample size.
2Measurement precision
If more data points are sampled to maintain key trends, then data analysis accuracy is improved, but computational complexity and processing time increase
Solution Approach 1:
The patent performs preliminary actions by first sorting the entire dataset by the first dimension before applying the sampling algorithm. This pre-sorting operation establishes a structured arrangement that enables subsequent partitioning and selective sampling to efficiently identify representative data points. The preliminary sorting reduces the computational complexity of the sampling process itself, as data points are already organized in a way that makes partition boundaries and extrema identification straightforward.
3Productivity
If data is sorted and partitioned before sampling, then sampling efficiency is improved, but initial processing time and memory requirements increase
Solution Approach 1:
The patent segments the sampling process into distinct phases: sorting phase, partitioning phase, and selection phase. By dividing the overall task into these segments, the system can optimize each phase independently. The sorting and partitioning operations, while requiring initial time and memory resources, enable highly efficient sampling in the final phase, reducing overall processing time for large datasets.
Data Source
AI summary
A computing device sorts a plurality of data points in a first dimension. A first data point has a first value, a second data point has a second value, and a third data point has a third value defined in a second dimension. (a) The second value is compared to the first and third values. (b) When the second value is less than the first value and greater than the third value, or the second value is greater than the first value and less than the third value, the data point is deleted. (c) The first data point is defined as the second data point. (d) The second data point is defined as the third data point. (e) The third data point is defined as a next data point. (a)-(e) are repeated until each of the plurality of data points is defined as the third data point to define a plurality of sampled data points as remaining data points of the plurality of data points.