Variable-Size Sampling Method for Data Streaming Uniformity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The K-sample method for data-streaming environments faces issues with uniformity confidence due to memory loss and restrictive sample range, leading to poor sampling performance and continuous reduction in uniformity confidence.
Innovation Solution
A variable-size sampling method that calculates a maximum window size to maintain a predetermined uniformity confidence level, dynamically adjusts sample size based on the sampling fraction, and allows for slot generation and replacement to ensure high uniformity confidence.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the K-sample method dynamically increases sample size to maintain sampling fraction, then sampling coverage is improved, but uniformity confidence continuously degrades due to memory loss
Solution Approach 1:
The patent implements dynamic window size adjustment based on uniformity confidence calculations. The window size is not fixed but adapts to maintain a predetermined uniformity confidence level, allowing the system to dynamically balance between sampling coverage and uniformity confidence preservation.
Solution Approach 2:
The patent changes the parameter of window size to optimize the trade-off between sampling fraction maintenance and uniformity confidence. By calculating maximum window size that satisfies uniformity confidence requirements, the system adjusts this parameter to prevent confidence degradation while maintaining adequate sampling coverage.
2Productivity
If slot generation is performed to increase sample size, then sampling fraction is maintained, but memory loss increases leading to uniformity confidence reduction
Solution Approach 1:
The patent performs preliminary calculation of maximum window size before executing sampling operations. This preliminary action determines the optimal window size that maintains uniformity confidence, preventing excessive memory loss before it occurs during the sampling process.
Solution Approach 2:
The patent implements a feedback mechanism where uniformity confidence is continuously monitored and used to adjust window size. The calculated uniformity confidence feeds back into the system to determine appropriate window size adjustments, creating a closed-loop control that prevents memory loss from degrading confidence.
3Quantity of substance
If fixed-size sampling is used, then memory usage is controlled, but sampling performance degrades under varying data stream conditions
Solution Approach 1:
The patent transitions from fixed-size sampling to dynamic window size sampling. The window size changes based on calculated maximum values that maintain uniformity confidence, allowing the system to adapt to varying data stream conditions while controlling memory usage through principled size determination.
Data Source
AI summary
Disclosed is a variable-size sampling method under a data-streaming environment, including: calculating a maximum window size that satisfies a lower limitation of a predetermined uniformity confidence level at all times; inputting a data stream to be sampled; comparing a data stream length input until a current time point with the maximum window size; inspecting a sample size and a sampling fraction if the maximum window size is larger than the data stream length; performing sampling by generating a slot to increase the sample size if the current sample size is smaller than a predetermined percentage (P %) of the data stream; and directly performing sampling without generating a slot if the current sample size is equal to or larger than the predetermined percentage (P %) of the data stream. As a result, degradation of uniformity confidence during variable-size sampling under a real-time streaming environment can be prevented to improve sampling performance.