Quantile Estimation Algorithm for Fraud Detection Memory Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing networked computing systems require significant resources for data storage and memory to calculate median, quantile, and MAD values, as they need to store and process entire datasets, which is inefficient for large data sets like transaction data in fraud detection systems.
Innovation Solution
A method that estimates quantile values using a processing engine that applies an estimation algorithm and natural logarithm transformation, allowing for reduced data storage, memory consumption, and CPU utilization by using predetermined marker quantiles and merging operations, thereby processing data without storing the entire dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the entire dataset is stored and processed to calculate median, quantile, and MAD values, then measurement precision is improved, but device complexity and memory consumption increase
Solution Approach 1:
The patent extracts only the necessary quantile information from the complete dataset by implementing an estimation algorithm that processes data in a streamlined manner. Instead of storing and processing the entire dataset, the system extracts quantile estimates through a simplified process that uses less memory and computational resources while maintaining acceptable accuracy for fraud detection purposes.
Solution Approach 2:
The patent changes the parameter of data representation by using estimation algorithms that compute quantile values without requiring the complete dataset to be loaded into memory. The system transforms the approach from exact calculation requiring full data storage to estimation that achieves sufficient precision with reduced data retention, thereby lowering device complexity and memory requirements.
2Measurement precision
If the entire dataset is stored and processed to calculate median, quantile, and MAD values, then measurement precision is improved, but processing time increases
Solution Approach 1:
The patent extracts quantile estimates from data using an efficient estimation algorithm that does not require loading the complete dataset into memory for processing. This extraction approach significantly reduces processing time while maintaining sufficient accuracy for fraud detection, as the system only needs to compute representative quantile values rather than process every data point.
Solution Approach 2:
The patent applies partial action by using estimation algorithms that compute quantile values with sufficient accuracy for practical purposes without performing the complete exact calculation on the entire dataset. This partial computation approach reduces processing time while delivering results that are adequate for fraud detection decision-making.
3Measurement precision
If the entire dataset is stored to calculate quantile values, then measurement precision is improved, but loss of substance increases
Solution Approach 1:
The patent extracts quantile estimates using an algorithm that processes data with minimal memory footprint. Instead of retaining the entire dataset in memory, the system extracts the necessary quantile information through efficient computation that uses a fraction of the memory resources, thereby reducing memory consumption while maintaining acceptable measurement precision.
Solution Approach 2:
The patent changes the memory usage parameter by implementing estimation algorithms that require significantly less memory than exact quantile calculation methods. The system transforms the computational approach from one that demands full dataset retention to estimation techniques that achieve sufficient accuracy with minimal memory consumption.
Data Source
AI summary
A system is provided for estimating quantile values for fraud assessments. The system includes a processor and a computer readable medium operably coupled thereto, to perform operations which include capturing one or more first data values for a quantile value profile associated with an entity, wherein the quantile value profile includes one of real values or a first plurality of quantile marker values calculated from the real values, accessing the quantile value profile for the entity, determining a first number of the one or more first data values, and based on the first number of the one or more first data values and the one of the real values or the first plurality of quantile marker values in the quantile value profile, performing one of a first merge operation, a second merge operation, or a third merge operation.