Clustering-Based Data Selection for Fraud Detection Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine Learning models face challenges in accurately detecting rare financial fraud events due to unbalanced representations of true and false events in training datasets, leading to high rates of false detection and resource-intensive manual verification.
Innovation Solution
A computerized method utilizing a risk-prediction-preparation module that clusters transactions into logical entities, selects data points for two datasets – one with fraudulent points and another with randomly selected non-fraudulent points, trains separate machine learning models, and combines them into an ensemble model for improved risk prediction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If sampling algorithms are used to generate training datasets from large-scale data, then the training dataset can be obtained for machine learning models, but large amount of computational resources such as memory and storage space are consumed
Solution Approach 1:
The patent segments the large-scale transaction data by grouping transactions into logical entities (e.g., customer accounts, merchant accounts) and then selects representative transactions from each entity. This segmentation approach avoids processing the entire dataset at once, reducing memory and storage requirements while maintaining model training quality.
2Reliability
If the training dataset includes all transactions to ensure complete representation, then the model can learn from all available data, but the computational complexity and resource consumption increase significantly
Solution Approach 1:
The patent applies local quality by selecting transactions based on their local characteristics within logical entities. Instead of uniform random sampling, it ensures that each logical entity contributes representative transactions that capture its specific patterns and behaviors, maintaining model accuracy while reducing overall data volume.
3Measurement precision
If clustering algorithms are used to improve model representation, then better models can be obtained from better representation of the population, but additional computational resources and time are required
Solution Approach 1:
The patent performs preliminary grouping of transactions into logical entities before the actual sampling process. This pre-organization allows for more efficient selection of representative transactions and reduces the computational burden during model training, as the data is already structured in a meaningful way that reflects population characteristics.
Data Source
AI summary
A risk-prediction-preparation module to generate a risk-prediction-model, is provided herein. The risk-prediction-preparation module includes accessing a data-storage of transactions to operate a group-by operation on transactions related to data-points, according to a logical-entity into entities. Then, clustering entities of a clean-financial dataset into clusters. Selecting data-points of: (a) entities from the clusters to a first dataset and (b) a preconfigured amount of entities randomly to a second dataset. Selecting all entities that have at least one ‘fraudulent’ data-points in at least one related data-point to add all the entities to the first dataset and the second dataset. Using vectorized and scaled extracted features for training a first machine-learning-model of fraud detection on the first dataset and training a second machine-learning-model of fraud detection on the second dataset to collect results. Using the results for combining the first machine-learning-model and the second machine-learning-model to an ensemble machine-learning-model for risk-prediction.