Clustering-Based Subsampling for Fraud Detection Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional fraud detection systems, especially those using machine-learning models, face challenges due to the rarity of fraudulent transactions, leading to biased models that often incorrectly classify fraudulent transactions as genuine, and subsampling techniques can result in loss of representative instances and redundancy.
Innovation Solution
A clustering-based subsampling procedure is employed to select a subset of training transactions, where a higher proportion of transactions from clusters containing fraudulent transactions are retained, and a lower proportion from clusters without fraudulent transactions, to create a reduced training set that maintains diverse and discriminative information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If random subsampling is used to reduce training data size, then training speed is improved, but representative information is lost
Solution Approach 1:
The patent applies local quality by treating different regions of the training data differently based on their fraud content. Clusters containing fraudulent transactions are identified and preserved with higher fidelity, while clusters without fraud undergo more aggressive subsampling. This localized differential treatment ensures that representative fraudulent information is retained while still achieving training speed improvement through subsampling of the majority genuine transactions.
Solution Approach 2:
The patent changes the subsampling parameter (sampling rate) based on the fraud content of each cluster. Instead of applying a uniform subsampling rate to all data, the method dynamically adjusts the sampling parameter: clusters with fraud transactions use a higher sampling rate to preserve representative instances, while clusters without fraud use a lower sampling rate to reduce data size. This parameter adaptation resolves the contradiction between speed and information retention.
2Quantity of substance
If uniform subsampling is applied to all clusters, then training data size is reduced, but fraud detection accuracy deteriorates
Solution Approach 1:
The patent implements local quality by applying different subsampling strategies to different clusters based on their fraud characteristics. Clusters containing fraudulent transactions are identified through clustering analysis, and these specific regions are preserved with higher sampling rates to maintain fraud detection accuracy. Meanwhile, clusters consisting only of genuine transactions are more aggressively subsampled to reduce overall data size. This localized differential subsampling resolves the contradiction between reducing data quantity and maintaining detection precision.
Solution Approach 2:
The patent segments the training data into multiple clusters based on transaction characteristics and fraud content. By dividing the homogeneous training set into heterogeneous clusters, the method can apply targeted subsampling to each segment. This segmentation allows the system to reduce overall data size while preserving the critical fraud-containing segments, thereby maintaining detection accuracy despite reduced data quantity.
3Measurement precision
If all training transactions are retained, then classification accuracy is maintained, but training time increases
Solution Approach 1:
The patent extracts and separates the essential fraudulent information from the complete training set through clustering analysis. By identifying clusters that contain fraudulent transactions and extracting these critical segments, the method preserves the core information needed for accurate classification. The remaining non-essential genuine transactions are then subsampled or removed, significantly reducing training time while maintaining classification accuracy through the preserved fraud-containing clusters.
Solution Approach 2:
The patent performs preliminary clustering analysis before the actual training process to identify and separate fraud-containing clusters. This preliminary action allows the system to pre-determine which data segments are critical for accuracy and which can be reduced. By performing this classification and segmentation beforehand, the method enables efficient training on a reduced dataset while ensuring that the essential fraud patterns are already identified and preserved for accurate classification.
Data Source
AI summary
Training a machine learning model includes selecting a subset of training transactions from a plurality of training transactions to be used for classifying transactions as either fraudulent or genuine, and classifying transactions as either fraudulent or genuine. Additionally, selecting the subset of training transactions includes clustering the plurality of training transactions into a plurality of clusters based on a similarity measure. Each training transaction includes an indication whether the training transaction is fraudulent or genuine. Further, selecting the subset of training transactions includes selecting a first proportion of the training transactions of a cluster of training transactions for the subset of training transactions if the cluster comprises at least one fraudulent training transaction, and selecting a second proportion of the training transactions of a cluster of training transactions for the subset of training transactions if the cluster is free of fraudulent training transactions.


