Clustering-Based Subsampling for Fraud Detection Training Data

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Traditional fraud detection systems, especially those using machine-learning models, face challenges due to the rarity of fraudulent transactions, leading to biased models that often incorrectly classify fraudulent transactions as genuine, and subsampling techniques can result in loss of representative instances and redundancy.

Innovation Solution

A clustering-based subsampling procedure is employed to select a subset of training transactions, where a higher proportion of transactions from clusters containing fraudulent transactions are retained, and a lower proportion from clusters without fraudulent transactions, to create a reduced training set that maintains diverse and discriminative information.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If random subsampling is used to reduce training data size, then training speed is improved, but representative information is lost

Engineering Contradiction:
Improvetraining speedVSAvoidrepresentative information
Core Design Contradiction:
ProductivityVSLoss of information

Solution Approach 1:

The patent applies local quality by treating different regions of the training data differently based on their fraud content. Clusters containing fraudulent transactions are identified and preserved with higher fidelity, while clusters without fraud undergo more aggressive subsampling. This localized differential treatment ensures that representative fraudulent information is retained while still achieving training speed improvement through subsampling of the majority genuine transactions.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent changes the subsampling parameter (sampling rate) based on the fraud content of each cluster. Instead of applying a uniform subsampling rate to all data, the method dynamically adjusts the sampling parameter: clusters with fraud transactions use a higher sampling rate to preserve representative instances, while clusters without fraud use a lower sampling rate to reduce data size. This parameter adaptation resolves the contradiction between speed and information retention.

Inventive Principle:
Principle #35Parameter changes

2Quantity of substance

If uniform subsampling is applied to all clusters, then training data size is reduced, but fraud detection accuracy deteriorates

Engineering Contradiction:
Improvetraining data sizeVSAvoidfraud detection accuracy
Core Design Contradiction:
Quantity of substanceVSMeasurement precision

Solution Approach 1:

The patent implements local quality by applying different subsampling strategies to different clusters based on their fraud characteristics. Clusters containing fraudulent transactions are identified through clustering analysis, and these specific regions are preserved with higher sampling rates to maintain fraud detection accuracy. Meanwhile, clusters consisting only of genuine transactions are more aggressively subsampled to reduce overall data size. This localized differential subsampling resolves the contradiction between reducing data quantity and maintaining detection precision.

Inventive Principle:
Principle #3Local quality

Solution Approach 2:

The patent segments the training data into multiple clusters based on transaction characteristics and fraud content. By dividing the homogeneous training set into heterogeneous clusters, the method can apply targeted subsampling to each segment. This segmentation allows the system to reduce overall data size while preserving the critical fraud-containing segments, thereby maintaining detection accuracy despite reduced data quantity.

Inventive Principle:
Principle #1Segmentation

3Measurement precision

If all training transactions are retained, then classification accuracy is maintained, but training time increases

Engineering Contradiction:
Improveclassification accuracyVSAvoidtraining time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The patent extracts and separates the essential fraudulent information from the complete training set through clustering analysis. By identifying clusters that contain fraudulent transactions and extracting these critical segments, the method preserves the core information needed for accurate classification. The remaining non-essential genuine transactions are then subsampled or removed, significantly reducing training time while maintaining classification accuracy through the preserved fraud-containing clusters.

Inventive Principle:
Principle #2Taking out (Extraction)

Solution Approach 2:

The patent performs preliminary clustering analysis before the actual training process to identify and separate fraud-containing clusters. This preliminary action allows the system to pre-determine which data segments are critical for accuracy and which can be reduced. By performing this classification and segmentation beforehand, the method enables efficient training on a reduced dataset while ensuring that the essential fraud patterns are already identified and preserved for accurate classification.

Inventive Principle:
Principle #10Preliminary action

Data Source

PatentUS11954685B2Method, apparatus and computer program for selecting a subset of training transactions from a plurality of training transactions
Publication Date: 2024.04.09 SONY GROUP CORP
  • US11954685B2 patent drawing
  • US11954685B2 patent drawing
  • US11954685B2 patent drawing

AI summary

Training a machine learning model includes selecting a subset of training transactions from a plurality of training transactions to be used for classifying transactions as either fraudulent or genuine, and classifying transactions as either fraudulent or genuine. Additionally, selecting the subset of training transactions includes clustering the plurality of training transactions into a plurality of clusters based on a similarity measure. Each training transaction includes an indication whether the training transaction is fraudulent or genuine. Further, selecting the subset of training transactions includes selecting a first proportion of the training transactions of a cluster of training transactions for the subset of training transactions if the cluster comprises at least one fraudulent training transaction, and selecting a second proportion of the training transactions of a cluster of training transactions for the subset of training transactions if the cluster is free of fraudulent training transactions.