Probabilistic Feature Engineering for Financial Anomaly Detection

Solution Overview

Problem

Financial institutions face challenges in accurately detecting fraud and financial crimes due to the rarity of fraud data, which limits the effectiveness of machine learning models, and current anomaly detection approaches rely on generic feature engineering that does not prioritize anomalous features.

Innovation Solution

A computerized method and system for generating a representative dataset for machine learning models using a Representative Dataset Generation module that processes financial transaction records to create probabilistic categorical and numerical features, combined into complex features, for enhanced anomaly detection from the initiation stage, employing techniques like data validation, fraud tagging, and feature engineering with inverse probability ratio and Gaussian probability density functions.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If generic feature engineering is used for anomaly detection, then the system can operate from initiation stage, but the detection accuracy is low due to inability to prioritize anomalous features

Engineering Contradiction:

Improvedetection accuracyVSAvoidtime to accumulate fraud data

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary feature engineering by generating probabilistic features (categorical and numerical) from available non-fraud data before actual fraud cases accumulate. This preliminary action creates a head start for the ML model, enabling it to begin detecting anomalies accurately from the initiation stage rather than waiting for fraud data to accumulate.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system transforms ordinary transaction features into probabilistic features by applying probability distributions (Gaussian for numerical, inverse probability ratio for categorical). This parameter transformation enables the features to inherently prioritize anomalous patterns, allowing the model to detect fraud accurately even without historical fraud labels.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If machine learning models wait for sufficient fraud data before deployment, then detection accuracy improves, but the model cannot detect fraud events in the early stages

Engineering Contradiction:

Improvedetection accuracyVSAvoidearly stage fraud detection capability

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system performs preliminary feature engineering by generating probabilistic features (categorical and numerical) from available non-fraud data before actual fraud cases accumulate. This preliminary action creates a head start for the ML model, enabling it to begin detecting anomalies accurately from the initiation stage rather than waiting for fraud data to accumulate.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The probabilistic feature generation process acts as an intermediary that bridges the gap between abundant non-fraud data and the need to detect rare fraud events. By transforming features through probability distributions, this intermediary enables the model to learn anomaly patterns from normal data alone, eliminating the need to wait for fraud labeled data.

Inventive Principle:

Principle #24Intermediary (Mediator)

3Reliability

If more fraud data is collected for model training, then the model can learn fraud patterns, but the complexity of data collection and processing increases

Engineering Contradiction:

Improvefraud pattern recognitionVSAvoiddata collection and processing complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The probabilistic feature generation process acts as an intermediary that bridges the gap between abundant non-fraud data and the need to detect rare fraud events. By transforming features through probability distributions, this intermediary enables the model to learn anomaly patterns from normal data alone, eliminating the need to wait for fraud labeled data.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The system enables the ML model to learn fraud detection patterns autonomously from non-fraud data through probabilistic feature engineering. The model self-trains by identifying anomalies in the probabilistic feature space without requiring external fraud labeled data, reducing the complexity of data collection and processing.

Inventive Principle:

Principle #25Self-service

Data Source

PatentUS12045840B2Probabilistic feature engineering technique for anomaly detection

Publication Date: 2024.07.23 ACTIMIZE LIMITED

AI summary

A computerized-method for generating a dataset for a Machine Learning (ML) model for an increased accurate financial crime detection from an initiation stage of the ML model implementation. The computerized-method includes retrieval of financial transaction records from a database of financial transaction records to arrange a dataset of financial transaction records, according to preconfigured techniques. Then, processing the records in the dataset; Then, operating feature engineering on preselected anomalous related features to yield probabilistic categorical features and to yield probabilistic numerical features, and then combining the probabilistic categorical features with the probabilistic numerical features to generate a complex features dataset, and providing the probabilistic categorical features, the probabilistic numerical features and the complex features dataset to an ML model, thus, increasing accuracy of detection that is performed right from an initiation stage of the ML model implementation.