Siamese Neural Network for Uplift Modeling Overfitting

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing uplift modeling techniques in digital marketing are prone to overfitting and inaccuracies, failing to effectively predict individual treatment effects and efficiently utilize data, particularly in determining the success of digital marketing interventions and e-commerce content presentation.

Innovation Solution

A Siamese neural network model is employed for individual treatment effect modeling, utilizing a twin neural network architecture that performs simultaneous simulations to predict the uplift assessment and re-trains based on gathered outcomes, optimizing predictions through a novel loss function and sparse model pruning techniques.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If tree-based methods are used for uplift modeling, then non-linear patterns are captured, but overfitting occurs and individual outcome predictions are inaccurate

Engineering Contradiction:

Improvenon-linear pattern captureVSAvoidprediction accuracy

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The model segments the prediction task into two separate neural networks (twin networks) that independently process treated and control group data. Each network learns distinct patterns from its respective group, and their outputs are combined to estimate individual treatment effects, thereby capturing non-linear patterns while reducing overfitting through separate learning processes

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Instead of directly modeling the treatment effect as done in traditional approaches, the patent inverts the approach by modeling the outcome distributions for treated and control groups separately, then deriving the treatment effect from the difference between these separate models. This indirect approach reduces overfitting while maintaining prediction accuracy

Inventive Principle:

Principle #13The other way round (Inversion)

2Manufacturing precision

If likelihood-based methods with manual manipulations are used, then regression optimization is achieved, but the process is complex and inefficient

Engineering Contradiction:

Improvemodel optimizationVSAvoidmanual manipulation complexity

Core Design Contradiction:

Manufacturing precisionVSDevice complexity

Solution Approach 1:

The patent replaces manual manipulation and complex regression optimization processes with an automated neural network-based system. The twin neural networks automatically learn optimal parameters through backpropagation and gradient descent, eliminating the need for manual regression equation modifications while achieving superior optimization

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The model performs self-service by automatically optimizing its own parameters through the neural network training process. The system independently adjusts weights and biases to minimize prediction errors without requiring manual intervention, thereby simplifying the optimization process while maintaining high precision

Inventive Principle:

Principle #25Self-service

3Loss of information

If traditional uplift models are used, then treatment effect estimation is provided, but real-time prediction and automated re-training capabilities are lacking

Engineering Contradiction:

Improvetreatment effect informationVSAvoidreal-time prediction automation

Core Design Contradiction:

Loss of informationVSExtent of automation

Solution Approach 1:

The patent implements feedback mechanisms where prediction outcomes are continuously fed back into the neural networks for re-training. This closed-loop system automatically updates the models with new data, enabling real-time adaptation and maintaining prediction accuracy without manual intervention, thereby achieving both comprehensive information retention and full automation

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS20230196406A1Siamese neural network model

Publication Date: 2023.06.22 THE TORONTO DOMINION BANK

AI summary

Methods, apparatus and techniques are provided for a twin neural model for uplift. The methods include obtaining a dataset corresponding to a prediction task; performing, via a randomizer, a random selection of whether to apply an artificial neural network comprising a Siamese neural network to the dataset or whether to perform a random prediction for the prediction task; determining subsequent to the random selection of the randomizer, a gathered outcome for each prediction scenario based on applying or not applying the model to the dataset including a difference between the gathered outcome for each scenario; and, feeding back the gathered outcome and the difference to retrain the Siamese neural network model for use in performing the prediction task.