Pre-trained Tabular Classification Model for Data Augmentation

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing deep generative models are ineffective for generating tabular data, particularly for smaller datasets or with low training costs, due to the need for substantial training time and hyperparameter tuning.

Innovation Solution

A pre-trained tabular classification model is used to generate tabular data by determining a class-conditional energy function, allowing for the sampling of data points from the underlying distribution modeled by the tabular classification model without additional training or hyperparameter tuning.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Manufacturing precision

If conventional deep learning-based models are used for tabular data generation, then the quality of generated data can be improved, but the training time and computational resources required increase substantially

Engineering Contradiction:
Improvequality of generated dataVSAvoidtraining time
Core Design Contradiction:
Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent applies preliminary action by pre-training a tabular classification model on a large dataset before using it for data generation. The pre-trained model's weights and biases are initialized from this training phase, allowing the subsequent energy-based generation process to start from a knowledgeable state rather than requiring extensive training for each generation task. This resolves the contradiction by performing the heavy learning work upfront, enabling fast adaptation to new data generation tasks.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent uses copying by creating synthetic data samples that replicate the statistical properties and distributions of the training data. The energy-based model generates new data points that copy the underlying patterns and relationships learned during pre-training, allowing high-quality data generation without retraining the model. This enables quality data production with minimal computational overhead.

Inventive Principle:
Principle #26Copying

2Manufacturing precision

If conventional deep learning-based models are used for tabular data generation, then the quality of generated data can be improved, but the computational resources and training effort required increase substantially

Engineering Contradiction:
Improvequality of generated dataVSAvoidtraining effort
Core Design Contradiction:
Manufacturing precisionVSDevice complexity

Solution Approach 1:

The pre-training phase performs the complex learning of data distributions and relationships in advance. The pre-trained classification model captures the essential patterns of tabular data, which are then leveraged during the generation phase without requiring additional complex training procedures. This separates the complex learning task from the generation task, reducing the computational effort needed for each generation operation.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent replaces the traditional mechanical training process with an energy-based sampling approach. Instead of using gradient descent and backpropagation for each generation task, the system uses energy functions and Markov chain Monte Carlo methods to sample from the learned distribution. This substitution eliminates the need for repeated heavy training while maintaining data quality.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

3Manufacturing precision

If conventional deep learning-based models are used for tabular data generation, then the quality of generated data can be improved, but the model becomes difficult to use across diverse datasets without dedicated training

Engineering Contradiction:
Improvequality of generated dataVSAvoidcross-dataset usability
Core Design Contradiction:
Manufacturing precisionVSAdaptability or versatility

Solution Approach 1:

The patent achieves universality by designing a pre-trained tabular classification model that can serve multiple purposes: both as a classifier and as a generator for diverse datasets. The pre-training on a large, diverse corpus of tabular data gives the model general knowledge that transfers to different domains and data distributions. This single pre-trained model can be adapted to generate data for multiple different datasets without requiring dedicated training for each, resolving the contradiction between quality and versatility.

Inventive Principle:
Principle #6Universality (Multi-functionality)

Solution Approach 2:

The patent uses parameter changes by adjusting the input context and energy function parameters to adapt the pre-trained model to different target datasets. Rather than retraining the model weights for each dataset, the system modifies the energy function parameters and sampling conditions to match the desired data distribution. This allows the same pre-trained model to generate high-quality data across diverse datasets through parameter adjustment alone.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS20250124220A1Tabular data generation
Publication Date: 2025.04.17 THE TORONTO DOMINION BANK
  • US20250124220A1 patent drawing
  • US20250124220A1 patent drawing
  • US20250124220A1 patent drawing

AI summary

A tabular data model, which may be pre-trained on a different data set, is used to generate data samples for a target class with a given set of context data points. The tabular data model is trained to predict class membership of a given data point with a set of context data points. Rather than use the predicted class directly, the class predictions are used to determine a class-conditional energy for a synthetic data point with respect to the target class. The synthetic data point may then be updated based on the class-conditional energy with a stochastic update algorithm, such as stochastic gradient Langevin dynamics or Adaptive Moment Estimation with noise. The value of the synthetic data point is sampled as a data point for the target class. This permits effective data augmentation for tabular data for downstream models.