Decision Tree Data Transformation Criteria for Reliable Lineage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

The lack of authoritative understanding of data transformations poses a critical technical hurdle for organizations, leading to potential errors and inaccuracies in data lineage, especially in regulatory reporting and mission-critical applications, due to manual and ad hoc documentation methods.

Innovation Solution

The application of machine learning tools to automatically derive data transformation criteria using a decision tree classifier, which predicts a target variable's value from a source dataset and generates pseudocode for producing the target variable, thereby ensuring accurate and reliable documentation of data lineage.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If manual documentation methods are used for data transformations, then flexibility and adaptability are maintained, but accuracy and reliability of data lineage documentation deteriorate

Engineering Contradiction:

Improveaccuracy of data lineage documentationVSAvoidcomplexity of documentation system

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces manual mechanical documentation processes with an automated machine learning system. The system uses decision tree classifiers to automatically discover and document data transformations, substituting human manual tracking with algorithmic automation. This resolves the contradiction by maintaining reliability through automated consistency while managing complexity through intelligent algorithms rather than procedural overhead.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system enables self-service automation where the documentation system automatically generates its own content by analyzing data transformations. The machine learning model autonomously discovers transformation criteria and documents lineage without requiring manual intervention, thereby improving reliability through consistent automated documentation while keeping the system relatively simple in operation.

Inventive Principle:

Principle #25Self-service

2Measurement precision

If automated machine learning methods are used to derive data transformation criteria, then accuracy and consistency of data lineage documentation improve, but computational resources and system complexity increase

Engineering Contradiction:

Improveprecision of transformation criteriaVSAvoidcomputational resources required

Core Design Contradiction:

Measurement precisionVSUse of energy by moving object

Solution Approach 1:

The system applies partial automation by focusing machine learning resources on the most critical data transformations and lineage paths. Rather than automating every possible transformation equally, the system prioritizes documentation of transformations that have the greatest impact on data quality and compliance, thereby achieving high precision where needed while conserving computational resources.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The system dynamically adjusts computational parameters such as model complexity, training data sampling rates, and transformation analysis depth based on the specific context and requirements. This allows the system to achieve high measurement precision when necessary while reducing computational resource consumption during routine or less critical documentation tasks.

Inventive Principle:

Principle #35Parameter changes

3Loss of information

If comprehensive tracking of all data transformations is implemented, then complete understanding of data lineage is achieved, but time and computational resources required increase significantly

Engineering Contradiction:

Improvecompleteness of data lineage informationVSAvoidtime for documentation

Core Design Contradiction:

Loss of informationVSLoss of time

Solution Approach 1:

The system implements continuous automated monitoring of data transformations as they occur, rather than performing periodic manual audits. The machine learning model continuously discovers and documents transformations in real-time or near-real-time, ensuring complete information capture without requiring significant time investment from human operators, as the automation runs continuously with minimal intervention.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS12524435B2Systems and methods for automatically deriving data transformation criteria

Publication Date: 2026.01.13 WELLS FARGO BANK NA

AI summary

Systems, apparatuses, methods, and computer program products are disclosed for automatically deriving data transformation criteria. An example method includes receiving, by communications circuitry, a source dataset and a target dataset and identifying, by a model generator, a target variable. The example method further includes training, by the model generator, a decision tree for the target variable using the source dataset and the target dataset such that the trained decision tree can predict a value for the target variable from new source data. The example method further includes deriving, by a derivation engine, a set of parameters and pseudocode for producing the target variable from the source dataset.