Predictive Model Input Transformation via Partial Regression Trends

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Predictive models often struggle with inconsistent relationships between independent variables and predictions, leading to irrelevant predictors being eliminated during dimensional reduction, potentially removing high-value variables that do not contribute to dependent variable determination.

Innovation Solution

The method involves selecting a predictor, excluding it from the model, computing label and variable residuals, and constructing a second model to predict the excluded predictor, generating a transformation to align its trend with the prediction model's trend, thereby improving predictor relationships and model accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Device complexity

If dimensional reduction is performed to eliminate irrelevant predictors, then model complexity is reduced, but high-value variables that do not contribute to dependent variable determination may be incorrectly removed

Engineering Contradiction:
Improvemodel complexityVSAvoidprediction accuracy
Core Design Contradiction:
Device complexityVSReliability

Solution Approach 1:

The patent replaces traditional mechanical dimensional reduction methods (stepwise regression, feature selection algorithms) with a causal inference approach based on partial regression plots and causation relationships. This substitution allows variables to be retained based on their causal relevance to the outcome rather than merely their statistical correlation, preventing the removal of high-value variables that have indirect but important causal pathways.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent transforms the approach to variable selection by changing from statistical parameters (correlation coefficients, p-values) to causal parameters (causation relationships, partial regression trends). This parameter change enables the identification of variables that may not show strong direct correlation but have important causal relationships through intermediate variables, thus preserving high-value predictors during dimensional reduction.

Inventive Principle:
Principle #35Parameter changes

2Stability of the object's composition

If predictors with inconsistent relationships are removed, then model stability is improved, but potential high-value predictors are eliminated

Engineering Contradiction:
Improvemodel stabilityVSAvoidprediction accuracy
Core Design Contradiction:
Stability of the object's compositionVSReliability

Solution Approach 1:

The patent replaces the mechanical filtering of unstable predictors with a causal analysis system that examines partial regression plots. Variables with inconsistent relationships are not automatically removed but are analyzed for underlying causal structures. The system identifies whether inconsistency arises from confounding factors or intermediate variables, allowing stable retention of predictors that have genuine causal relationships even when direct correlations vary.

Inventive Principle:
Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent introduces causation relationships as intermediary concepts between predictors and outcomes. When a predictor shows inconsistent direct relationship with the outcome, the system searches for intermediate variables that mediate this relationship. The partial regression plot serves as an intermediary analytical tool that decomposes the relationship into direct and indirect effects, allowing the model to maintain stability while preserving predictors with complex causal pathways.

Inventive Principle:
Principle #24Intermediary (Mediator)

3Loss of time

If traditional dimensional reduction is applied, then computation time is reduced, but understanding of causation relationships between predictors and predictions is lost

Engineering Contradiction:
Improvecomputation timeVSAvoidcausation relationship understanding
Core Design Contradiction:
Loss of timeVSLoss of information

Solution Approach 1:

The patent performs preliminary causal analysis using partial regression plots before finalizing the predictive model. This preliminary action involves generating causation relationship maps that identify direct and indirect effects of predictors. By establishing this causal understanding beforehand, the model can make faster predictions without repeatedly analyzing causal relationships, thus reducing computation time while preserving causation understanding.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent segments the analysis into distinct phases: (1) exploratory causal analysis using partial regression plots to identify causation relationships, (2) model building with identified causal structures, and (3) prediction execution using the established model. This segmentation allows computationally intensive causal analysis to be performed only once during model development, while subsequent predictions use the pre-established causal framework, thereby reducing overall computation time without losing causation understanding.

Inventive Principle:
Principle #1Segmentation

Data Source

PatentUS10229368B2Machine learning of predictive models using partial regression trends
Publication Date: 2019.03.12 KYNDRYL INC
  • US10229368B2 patent drawing
  • US10229368B2 patent drawing
  • US10229368B2 patent drawing

AI summary

An input is selected from a set of inputs used by a prediction model to produce an initial predicted value of an outcome. A changed predicted value of the outcome is produced by removing the selected input from the inputs to the model. An actual value of the outcome is obtained. A label residual is computed using the actual value and the changed predicted value. A second prediction model is formed to predict a value of the selected input. A variable residual is computed using an actual value and the predicted value of the selected input. An expression is generated of a plot of the label residual and the variable residual. The selected input is transformed, to form a transformed selected input, where the model produces a second predicted value of the outcome by using the transformed selected input.