Data Transition Capture for Optimal Machine Learning Model Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models in fraud detection and business risk analysis lack the ability to trace and measure data changes, transitions, and transformations, making it difficult to explain the results and select an optimal model.
Innovation Solution
A system and method that captures data transitions in machine learning models by comparing new electronic data sets to previous training data sets, identifying added, removed, and changed attributes, and recording these changes as data state transitions. This system also generates an implication tree based on present and past data states to select an optimal machine learning model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data intensive machine learning models are used to analyze large data sets with numerous parameters and metrics, then model effectiveness and detection capability are improved, but the ability to trace and explain data transformations is lost
Solution Approach 1:
The patent introduces an intermediary component that captures and stores data lineage information between data transformations and machine learning model executions. This intermediary mechanism tracks the origin, transformations, and flow of data without interfering with the model's analytical capabilities, thereby maintaining both model effectiveness and traceability.
Solution Approach 2:
The system creates copies of data lineage metadata that parallel the actual data flow. These copies record transformation histories, attribute changes, and data provenance information, enabling explanation and tracing without altering or slowing down the original data intensive processing pipelines.
2Measurement precision
If numerous parameters and metrics are analyzed to detect fraud and risk, then detection accuracy is improved, but the complexity of tracing data transformations increases
Solution Approach 1:
The patent segments the complex data transformation tracing problem into manageable components by tracking data lineage at different granularities - from individual attributes to data sets to transformation operations. This segmentation allows precise detection across numerous parameters while organizing the tracing complexity into structured, queryable units.
Solution Approach 2:
The system adds a new dimension to data processing by incorporating metadata layers that capture transformation histories and lineage information. This additional dimension organizes the complexity of tracing numerous parameters through structured metadata relationships, enabling efficient navigation and analysis without increasing operational complexity.
3Adaptability or versatility
If data transformations are captured and recorded in detail, then data governance and model selection capability are improved, but the system complexity and processing overhead increase
Solution Approach 1:
The patent implements preliminary action by capturing and storing data lineage metadata at the point of transformation, before the data is needed for model training or analysis. This advance preparation organizes transformation histories and provenance information in advance, enabling efficient model selection and comparison without adding complexity during critical analysis phases.
Solution Approach 2:
The system employs self-service mechanisms where the data lineage tracking infrastructure automatically captures, stores, and indexes transformation information as data flows through the system. This automated self-service approach minimizes manual configuration and reduces operational complexity while maintaining detailed records for model selection and governance.
Data Source
AI summary
A computer-implemented system, method and computer program product for capturing data transitions and selecting machine learning models that includes: providing a machine learning model trained with a previous training data set; receiving a new data set; comparing the new data set to the previous data set; and identifying and recording new, removed, and/or changed set of attributes added to the new data set. Future possible data state transitions are generated based upon the present data state; and an implication tree is generated based upon the present data state, the pass-through data states, and the future possible data state transitions. Performance metrics of each node in the implication tree are clustered, the nodes demonstrating a high variance optionally are discarded; reachability scores for remaining nodes are calculated; and a node (representing a machine learning model to run) is selected based upon its reachability score.