Provenance Data Analysis for Machine Learning Model Quality

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Users face difficulties in determining how to train machine learning models on cloud-based platforms due to the vast number of available datasets, algorithms, and infrastructures, leading to challenges in selecting suitable models and improving model quality.

Innovation Solution

The method involves collecting and analyzing provenance data to identify model quality improvements and recommend related models based on dataset and model context, user profiles, and access patterns, providing suggestions for dataset refinements, data transformations, and model parameters.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If cloud-based platforms offer a large choice of datasets, algorithms and infrastructures, then model training capabilities are improved, but user difficulty in determining how to train models increases

Engineering Contradiction:

Improvemodel training capabilitiesVSAvoiduser difficulty in determining how to train models

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The system collects provenance data from model training operations and uses it to generate recommendations, creating a feedback loop that improves ease of operation over time while maintaining platform versatility

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent introduces an intermediary recommendation system that processes provenance data and presents simplified guidance to users, mediating between the complex platform capabilities and user decision-making

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If provenance data is collected and analyzed, then model quality improvement recommendations are improved, but data processing complexity increases

Engineering Contradiction:

Improvemodel quality assessment accuracyVSAvoiddata processing complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The system extracts only the necessary provenance data elements required for quality assessment, separating essential information from unnecessary complexity in data collection and processing

Inventive Principle:

Principle #2Taking out (Extraction)

Data Source

PatentUS11205138B2Model quality and related models using provenance data

Publication Date: 2021.12.21 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A method, computer system, and a computer program product for utilizing provenance data to improve machine learning is provided. Embodiments of the present invention may include collecting provenance data. Embodiments of the present invention may include identifying model quality improvements based on the collected provenance data. Embodiments of the present invention may include identifying related models based on the collected provenance data. Embodiments of the present invention may include recommending model quality improvements to a user.