Ensemble Data Curation Pipeline for Label Propagation

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing systems for generating labeled datasets for supervised machine learning face challenges with accuracy and consistency due to noise and mislabeling, leading to degraded performance in classification tasks.

Innovation Solution

A label propagation system that automates the labeling process, uses clustering and dedicated classifiers to identify and correct labels, and iteratively refines the dataset, ensuring accuracy and consistency through a voting mechanism and data curation.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual validation and correction of labels is performed, then label accuracy is improved, but time consumption increases significantly

Engineering Contradiction:

Improvelabel accuracyVSAvoidtime consumption

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system enables self-service by implementing an automated label propagation mechanism where the system automatically corrects mislabeled data through iterative label propagation and voting mechanisms, eliminating the need for continuous manual validation while maintaining high label accuracy

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The system implements feedback through iterative label propagation where labels are continuously refined based on voting results from multiple classifiers, and the process repeats until convergence or maximum iterations are reached, automatically improving label accuracy without additional manual intervention

Inventive Principle:

Principle #23Feedback

2Loss of time

If manual validation of test dataset labels is performed, then time consumption is reduced, but determination of model performance impact by mislabeled data becomes impossible

Engineering Contradiction:

Improvetime consumptionVSAvoidmodel performance impact information

Core Design Contradiction:

Loss of timeVSLoss of information

Solution Approach 1:

The system implements comprehensive feedback by tracking label changes across iterations, recording which samples were mislabeled and how they were corrected, thereby providing complete information about the impact of mislabeled data on model performance while maintaining efficiency through automation

Inventive Principle:

Principle #23Feedback

3Productivity

If label propagation is performed without automated curation, then data processing speed is improved, but label consistency and accuracy deteriorate due to noise and mislabeling

Engineering Contradiction:

Improvedata processing speedVSAvoidlabel consistency

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system introduces an intermediary curation layer between raw labeled data and the training process, where a dedicated curation module performs automated validation and correction of labels through voting mechanisms and consistency checks, ensuring high label reliability while maintaining processing speed

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS11372893B2Ensemble-based data curation pipeline for efficient label propagation

Publication Date: 2022.06.28 NTT SECURITY HLDG CORP

AI summary

A method and architecture allow labeled data to be initialized and curated by the use of label propagation, clustering and creation of dedicated classifiers for each dataset. These dedicated classifiers are retrained as the dataset grows and ensure that each dataset is kept free from outliers and noise.