Neural Network Training Data Cleaning via Iterative Subset Cycling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Training neural networks with incorrectly labeled data elements leads to poor performance due to interference with learning algorithms, as many training sets contain a substantial number of incorrectly labeled data examples.

Innovation Solution

A method involving cycling between two subsets of a labeled training data set to refine and filter each subset using progressively improved models, ultimately combining the refined subsets to create a cleaned data set for training a target neural network with improved accuracy.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If a training data set with incorrectly labeled data elements is used to train a neural network, then the training process can proceed without data cleaning, but the accuracy and performance of the trained neural network deteriorates

Engineering Contradiction:

Improveaccuracy of trained neural networkVSAvoidcomplexity of data cleaning process

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The training data set is divided into multiple subsets, and the cleaning process is applied iteratively to each subset. This segmentation allows the complex cleaning task to be broken down into manageable steps, where each iteration refines a portion of the data using a model trained on previously cleaned data.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The method performs preliminary cleaning actions by training a model on an initially cleaned subset of data, then using this model to identify and correct incorrectly labeled data elements in remaining subsets. This preliminary model serves as a foundation for subsequent cleaning iterations, progressively improving overall data quality.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If iterative cycling between subsets is performed to refine and filter data, then the cleanliness and accuracy of the training data set improves, but the time and computational resources required increase

Engineering Contradiction:

Improvecleanliness of training dataVSAvoidtime for data cleaning process

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The data cleaning process employs periodic iterative cycling between different data subsets. In each iteration, a model is trained on one subset and then used to clean another subset, with results fed back into subsequent iterations. This periodic action progressively refines data cleanliness while managing computational workload through structured repetition.

Inventive Principle:

Principle #19Periodic action

Solution Approach 2:

The method applies cleaning actions partially by focusing on specific subsets in each iteration rather than attempting to clean the entire data set simultaneously. This partial action approach allows the process to make incremental improvements to data cleanliness without overwhelming computational resources, balancing thoroughness with efficiency.

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11455534B2Data set cleaning for artificial neural network training

Publication Date: 2022.09.27 MACRONIX INTERNATIONAL CO LTD

AI summary

A technology for cleaning a training data set for a neural network using dirty training data starts by accessing a labeled training data set that includes relatively dirty labeled data elements. The labeled training data set is divided into a first subset A and a second subset B. The procedure includes cycling between the subsets A and B, including producing refined model-filtered subsets of subsets A and B to provide a cleaned data set. Each refined model-filtered subset can have improved cleanliness and increased numbers of elements.