Neural Network Training Data Expansion via Reliability Filtering

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Insufficient training data can degrade the performance of neural networks, leading to reduced accuracy and increased risk of over-fitting, especially when the data is unevenly distributed or features have varying distances.

Innovation Solution

The method involves extracting annotation data from unannotated external data using a neural network trained on initial data, selecting data with high reliability, expanding the training dataset, and retraining the network based on the expanded data to enhance performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If training data is expanded using unannotated external data, then the quantity of training data increases, but the reliability of the training data decreases due to potential annotation errors

Engineering Contradiction:

Improvequantity of training dataVSAvoidreliability of training data

Core Design Contradiction:

Quantity of substanceVSReliability

Solution Approach 1:

The system performs self-verification by using the trained neural network to annotate unannotated data, then verifies these annotations through iterative retraining processes. The network continuously refines its own annotations by comparing against newly added verified data, creating a feedback loop that improves reliability while expanding data quantity.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

Before fully integrating unannotated external data into the training set, the system performs preliminary annotation using the existing trained neural network. This preliminary action filters and prepares the data in advance, ensuring only appropriately annotated data is added, thus maintaining reliability while increasing quantity.

Inventive Principle:

Principle #10Preliminary action

2Measurement precision

If the neural network is retrained iteratively with expanded data, then the performance and accuracy of the neural network improve, but the training time and computational resources increase

Engineering Contradiction:

Improveaccuracy of neural networkVSAvoidtraining time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs partial retraining by selectively updating network parameters using only the newly added verified data rather than retraining on the entire dataset. This approach achieves performance improvement with reduced computational overhead and training time compared to complete retraining.

Inventive Principle:

Principle #16Partial or excessive action

Solution Approach 2:

The iterative retraining process continuously incorporates newly verified data into the training set, maintaining continuous improvement of network performance. This continuous action allows the system to progressively enhance accuracy without requiring complete retraining from scratch each time, reducing overall training time.

Inventive Principle:

Principle #20Continuity of useful action

Data Source

PatentUS20210264260A1Method and device for training neural network

Publication Date: 2021.08.26 SAMSUNG ELECTRONICS CO LTD

AI summary

The present disclosure relates to neural network training. The neural network training relates to a training method, a training device, and a system including the neural network. The neural network training includes extracting annotation data and first reliability values for first data using a neural network trained based on training data, selecting second data from among the first data based on the second data having second reliability values greater than or equal to a threshold value, expanding the training data based on the second data, and retraining the neural network based on the expanded training data