Iterative Neural Network Label Generation via Disjunctive Subsets

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Current machine learning methods, particularly those using neural networks, rely heavily on supervised learning which requires manually annotated labeled data, making the process time-consuming and costly, and prone to errors that can propagate throughout the training process.

Innovation Solution

A method for generating labels iteratively using disjunctive subsets, where models are trained and used to predict labels for each other, starting from an initial labeled subset, allowing for the improvement of label quality and preventing error propagation, with the option to use automatic or non-trained pattern recognition algorithms.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual labeling is used for training data, then label quality can be ensured, but the process becomes time-consuming and costly

Engineering Contradiction:
Improvelabel qualityVSAvoidlabeling time
Core Design Contradiction:
Measurement precisionVSLoss of time

Solution Approach 1:

The system uses automatically generated labels from initial models to train subsequent models, which then generate improved labels autonomously without human intervention. The models serve themselves by continuously improving their own labeling capability through iterative training on disjunctive data subsets.

Inventive Principle:
Principle #25Self-service

Solution Approach 2:

The iterative process incorporates feedback loops where models train on generated labels, evaluate performance, and use the results to improve subsequent labeling. The disjunctive subset approach ensures that feedback from one iteration informs and improves the next iteration's label generation quality.

Inventive Principle:
Principle #23Feedback

2Measurement precision

If manual labeling is used for training data, then accurate labels can be obtained, but costs increase significantly

Engineering Contradiction:
Improvelabel accuracyVSAvoidlabeling cost
Core Design Contradiction:
Measurement precisionVSLoss of energy

Solution Approach 1:

Instead of manually creating labels, the system creates initial labels automatically and uses these copies as training data. Subsequent models generate improved label copies through iterative training, replacing the need for expensive manual label creation while maintaining accuracy through the copying and refinement process.

Inventive Principle:
Principle #26Copying

Solution Approach 2:

The system changes the parameter of label generation from manual human annotation to automated model prediction. By adjusting the iteration count and model complexity parameters, the system achieves high label accuracy at reduced cost compared to manual methods.

Inventive Principle:
Principle #35Parameter changes

3Measurement precision

If iterative model training is implemented, then label quality improves, but computational complexity increases

Engineering Contradiction:
Improvelabel qualityVSAvoidcomputational complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The data set is segmented into multiple disjunctive subsets that are used alternately for training and validation across different iterations. This segmentation allows the computational workload to be distributed and managed more efficiently, reducing peak complexity requirements while maintaining iterative improvement.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The system dynamically adjusts the training process by alternating between different data subsets and models in each iteration. This dynamic approach allows computational resources to be optimized at each step, improving label quality progressively without requiring maximum computational complexity throughout the entire process.

Inventive Principle:
Principle #15Dynamics

4Reliability

If models are trained on all available data, then comprehensive learning is achieved, but overfitting occurs

Engineering Contradiction:
Improvemodel generalizationVSAvoidrecognition performance
Core Design Contradiction:
ReliabilityVSMeasurement precision

Solution Approach 1:

The data is segmented into disjunctive subsets where each model trains only on a portion of the data during each iteration. This segmentation prevents any single model from overfitting to the entire data set while still achieving comprehensive learning through the iterative process that eventually utilizes all subsets.

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

Instead of training models on all data simultaneously, the system inverts the approach by training models on disjunctive subsets and using them to generate labels for other subsets. This inverted training strategy prevents overfitting while maintaining the ability to learn from the complete data set through the iterative label generation process.

Inventive Principle:
Principle #13The other way round (Inversion)

Data Source

PatentUS20210192345A1Method for generating labeled data, in particular for training a neural network, by using unlabeled partitioned samples
Publication Date: 2021.06.24 ROBERT BOSCH GMBH
  • US20210192345A1 patent drawing
  • US20210192345A1 patent drawing
  • US20210192345A1 patent drawing

AI summary

A method and a device for generating labeled data, for example training data, in particular for a neural network.