Noise-Based Neural Network Training for Bias Reduction

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Neural networks are biased due to traditional training data selection, leading to inefficiencies and inaccuracies, and the process of labeling data for training is costly and time-consuming.

Innovation Solution

Training neural networks using noise data, which encompasses all possible image variations, allowing the model to learn without specific biases and reducing the need for labeled data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional labeled training data is used, then the neural network can learn specific patterns, but the model becomes biased and requires expensive manual labeling

Engineering Contradiction:
Improveprediction accuracyVSAvoidtraining data preparation complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

Instead of training the neural network to generate specific target outputs directly, the patent inverts the approach by training the network to predict noise (random values). The training process uses noise labels instead of meaningful labels, and the network learns to map inputs to noise outputs. This inversion eliminates the need for expensive manual labeling while still enabling the network to learn effective feature representations that improve downstream task performance.

Inventive Principle:
Principle #13The other way round (Inversion)

2Productivity

If labeled training data is manually created, then specific patterns can be learned, but the process is time-consuming and costly

Engineering Contradiction:
Improvetraining efficiencyVSAvoiddata labeling time
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent makes the training process self-service by automatically generating noise labels without requiring human annotators. The system itself creates the training labels through random noise generation, eliminating the need for external human resources. This self-service approach to label generation dramatically reduces both time and cost while maintaining training effectiveness.

Inventive Principle:
Principle #25Self-service

3Adaptability or versatility

If selected training images are used, then the model learns relevant patterns, but the data is biased toward specific desired outputs

Engineering Contradiction:
Improvemodel generalizationVSAvoidmodel bias
Core Design Contradiction:
Adaptability or versatilityVSReliability

Solution Approach 1:

The patent applies homogeneity by using uniform noise distribution across all training samples regardless of input content. Instead of selecting diverse labeled images that introduce bias, the system uses identical noise labeling strategy for all inputs, creating a homogeneous training regime that prevents the model from learning biased associations between specific inputs and specific outputs, thereby improving generalization.

Inventive Principle:
Principle #33Homogeneity

Data Source

PatentUS11593653B2System and method for noise-based training of a prediction model
Publication Date: 2023.02.28 AIVITAE LLC
  • US11593653B2 patent drawing
  • US11593653B2 patent drawing
  • US11593653B2 patent drawing

AI summary

In some embodiments, noise data may be used to train a neural network (or other prediction model). In some embodiments, input noise data may be obtained and provided to a prediction model to obtain an output related to the input noise data (e.g., the output being a prediction related to the input noise data). One or more target output indications may be provided as reference feedback to the prediction model to update one or more portions of the prediction model, wherein the one or more portions of the prediction model are updated based on the related output and the target indications. Subsequent to the portions of the prediction model being updated, a data item may be provided to the prediction model to obtain a prediction related to the data item (e.g., a different version of the data item, a location of an aspect in the data item, etc.).