Encoder Decoder Noise Injection Latent Space Stability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing generative deep learning models face instability in machine learning due to variations in restoration errors, especially when the number of parameters increases, leading to poor convergence of encoder and decoder parameters.

Innovation Solution

A machine learning program that adds noise to the output of the encoder to generate a latent variable, which is then input to a decoder for inverse computation, while training the encoder and decoder using a loss function that includes the noisy latent data and the error between the input and output data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If the number of parameters in the encoder and decoder is increased to improve model capacity, then the model can capture more complex features, but the machine learning becomes unstable and parameter convergence deteriorates

Engineering Contradiction:

Improvemodel capacityVSAvoidmachine learning stability

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent introduces a latent variable z as an intermediary between the encoder and decoder. The encoder transforms input data x into latent variable z, which then serves as input to the decoder. This intermediary latent representation simplifies the direct mapping between input and output, enabling stable training even with large numbers of parameters by breaking down the complex transformation into manageable stages through the latent space bottleneck.

Inventive Principle:

Principle #24Intermediary (Mediator)

Solution Approach 2:

The patent transforms the data representation from the original high-dimensional input space to a lower-dimensional latent space through the encoder. By projecting data into this compressed latent dimension and then reconstructing it through the decoder, the model captures essential features while reducing computational complexity and improving training stability, effectively managing the trade-off between model capacity and convergence reliability.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Reliability

If noise is added to the output of the encoder to create latent variable, then the machine learning stability improves and parameter convergence is ensured, but the information loss increases

Engineering Contradiction:

Improvemachine learning stabilityVSAvoidinformation loss in latent representation

Core Design Contradiction:

ReliabilityVSLoss of information

Solution Approach 1:

The patent implements a feedback mechanism through the loss function that measures the reconstruction error between the original input data x and the reconstructed data x'. This feedback signal guides the optimization of both encoder and decoder parameters, enabling the model to learn representations that minimize information loss while maintaining training stability. The feedback loop ensures that the latent variable z captures sufficient information for accurate reconstruction despite the intermediate noise addition.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent converts the potentially harmful effect of noise addition into a beneficial regularization mechanism. By adding noise to the encoder output and using it as the decoder input, the model is forced to learn more robust latent representations that are invariant to small perturbations. This approach, combined with the loss function, transforms what could be pure information loss into a useful constraint that improves generalization and training stability.

Inventive Principle:

Principle #22Blessing in disguise (Convert harm into benefit)

3Reliability

If the loss function includes the noisy latent data and reconstruction error, then the encoder and decoder parameters converge consistently, but the computational complexity increases

Engineering Contradiction:

Improveparameter convergenceVSAvoidcomputational complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the computational task into distinct operations: encoding input data x to latent variable z, adding noise to z, decoding z back to reconstructed data x', and computing the loss between x and x'. This segmentation allows each component to be optimized independently and simplifies the overall computational graph, making the loss function more manageable despite including multiple data processing steps. The modular structure reduces the practical computational burden while maintaining convergence reliability.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS20250094799A1Computer-readable recording medium storing machine learning program, machine learning method, and information processing apparatus

Publication Date: 2025.03.20 FUJITSU LTD

AI summary

A non-transitory computer-readable recording medium stores a machine learning program for causing a computer to execute a process including: inputting first data to an encoder, and acquiring third data obtained by adding noise to second data output by the encoder; inputting the third data to a decoder that corresponds to inverse computation of the encoder, and acquiring fourth data output by the decoder; and training the encoder and the decoder based on a loss function that includes the third data and an error between the first data and the fourth data.