Conditional Neural Network Learning via Probabilistic Temperature Sampling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conditional learning methods for neural networks require pre-annotated condition information and data preparation, which is cumbersome and inefficient.

Innovation Solution

An information learning system that includes a condition generation unit, a condition connection unit, and an optimization unit, where the condition generation unit probabilistically changes the temperature of activation to generate conditions from training data, allowing the neural network to learn without pre-prepared condition data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If conditional learning methods (CVAE, CGAN) are used to enable multiple outputs for single input, then the neural network can express various outputs by using condition information, but the design and annotation of training data including condition information becomes cumbersome and inefficient

Engineering Contradiction:

Improveability to express various outputsVSAvoidease of preparing training data

Core Design Contradiction:

Adaptability or versatilityVSEase of manufacture

Solution Approach 1:

The system enables the neural network to automatically generate condition information from the training data itself without requiring external annotation or manual preparation. The condition generation unit extracts conditions directly from the input data, allowing the network to serve its own data preparation needs and eliminating the cumbersome manual annotation process while maintaining the ability to produce diverse outputs

Inventive Principle:

Principle #25Self-service

2Reliability

If condition information is accumulated in advance in a database with annotation, then conditional learning can be performed, but the data preparation process becomes time-consuming and reduces productivity

Engineering Contradiction:

Improvequality of conditional learningVSAvoidspeed of data preparation

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system performs preliminary extraction of condition information during the training phase by having the condition generation unit create conditions from the training data on-the-fly. This preliminary action eliminates the need for time-consuming pre-annotation and database accumulation, allowing the neural network to immediately utilize condition information for learning while maintaining reliable learning quality

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system replaces the manual mechanical process of data annotation and condition preparation with an automated computational process. The condition generation unit uses algorithms to automatically generate condition information from training data, substituting the slow manual annotation process with fast computational processing that maintains learning reliability while dramatically improving productivity

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Data Source

PatentUS20220414465A1Information learning system, information learning method, information learning program, and information learning apparatus

Publication Date: 2022.12.29 NEC CORP

AI summary

An information learning system includes: a condition generation unit that generates a condition from training data that are inputted to a neural network; a condition connection unit that connects the condition to a feature quantity of the training data; and an optimization unit that optimizes a parameter of the neural network by using the feature quantity to which the condition is connected. The condition generation unit includes a temperature sampling unit that probabilistically changes a temperature of activation. This makes it possible to appropriately perform the learning of a conditional neural network.