Neural Network Training Device with Additional Unit for Speech Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks used in pattern detection devices face high computational costs due to the need to calculate posterior probabilities for all classes, even when only a few classes are relevant, such as detecting a keyword in a speech signal, as they typically use the softmax function which requires input values from all units.
Innovation Solution
A training device and method that trains a neural network with an output layer including N units for classes and an additional unit, using a propagator, probability calculator, and updater to reduce computational cost by only calculating input values for relevant classes, employing a function that generates logarithmic posterior probabilities for specific classes without needing all input values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the softmax function is used for the output layer of the neural network, then the neural network can output posterior probabilities for all classes, but the computational cost increases because input values for all units must be calculated
Solution Approach 1:
The patent extracts only the necessary computation for the target class by introducing an additional unit that directly calculates the logarithmic posterior probability for the specific target class, bypassing the need to compute all N class probabilities required by the traditional softmax function. This extraction principle reduces computational cost while maintaining the accuracy needed for pattern detection tasks.
Solution Approach 2:
Instead of performing the complete softmax calculation for all N classes, the patent implements a partial action by adding just one additional unit that computes only the required logarithmic posterior probability for the target class. This partial computation approach achieves the necessary precision without the excessive computational burden of calculating all class probabilities.
2Adaptability or versatility
If the neural network outputs posterior probabilities for all classes, then comprehensive classification is achieved, but the device complexity increases
Solution Approach 1:
The patent extracts only the essential output needed for pattern detection by adding a single additional unit that provides the logarithmic posterior probability for the target class. This extraction approach maintains comprehensive classification capability for the target pattern while reducing output layer complexity compared to computing all N class probabilities.
3Measurement precision
If the neural network calculates input values for all units, then accurate posterior probabilities for all classes are obtained, but the processing time increases
Solution Approach 1:
The patent extracts only the necessary computation for the target class by introducing an additional unit that directly calculates the logarithmic posterior probability, bypassing the time-consuming calculation of all N class probabilities. This extraction principle maintains accuracy for the target pattern while significantly reducing processing time.
Solution Approach 2:
Instead of performing the complete calculation for all classes, the patent implements partial action by computing only the required logarithmic posterior probability for the target class through the additional unit, achieving necessary precision with reduced processing time.
Data Source
AI summary
According to an embodiment, a training device trains a neural network that outputs a posterior probability that an input signal belongs to a particular class. An output layer of the neural network includes N units respectively corresponding to classes and one additional unit. The training device includes a propagator, a probability calculator, and an updater. The propagator supplies a sample signal to the neural network and acquires (N+1) input values for each unit at the output layer. The probability calculator supplies the input values to a function to generate a probability vector including (N+1) probability values respectively corresponding to the units at the output layer. The updater updates a parameter included in the neural network in such a manner to reduce an error between a teacher vector including (N+1) target values and the probability vector. A target value corresponding to the additional unit is a predetermined constant value.