Neural Network Output Layer Update for Voice Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional voice recognition models require extensive relearning and large amounts of data to add new words or characters, resulting in high costs and time consumption.
Innovation Solution
A model learning apparatus and method that adds a unit corresponding to a word or character to the output layer of a neural network model, calculates the output probability distribution, and updates the output layer parameters based on the correct unit number and calculated distribution, allowing for efficient addition of new linguistic elements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional relearning is performed to add new words or characters to the voice recognition model, then the model can recognize new linguistic elements, but large amounts of learning data and time are required, resulting in high cost
Solution Approach 1:
The patent segments the learning process into two distinct phases: initial comprehensive learning of the base model using full learning data, and subsequent incremental learning for adding new words or characters using only small amounts of additional data. This segmentation allows the system to avoid complete relearning while still achieving adaptation to new linguistic elements.
Solution Approach 2:
The patent performs preliminary action by pre-training the voice recognition model with comprehensive learning data before deployment. This preliminary learning establishes a robust base model that can later be incrementally updated with minimal additional data, eliminating the need for extensive relearning when new words or characters need to be added.
2Adaptability or versatility
If conventional relearning is performed to add new words or characters, then the model updates its recognition capabilities, but large amounts of learning data are required, resulting in high cost
Solution Approach 1:
The patent segments the learning data usage into initial comprehensive training and subsequent incremental updates. The base model is trained on full learning data upfront, while new words or characters are added using only small amounts of additional data, dramatically reducing the data volume required for updates.
Solution Approach 2:
The patent performs preliminary action by pre-training the model with comprehensive learning data before deployment. This preliminary learning establishes a robust base model that can later be incrementally updated with minimal additional data, eliminating the need for extensive data collection for each update.
3Adaptability or versatility
If the output layer of the neural network model is modified to add new units, then the model can recognize new words or characters, but the parameter space increases requiring more data for training
Solution Approach 1:
The patent segments parameter learning into two phases: base model parameters are learned comprehensively from full learning data, while new parameters for added words or characters are learned incrementally from small additional data sets. This segmentation prevents the need to relearn all parameters when the output layer is expanded.
Solution Approach 2:
The patent performs preliminary action by pre-learning the base model parameters before expanding the output layer. When new units are added to the output layer, only the new parameters need to be learned from minimal additional data, rather than relearning the entire parameter space.
Data Source
AI summary
Apparatuses and methods are provided for adding a word or a character to a machine learning model for training the machine learning model. In particular, a model learning apparatus executes operations comprising adding a word or a character to be added to a neural network as a machine learning model to the output layer of the neural network. The operations further comprise calculating an output probability distribution of an output from the output layer of the neural network when a feature amount of the word or the character is input to the neural network. Given the output probability distribution and a correct feature amount of the word or the character, the operations further comprise updating a parameter of the output layer of the neural network.