Deep convex network with joint use of nonlinear random projection, restricted boltzmann machine and batch-based parallelizable optimization

Abstract

A method is disclosed herein that includes an act of causing a processor to access a deep-structured, layered or hierarchical model, called deep convex network, retained in a computer-readable medium, wherein the deep-structured model comprises a plurality of layers with weights assigned thereto. This layered model can produce the output serving as the scores to combine with transition probabilities between states in a hidden Markov model and language model scores to form a full speech recognizer. The method makes joint use of nonlinear random projections and RBM weights, and it stacks a lower module's output with the raw data to establish its immediately higher module. Batch-based, convex optimization is performed to learn a portion of the deep convex network's weights, rendering it appropriate for parallel computation to accomplish the training. The method can further include the act of jointly substantially optimizing the weights, the transition probabilities, and the language model scores of the deep-structured model using the optimization criterion based on a sequence rather than a set of unrelated frames.

Description

Technical field [0001] The present invention relates to technology related to automatic classification. Background technique [0002] Speech recognition has been a subject of extensive research and commercial development. For example, voice recognition systems have been incorporated into mobile phones, desktop computers, automobiles, etc. to provide specific responses to voice input provided by users. For example, in a mobile phone equipped with voice recognition technology, the user can speak the name of a contact listed in the mobile phone, and the mobile phone can initiate a call to the contact. In addition, many companies are currently using voice recognition technology to help customers in identifying company employees, identifying product or service issues, and so on. [0003] Partly motivated by the requirement to utilize certain similar attributes in human speech generation and perception systems, research on Automatic Speech Recognition (ASR) has pioneered a layered arch...

AI Technical Summary

Problems solved by technology

Although this learning algorithm has been shown to be powerful when combined with fine-tuning the weights assigned to the DBN, such a learning algorithm is extremely difficult to parallelize across machines, making learning somewhat tedious

Images