Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Acoustic model modeling method of residual error long short-term memory recurrent neural network

A technology of cyclic neural network and long-term and short-term memory, applied in speech analysis, speech recognition, instruments, etc., can solve the problems of gradient explosion, gradient disappearance, acoustic environment affecting the recognition accuracy of speech recognition system, etc., to improve performance and prevent over-simulation The effect of solving the problem and improving the generalization ability of the model

Active Publication Date: 2017-10-24
TSINGHUA UNIV
View PDF16 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, there are two major problems in the simple feedforward fully connected network: one is that the feedforward fully connected network is difficult to model the historical information of the speech signal; the other is trained in the stochastic gradient descent (Stochastic Gradient Descent, SGD) There may be problems with gradient disappearance (Vanishing Gradient) or gradient explosion (Exploding Gradient) in the process
[0008] However, in practical applications, such methods are still far from the requirements of large-scale commercialization, because the complexity of the acoustic environment still seriously affects the recognition accuracy of the speech recognition system, especially the noise resistance and robustness of the acoustic model. There is still some room for improvement

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Acoustic model modeling method of residual error long short-term memory recurrent neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The implementation of the present invention will be described in detail below in conjunction with the drawings and examples.

[0035] The present invention proposes a method and device for a residual long-short-term memory cyclic neural network acoustic model, especially for continuous speech recognition scenarios. These methods and devices are not limited to continuous speech recognition, and may be any methods and devices related to speech recognition.

[0036] figure 1 It is a flow chart of the acoustic model of the residual long short-term memory recurrent neural network of the present invention, including the following:

[0037] Such as figure 1 The input 101 shown is the speech signal feature x t ; Others are residual long short-term memory cyclic neural network submodule 102, which is composed of memory cell 103, input gate 104, output gate 105, forgetting gate 106, multiplier 107; the output of long short-term memory neural network submodule 102 As the input...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Provided is an acoustic model modeling method of a residual error long short-term memory recurrent neural network. The method comprises: directly connecting input of a standard long short-term memory neural network framework with an internal node, splicing an original vector of the node with an input vector, and then projecting to an original dimensionality. The method provides a position of an internal node for direct connection. Experiments show that the position can bring about performance improvement of identification, and further improve system performance combined with a Dropout technology based on frames. The method improves performance of a speech recognition system based on a long short-term memory recurrent neural network, and prevents an overfitting problem combined with the Dropout technology based on frames, and realizes an objective of improving model generalization ability. The method can be widely applied in the fields of various man-machine interaction related to speech recognition.

Description

technical field [0001] The invention belongs to the field of audio technology, in particular to an acoustic model modeling method of a residual long-short-term memory cyclic neural network. Background technique [0002] With the development of artificial intelligence and computer technology, especially the development of computing hardware such as graphics processors, artificial neural networks (Artificial Neural Network, ANN) are widely used in automatic speech recognition systems. The error rate of speech recognition has also been significantly reduced with the introduction of neural networks and the increase of data sets, so it has become a research hotspot in academia and industry. [0003] The acoustic model plays an important role in the current mainstream speech recognition system, and the improvement of its performance is of great significance for improving the performance of speech recognition. Before the neural network was widely used, the basic architecture of th...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G10L15/06G10L15/16G10L25/30G10L15/14
CPCG10L15/06G10L15/142G10L15/16G10L25/30G10L2015/0635
Inventor 黄露杨毅孙甲松
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products