Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Character based neural network training method and device

A neural network and pre-training technology, applied in the field of neural network training, can solve problems such as low training efficiency

Active Publication Date: 2016-01-27
TSINGHUA UNIV +1
View PDF4 Cites 16 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The application provides a word-based neural network training method and device to solve the problem of low training efficiency in the word-based neural network language model

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Character based neural network training method and device
  • Character based neural network training method and device
  • Character based neural network training method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0054] In order to make the above objects, features and advantages of the present application more obvious and comprehensible, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0055] refer to figure 1 , showing a method for training a word-based neural network in Embodiment 1 of the present application, including:

[0056] Step 101: Obtain the word vector of each word by word for each training sentence.

[0057] For example, if the training sentence is "computer mobile phone", the training sentence after the word recognition of the training sentence is: computer / brain / hand / machine / , and then obtain the word vector of each word.

[0058] Step 102: Input the word vector as a parameter of the first neural network into the first neural network for pre-training, and obtain a pre-training result, wherein the result includes the upper-text feature vector of the word vector and the l...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a character based neural network training method and device. The method comprises that the vector of each character in each training sentence is obtained; the character vector serves as a parameter of a first neural network and input to the first neural network to implement pre-training, and a pre-training result, which comprises characteristic vectors of the preceding and following texts of the character vector, is obtained; and the characteristic vectors of the preceding and following texts of the character vector serves as parameters of a second neural network, and input to the second neural network to train the second neural network. The character based neural network training method and device can be used to solve the problem that the training efficiency is low in a word based neural network language model.

Description

technical field [0001] The present application relates to the field of natural language processing, in particular to a word-based neural network training method and device. Background technique [0002] In fields such as natural language processing and speech recognition, language models are used to describe the collocation relationship of words within a language. A high-quality language model is of great significance to continuous speech recognition and machine translation. [0003] The current mainstream language model is a probability-based statistical language model (n-gram). The significant defect of this statistical language model is that it is difficult to obtain effective probability statistics for new words or low-frequency words. Based on this, various smoothing algorithms have been invented, such as discount, back-off, interpolation and so on. The above method has improved the performance of n-gram on low-frequency words to a certain extent, but due to the defec...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06N3/02
Inventor 王东游世学刘荣乔亚飞
Owner TSINGHUA UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com