Artificial neural network with side input for language modelling and prediction

An artificial neural network, neural network technology, applied in the field of artificial neural network with secondary input for language modeling and prediction, can solve problems such as dependence on statistical correlation

Pending Publication Date: 2019-03-15
MICROSOFT TECH LICENSING LLC
View PDF5 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0002] One of the often criticized shortcomings of n-gram based predictive language

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Artificial neural network with side input for language modelling and prediction
  • Artificial neural network with side input for language modelling and prediction
  • Artificial neural network with side input for language modelling and prediction

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] figure 1 A simple artificial neural network 100 depicts the state of the prior art. In essence, artificial neural network such as artificial neural network 100 is a mathematical functional chain tissue in the direction dependent layer, said direction dependent layer, such as input layer 101, hiding layer 102, and output layer 103, each layer Includes multiple units or nodes, 110-131. Artificial neural network 100 is called a "feedforward neural network", which is because the output of each layer 101-103 is used as input (or, in the case of output layer 103, it is artificial neural network 100). Output) and there is no backward step or loop. It should be understood that figure 1 The number of units 110-131 depicted in the present is exemplary, and the general artificial neural network includes more units in each layer 101-103.

[0041] In the operation of artificial neural network 100, an input is provided at the output layer 101. This generally involves mapping the real wo...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention relates to an improved artificial neural network for predicting one or more next items in a sequence of items based on an input sequence item. The artificial neural network is implemented on an electronic device comprising a processor, and at least one input interface configured to receive one or more input sequence items, wherein the processor is configured to implement theartificial neural network and generate one or more predicted next items in a sequence of items using the artificial neural network by providing an input sequence item received at the at least one input interface and a side input as inputs to the artificial neural network, wherein the side input is configured to maintain a record of input sequence items received at the input interface.

Description

Background technique [0001] Modern mobile electronic devices such as mobile phones and tablets typically receive user input typing via soft keyboard, which includes a variety of additional functions that are transmitted simply receiving keyboard inputs. One of these additional functions is a function of predicting the user will be entered via the next word via the keyboard in the case where one or more previous words are known. This prediction is typically generated using a predictive language model based on N-element (n), for example, in European patents in patent number 2414915. [0002] One of the disadvantages that are often criticized based on N-element predictive language models is that they rely on only a few previous words statistical relevance. In contrast, artificial neural networks, and especially circulating neural network language models have been shown to be better implemented in language prediction ("circular neural network based on language model", MIKOLOV, etc. Pe...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N3/04G06N3/08G10L15/16G06F3/023
CPCG06N3/084G06F3/0237G06F40/274G06N3/044G06N3/04G06N3/08
Inventor J·伊索-西皮莱M·J·威尔森
Owner MICROSOFT TECH LICENSING LLC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap