Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Dynamic identifier representation method, device and system for sequence learning

A dynamic identification and identifier technology, applied in the direction of text database query, unstructured text data retrieval, etc., to achieve the effect of stable convergence and small vocabulary

Active Publication Date: 2019-02-12
PEKING UNIV
View PDF11 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the exact meaning of an identifier often changes with the context in which it appears, so the static embedding method cannot always accurately express the exact meaning of an identifier, especially for those polysemous words and unknown identifiers

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dynamic identifier representation method, device and system for sequence learning
  • Dynamic identifier representation method, device and system for sequence learning
  • Dynamic identifier representation method, device and system for sequence learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0033] figure 1 Flowchart of the dynamic identifier representation method for sequence learning of the present invention. Including the following steps:

[0034] S1. The context vector and the identifier vector are used as input into the correlation sub-module, and a correlation operation is performed to obtain a correlation vector. The correlation operation can use many correlation functions, and the simplest correlation function is a splicing function. Assuming that the dimension of the context vector is n and the dimension of the identifier vector is m, then the dimension of the splicing function is n+m. Another correlation function that can be employed is the concatenation function of the fully connected variant. A well-designed correlation function can achieve better performance.

[0035] S2. Input the correlation vector into the Softmax sub-module, and perform a normalization operation (softmax) to obtain a combination vector. The combination vector is a probability...

Embodiment 2

[0038] According to another aspect of the present invention, the present invention also provides a dynamic identifier representation module for sequence learning. Including the following structures connected in the following order:

[0039] The correlation sub-module uses the context vector and the identifier vector as the input of the correlation sub-module to perform a correlation operation to obtain a correlation vector. The correlation operation can use many correlation functions, and the simplest correlation function is a splicing function. Assuming that the dimension of the context vector is n and the dimension of the identifier vector is m, then the dimension of the splicing function is n+m. Another correlation function that can be employed is the concatenation function of the fully connected variant. A well-designed correlation function can achieve better performance.

[0040] The Softmax sub-module inputs the correlation vector into the Softmax sub-module, performs...

Embodiment 3

[0043] Such as figure 2As shown, according to another aspect of the present invention, the present invention also provides a DTR-RNN model. The DTR-RNN model is a group of RNN variants with DTR modules. The structure and function of the DTR module here are exactly the same as those in the above-mentioned embodiment 2, and will not be repeated here. Ordinary RNN uses the identifier embedding generated by the lookup table as unit input, while the DTR-RNN model of the present invention generates dynamic identifier representation as the input of RNN unit. In this way, the current context information is encoded into the RNN unit, so that the DTR module uses the hidden state as the context vector. Then the output of the DTR module is used as the input of the RNN unit. For each time step, the DTR module takes as input the identifier of the current time step (as an identifier vector) and the hidden state of the previous time step (as a context vector), and outputs a dynamic identi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a dynamic identifier representation method, device and system for sequence learning. The method comprises the following steps of taking context vector and identifier vector as inputs, inputting the context vector and identifier vector into a correlation sub-module, performing correlation operation to obtain a correlation vector; inputting the correlation vector into a Softmax sub-module, performing normalization operation to obtain a combination vector; inputting the combination vector into a memory slot so that the slots in the memory slot are linearly operated according to the combination vector to obtain the dynamic identifier representation. The LSTM model using the DTR module of the present invention can converge faster and more stably, better understand unknownidentifiers, provide more competitive accuracy, and has much smaller vocabulary than the traditional LSTM method.

Description

technical field [0001] The invention relates to the technical field of computer software engineering, in particular to a dynamic identifier representation method, device and system for sequence learning. Background technique [0002] Sequence learning plays an important role in natural language processing (NLP) and program source code analysis. The representation of identifiers is very important for sequence learning. [0003] The representation of discrete identifiers is very important in sequence learning. A common approach is to use a static one-to-one lookup table to generate persistent static embeddings of input identifiers, and use a single embedding to represent all identifiers. However, the exact meaning of an identifier often changes with the context in which it appears, so static embedding methods cannot always accurately express the exact meaning of an identifier, especially for polysemy and unknown identifiers. Contents of the invention [0004] In order to ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/33
Inventor 李戈金芝
Owner PEKING UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com