Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Knowledge base question-answering method fusing multi-head-attention mechanism and relative position encoding

A technology of relative position and attention, applied in reasoning methods, neural learning methods, biological neural network models, etc., can solve problems such as limiting the parallel computing capabilities of models, and achieve the goal of enhancing the ability of relative position information of words and improving parallel computing capabilities Effect

Active Publication Date: 2021-11-26
CHONGQING UNIV OF POSTS & TELECOMM
View PDF9 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, because of the sequential calculation characteristics of LSTM and BiLSTM, the calculation of each time slice t depends on the calculation results at time t-1, which limits the parallel computing capability of the model to a certain extent.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Knowledge base question-answering method fusing multi-head-attention mechanism and relative position encoding
  • Knowledge base question-answering method fusing multi-head-attention mechanism and relative position encoding
  • Knowledge base question-answering method fusing multi-head-attention mechanism and relative position encoding

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0050] Embodiments of the present invention are described below through specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes, and various modifications or changes can be made to the details in this specification based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments are only schematically illustrating the basic concept of the present invention, and the following embodiments and the features in the embodiments can be combined with each other in the case of no conflict.

[0051] Wherein, the accompanying drawings are for illustrative purposes only, and represent only schematic diagrams, rather than physical drawings, and should...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a knowledge base question-answering method fusing a multi-head-attention mechanism and relative position encoding, belonging to the field of natural language processing. According to the invention, a Transformer encoder is introduced to replace BiLSTM to encode questions; meanwhile, due to the structure problem of Transformer, the Transformer encoder is insufficient in capability of acquiring information of relative position words in sentences; and the relative position encoding thought in Transform-XL is adopted in the invention, an absolute position encoding formula used in Transformer is rewritten, relative position encoding is used for replacing absolute position encoding, and thus, the situation that the capacity of acquiring the information of relative position words is insufficient is made up for.

Description

technical field [0001] The invention belongs to the field of natural language processing, and relates to a knowledge base question-answering method integrating multi-head attention mechanism and relative position coding. Background technique [0002] Natural question answering based on knowledge base is a classic task in the field of natural language processing. Given a natural language question, the question is analyzed, and the knowledge base information is used for query and reasoning to obtain the answer. Knowledge base question answering is a research hotspot in the fields of artificial intelligence, natural language processing, and information retrieval. It can answer users' natural language questions in concise and precise language, making the use of software products easier and more efficient. Applying the natural language question answering system to the field of Internet customer service can greatly reduce the manpower required by the enterprise, thereby reducing t...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/332G06F40/211G06F40/35G06N3/04G06N3/08G06N5/04
CPCG06F16/3329G06F40/211G06F40/35G06N3/049G06N3/08G06N5/04G06N3/044Y02D10/00
Inventor 甘玲肖阳
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com