Unlock instant, AI-driven research and patent intelligence for your innovation.

Dialogue generation method based on transformer architecture

A background knowledge, the above technology, applied in the field of dialogue generation based on the transformer architecture, can solve the problems of small amount of information, meaningless, boring and so on

Active Publication Date: 2020-06-12
WUHAN UNIV
View PDF14 Cites 11 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Solve the problem that dialogue generation tends to produce meaningless or boring answers, the amount of information is small, and it does not match the facts

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dialogue generation method based on transformer architecture
  • Dialogue generation method based on transformer architecture
  • Dialogue generation method based on transformer architecture

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0064] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0065] Discuss the specific embodiment of the present invention below in conjunction with accompanying drawing as, a kind of dialogue generation method based on Transformer architecture, this method comprises the following steps:

[0066] Step 1: Map the words in the dialogue and background knowledge into word embedding, position embedding, segmentation embedding, and further obtain word vectorization representation;

[0067] Preferably, the dial...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a dialogue generation method based on a transformer architecture, is used for solving the dialogue generation problem based on background knowledge, and belongs to the field ofnatural language processing. The method mainly comprises the following steps of: inputting dialogue preceding texts and background knowledge into a finely tuned text pre-training model to respectivelyobtain corresponding vectorized representations; and performing a reading understanding task, and obtaining probability distribution that background knowledge conforms to the current context throughthe trained reading understanding model. And performing a text generation task, and obtaining a comprehensive semantic vector through a multi-head attention mechanism according to the knowledge segment with the highest probability and the vectorized representation of the dialogue preceding text. And generating a reply according to the comprehensive semantic vector in combination with a replicationmechanism. By using the method, a natural reply which has information content and conforms to the current context can be generated.

Description

technical field [0001] The invention belongs to the field of natural language processing, and in particular relates to a dialog generation method based on a transformer architecture. Background technique [0002] The purpose of artificial intelligence is to make robots more and more like people, so being able to communicate is one of the necessary skills for robots. With the development of deep learning, the sequence-to-sequence (Seq2Seq) model is widely used in conversation modeling. In order to solve the problem that the original model tends to produce meaningless or boring answers, the amount of information is small, and does not match the facts, etc., in the dialogue system Introduced knowledge information. There are two main forms of knowledge: structured knowledge such as knowledge graphs, and unstructured knowledge such as documents. Unstructured knowledge is obviously more suitable for open domain dialogue systems due to its natural properties. There are currently...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/33G06F16/332G06F40/30
CPCG06F16/3329G06F16/3344
Inventor 蔡贤涛袁一鸣
Owner WUHAN UNIV