Unlock instant, AI-driven research and patent intelligence for your innovation.

A dialog generation method based on transformer architecture

A background knowledge and decoder technology, applied in the field of dialogue generation based on the transformer architecture, can solve problems such as small amount of information, meaninglessness, and factual inconsistency

Active Publication Date: 2021-09-03
WUHAN UNIV
View PDF14 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

Solve the problem that dialogue generation tends to produce meaningless or boring answers, the amount of information is small, and it does not match the facts

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A dialog generation method based on transformer architecture
  • A dialog generation method based on transformer architecture
  • A dialog generation method based on transformer architecture

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

[0068] Discuss the specific embodiment of the present invention below in conjunction with accompanying drawing as, a kind of dialogue generation method based on Transformer architecture, this method comprises the following steps:

[0069] Step 1: Map the words in the dialogue and background knowledge into word embedding, position embedding, segmentation embedding, and further obtain word vectorization representation;

[0070] Preferably, the dialogue above in ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention proposes a dialog generation method based on a transformer architecture, which is used to solve the problem of dialog generation based on background knowledge, and belongs to the field of natural language processing. The main steps are to input the dialogue context and background knowledge into the fine-tuned text pre-training model, and obtain corresponding vectorized representations respectively. Do the reading comprehension task, and obtain the probability distribution of the background knowledge in line with the current context through the trained reading comprehension model. For the text generation task, the comprehensive semantic vector is obtained through the multi-head attention mechanism based on the knowledge fragment with the highest probability and the vectorized representation of the dialogue. Replies are generated based on synthetic semantic vectors combined with a replication mechanism. Use this method to generate informative, natural, and contextual responses.

Description

technical field [0001] The invention belongs to the field of natural language processing, and in particular relates to a dialog generation method based on a transformer architecture. Background technique [0002] The purpose of artificial intelligence is to make robots more and more like people, so being able to communicate is one of the necessary skills for robots. With the development of deep learning, the sequence-to-sequence (Seq2Seq) model is widely used in conversation modeling. In order to solve the problem that the original model tends to produce meaningless or boring answers, the amount of information is small, and does not match the facts, etc., in the dialogue system Introduced knowledge information. There are two main forms of knowledge: structured knowledge such as knowledge graphs, and unstructured knowledge such as documents. Unstructured knowledge is obviously more suitable for open domain dialogue systems due to its natural properties. There are currently...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/33G06F16/332G06F40/30
CPCG06F16/3329G06F16/3344
Inventor 蔡贤涛袁一鸣
Owner WUHAN UNIV