Bert model-based intention recognition and slot value filling combined prediction method

A prediction method and technology of intent, applied in character and pattern recognition, neural learning methods, biological neural network models, etc., can solve the problems affecting the quality of the final dialogue system, low accuracy of slot value filling, etc., to reduce task error prediction, Avoid the effect of superimposed error rate and high prediction accuracy

Active Publication Date: 2021-05-14
CHONGQING UNIV OF POSTS & TELECOMM
View PDF10 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, the existing joint prediction method based on deep learning ignores the internal relationship between intent recognition and slot value filling, or simply splices the intent representation vector and slot value sequence vector to express the internal relationship between the two, resulting in accurate slot value filling. The rate is not high, which affects the quality of the final dialogue system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Bert model-based intention recognition and slot value filling combined prediction method
  • Bert model-based intention recognition and slot value filling combined prediction method
  • Bert model-based intention recognition and slot value filling combined prediction method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0053] Dataset and model input:

[0054] The datasets used are the dataset from the Snips personal voice assistant and the Atis dataset from the flight reservation system. Among them, the training set, test set and verification set data in the Snips dataset are 13084, 700 and 700 sentences respectively, and the training set, test set and verification set data in the Atis dataset are 4478, 500 and 893 sentences respectively.

[0055] The input representation of the model is a word embedding (E w ), position embedding (E p ) and segment embedding (E s ) cascade. Use the WordPiece model to process the input sentence, insert the [CLS] tag before the sentence as a classification token, insert [SEP] at the end of the sentence as the ending token, and generate a word embedding (E w =(E CLS ,E w1 ......,E wT ,E Seq )). Since we are targeting intent classification and slot value identification for a single sentence, the segment embeddings for each sentence are all 0s. The pos...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to the technical field of intelligent questions and answers, in particular to a Bert model-based intention recognition and slot value filling joint prediction method, which comprises the following steps of: inputting a target text to obtain a word vector, a segment vector and a position vector of the target text, splicing the word vector, the segment vector and the position vector as an input vector of a Bert model, and performing prediction on the input vector of the Bert model; inputting a trained Bert model, outputting an intention representation vector and a slot value sequence representation vector by the trained Bert model, performing weight calculation on the intention representation vector and the slot value sequence representation vector in a Gate layer to calculate a joint action factor, acting the joint action factor on the slot value sequence representation vector, and finally outputting predicted intention classification and a slot value sequence. According to the method, a Gate mechanism is used on a Bert layer, the internal relation between intention recognition and slot value filling is fully utilized, and the task error prediction rate is reduced.

Description

technical field [0001] The invention relates to the technical field of intelligent question answering, in particular to a joint prediction method for intent recognition and slot value filling based on Bert model. Background technique [0002] With the leap of artificial intelligence technology, the interaction between humans and machines has become more and more frequent, and the way humans convey instructions to machines has also shifted from button operation to voice interaction. Machines can help people complete specific tasks through various modes of interaction. In order to solve the communication problem of human-computer dialogue, intelligent dialogue system has become one of the core technologies in the field of artificial intelligence, among which task-based dialogue system is designed to complete specific tasks, such as air ticket reservation, restaurant reservation and other applications. The implementation process of a task-based dialogue system mainly includes f...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/332G06F16/35G06F40/211G06F40/279G06K9/62G06N3/04G06N3/08
CPCG06F16/3329G06F16/35G06F40/279G06F40/211G06N3/08G06N3/047G06N3/045G06F18/2415
Inventor 张璞明欢欢朱洪倩
Owner CHONGQING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap