Supercharge Your Innovation With Domain-Expert AI Agents!

Statement generation method for dialogue system evaluation

A dialogue system and sentence technology, which is applied in special data processing applications, biological neural network models, digital data information retrieval, etc. The effect of load reduction and stable transmission

Active Publication Date: 2020-10-02
NAT UNIV OF DEFENSE TECH
View PDF8 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, existing user simulators only focus on the dialogue strategy part, interacting with dialogue systems through structured formal languages ​​or template-based statements
Such a user simulator cannot interact with the dialog system in natural language
Therefore it is not enough to evaluate a complete task-oriented dialogue system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Statement generation method for dialogue system evaluation
  • Statement generation method for dialogue system evaluation
  • Statement generation method for dialogue system evaluation

Examples

Experimental program
Comparison scheme
Effect test

Embodiment approach

[0064] As an implementation, all the sentences are connected and then sent to the bidirectional gated recurrent unit (BidirectionGated Recurrent Unit, BiGRU) to obtain the hidden representation of each word in the sentence, including:

[0065] Will D t-1 All terms in the statement are concatenated as X = {x 1 , x 2 , x 3 , … , x I}, where t represents the tth round of the dialogue, D t-1 Represents the dialogue sentences from the first round to the t-1 round, I is the total number of words, and X is sent to the bidirectional gated recurrent unit to obtain the hidden representation H of each word enc =[h 1 enc , h 2 enc , … , h I enc ]

[0066] .

[0067] In model training on corpora, the context comes from the conversation history. Such a context is dynamically generated during the interaction.

[0068] As an implementation manner, tracking the selected sub-goal and judging whether it has completed delivery includes:

[0069] Calculate the transmission probabi...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a statement generation method for dialogue system evaluation. The method comprises steps of coding context in a conversation history; obtaining a hidden representation of eachword, selecting an appropriate sub-target through hidden representation of each word; tracking selected sub-targets, calculating sub-target tracking results, and tracking results based on the trackingresults, performing the next round of target selection by using a standard memory network; selecting according to the target, reusing sentences in the dialogue history to generate sentences; generating proper user statements dynamically according to replies of a dialogue system by reusing keywords and phrases in the dialogue simulating process, carrying out interaction with the dialogue system ina natural language, thus the load of a user simulator during sentence generation is reduced, and a user target can be transmitted more stably.

Description

technical field [0001] The invention relates to the technical field of dialog system evaluation, in particular to a sentence generation method for dialog system evaluation. Background technique [0002] Task-oriented dialogue systems have aroused widespread interest among researchers due to their great potential in practical applications. These systems help users complete specific tasks through natural language interaction, such as finding a restaurant or booking a flight. However, current evaluations of task-oriented dialogue systems are based on static corpora. However, during the interaction process, the dialogue often deviates from the dialogue process in the corpus, making the evaluation based on the static corpus unsuitable for measuring the performance of dialogue systems in dynamic scenarios. This bias is a critical bottleneck in applying offline-developed dialogue systems to real-world scenarios. [0003] The ideal evaluation method is to hire real users to inter...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F11/36G06N3/04G06F16/332
CPCG06F11/3688G06F16/3329G06N3/044G06N3/045
Inventor 李新梦杨妹许凯彭勇焦鹏秦龙张琪谢旭尹全军
Owner NAT UNIV OF DEFENSE TECH
Features
  • R&D
  • Intellectual Property
  • Life Sciences
  • Materials
  • Tech Scout
Why Patsnap Eureka
  • Unparalleled Data Quality
  • Higher Quality Content
  • 60% Fewer Hallucinations
Social media
Patsnap Eureka Blog
Learn More

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2025 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com