Multi-View Query Generation Model for Faithful Dialogue Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current query generation models face challenges with faithfulness and data hunger issues, often missing important concepts in dialogue contexts due to high variations in annotated gold queries and requiring large amounts of training data.
Innovation Solution
A multi-view query generation model framework that includes static and dynamic views to highlight main content, using static views to mitigate annotation inconsistencies and dynamic views to address model distribution inconsistencies, with pre-training and fine-tuning stages to improve query accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If standard cross-entropy loss is used for training query generation models, then the models can be trained with available annotated data, but the models face faithfulness issues and require large amounts of training data due to high variations in annotated gold queries
Solution Approach 1:
The patent segments the query generation task into multiple views (static view and dynamic view). The static view captures common words between dialogue history and gold queries, while the dynamic view handles task-specific variations. This segmentation allows the model to learn from annotated data more effectively without requiring excessive training samples, as each view focuses on specific aspects of query generation.
Solution Approach 2:
The patent introduces an intermediary masking mechanism that hides task-specific words in the static view while preserving common words. This intermediary approach allows the model to first learn from reliable common patterns and then adapt to specific task variations, improving faithfulness while reducing the amount of training data needed compared to directly learning from highly variable annotated queries.
2Manufacturing precision
If models are trained to perfectly generate gold queries, then query accuracy may improve, but the models miss important concepts from dialogue contexts due to high variations in annotated gold queries
Solution Approach 1:
The patent divides the query generation process into two distinct views: static view that learns from common words across multiple annotated queries, and dynamic view that adapts to task-specific variations. This segmentation ensures that important concepts from dialogue contexts captured in the static view are not lost when generating final queries, while still achieving high accuracy through dynamic adaptation.
Solution Approach 2:
The patent applies different quality requirements to different parts of the query generation process. The static view focuses on accurately capturing common important concepts from dialogue contexts, while the dynamic view focuses on adapting to specific task requirements. This local quality approach ensures that both conceptual accuracy and task-specific precision are maintained.
Data Source
AI summary
A method of training a model for query generation, the method performed by at least one processor and including receiving a training instance query corresponding to a dialogue history. The method further including generating a first static view of the model based on a number of common words between the training instance query and the dialogue history. The method further including generating a second static view of the model based on one or more tokens not covered by the dialogue history, the one or more tokens corresponding to one or more query words. The method further including generating a dynamic view of the model based on a score operation that compares a candidate query generated from the model with a target query. The method further including training the model based at least on the first static view, the second static view, and the dynamic view.