Hierarchical Context Tagging for Multi-Span Utterance Rewriting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing utterance rewriting models struggle with low phrase coverage and grammaticality issues due to the inability to effectively handle coreferences, anaphora, and ellipsis in multi-turn dialogues, particularly in languages like English, leading to performance degradation.
Innovation Solution
A hierarchical context tagger (HCT) that tags source tokens with token-level edit actions and slotted rules, allowing for the insertion of out-of-context tokens and multiple spans, using a multi-span tagger (MST) to predict context spans and a syntax-guided method to extract multi-span labels, thereby improving dialogue rewriting performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If sequence tagging is used to predict edit actions for each source token, then the learning problem is simplified and search space is reduced, but target phrase coverage becomes low because out-of-context tokens and non-contiguous spans cannot be inserted
Solution Approach 1:
The patent segments the rewriting task into two independent components: (1) sequence tagging to predict edit actions and context span indices for each source token, and (2) a separate context span inserter that retrieves and inserts spans from the context based on predicted indices. This segmentation allows the tagger to focus on structural decisions while the inserter handles actual span insertion, enabling multi-span and non-contiguous insertions without increasing tagger complexity.
Solution Approach 2:
The patent introduces context span indices as an intermediary representation between the sequence tagging output and the final rewritten utterance. Instead of directly inserting spans, the tagger predicts indices that point to context spans, which are then retrieved and inserted by a separate module. This intermediary approach decouples the tagging task from the insertion task, allowing flexible span insertion while keeping the tagging problem simple.
2Loss of information
If word-level edit matrix is predicted between context-source pairs to add arbitrary non-contiguous context phrases, then more target phrases can be covered, but the number of tags increases O(m) times and ungrammatical outputs become easier to produce
Solution Approach 1:
The patent segments the edit operations into token-level decisions (keep, delete, or insert context span before each source token) rather than predicting a full word-level edit matrix. This segmentation reduces the tag vocabulary from O(m) possible context tokens to just a few edit actions plus context span indices, dramatically reducing the number of tags while still enabling arbitrary non-contiguous span insertion through the predicted indices.
3Adaptability or versatility
If standard sequence-to-sequence models are used for utterance rewriting, then the task can be performed with existing architectures, but models struggle with coreference, anaphora, and ellipsis in longer form conversations
Solution Approach 1:
The patent applies preliminary action by pre-processing the dialogue context to identify and index all possible context spans before the rewriting task. This pre-computed context span database allows the sequence tagger to reference relevant context segments efficiently during rewriting, improving handling of coreference and anaphora without requiring the model to generate context spans from scratch during the rewriting process.
Data Source
AI summary
Hierarchical context tagging for utterance rewriting comprising computer code for obtaining source tokens and context tokens, encoding the source tokens and the context tokens to generate source contextualized embeddings and context contextualized embeddings, tagging the source tokens with tags indicating a keep or delete action for each source token of the source tokens, selecting a rule to insert before the each source token, wherein the rule contains a sequence of one or more slots, and generating spans from the context tokens, wherein each span corresponds to one of the one or more slots of the selected rule.