Hierarchical Context Tagging for Multi-Span Utterance Rewriting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing dialogue rewriting models struggle with low phrase coverage and grammatical errors due to the inability to effectively handle coreference, anaphora, and ellipsis in multi-turn conversations, particularly when adding out-of-context tokens or non-contiguous spans.
Innovation Solution
A hierarchical context tagger (HCT) that tags source tokens with edit actions and slotted rules, allowing for the insertion of multiple context spans, and uses a syntax-guided method to predict multi-span labels, reducing the search space and improving coverage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single-span tagger is used to predict edit actions, then the learning problem is simplified with a small vocabulary, but the phrase coverage is low because out-of-context tokens and non-contiguous spans cannot be inserted
Solution Approach 1:
The patent segments the rewriting task into two independent stages: (1) a sequence tagger that predicts edit actions and slotted rules for each source token, and (2) a span extractor that retrieves context spans to fill the slots. This segmentation allows the tagger to maintain simplicity while the span extractor handles complex insertions of out-of-context tokens and non-contiguous spans, resolving the contradiction between simplicity and phrase coverage.
Solution Approach 2:
The patent introduces slotted rules as an intermediary mechanism between the sequence tagger and the span extractor. The slotted rules serve as a mediator that bridges the simple tagger output with the complex span insertion requirements, enabling the system to achieve high phrase coverage without requiring the tagger itself to be complex.
2Adaptability or versatility
If an edit matrix approach is used to predict word-level edits, then arbitrary non-contiguous context phrases can be added before each source token, but the number of tags increases to O(m) times more than a sequence for m context tokens
Solution Approach 1:
The patent extracts the span extraction function from the tagging process itself. Instead of having the tagger directly output complex edit matrices with O(m) tags, the system extracts spans from the context using a separate span extractor module. This extraction approach maintains high phrase coverage for arbitrary non-contiguous context phrases while avoiding the explosion in the number of tags.
Solution Approach 2:
The patent segments the rewriting process into tagger and span extractor components. The tagger produces a simplified sequence of edit actions and slotted rules, while the span extractor handles the complex retrieval of context spans. This segmentation reduces the tagging complexity from O(m) to a linear sequence of tags while maintaining the ability to insert arbitrary non-contiguous phrases.
3Adaptability or versatility
If a seq-to-seq approach with LSTM decoder is used, then the model can generate rewritten utterances, but the search space becomes large again introducing the same large search space issue that sequence tagging was designed to avoid
Solution Approach 1:
The patent performs preliminary action by having the sequence tagger predict edit actions and slotted rules for all source tokens before the span extraction step. This preliminary tagging creates a structured plan that guides the subsequent span extraction, significantly reducing the effective search space compared to a standard seq-to-seq approach where the model would need to search through all possible span combinations during generation.
Data Source
AI summary
Hierarchical context tagging for utterance rewriting comprising computer code for obtaining source tokens and context tokens, encoding the source tokens and the context tokens to generate source contextualized embeddings and context contextualized embeddings, tagging the source tokens with tags indicating a keep or delete action for each source token of the source tokens, selecting a rule to insert before the each source token, wherein the rule contains a sequence of one or more slots, and generating spans from the context tokens, wherein each span corresponds to one of the one or more slots of the selected rule.