Deep Hierarchical Maxpool Network for Abstract Anaphora Resolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems are limited in resolving abstract anaphoric references in conversational systems, failing to handle complex queries and requiring manual review, as they primarily rely on mention-ranking models and knowledge graphs for co-reference resolution, which are not capable of handling posterior or self-contained queries effectively.
Innovation Solution
A processor-implemented method using a Deep Hierarchical Maxpool Network (DHMN) based model with a Character to Word Encoder (CWE) network to generate sequences of utterances, identify contextual dependencies, and classify incoming sequences into pre-created responses, dynamically optimizing the model to resolve anaphoric references and prevent misclassification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional mention-ranking models and knowledge graphs are used for co-reference resolution, then the system can handle simple pronoun resolution, but it fails to resolve abstract anaphoric references and posterior queries effectively
Solution Approach 1:
The patent replaces conventional mention-ranking models and knowledge graphs with a Deep Hierarchical Maxpool Network (DHMN) based neural network system. This substitution enables the system to automatically learn contextual dependencies and resolve abstract anaphoric references without relying on manual knowledge graph construction or traditional NLP algorithms, thereby improving both adaptability and reliability.
Solution Approach 2:
The patent introduces a hierarchical structure with multiple layers (character level, word level, utterance level, and session level) to process linguistic data. This dimensional expansion allows the system to capture dependencies at different granularities simultaneously, enabling effective resolution of abstract anaphoric references that span multiple utterances in conversational contexts.
2Productivity
If standard NLP algorithms are applied to identify entities and associate them with pronouns, then co-reference resolution works for simple pronouns, but the system cannot handle complex queries requiring multiple fact resolution
Solution Approach 1:
The patent segments the processing of conversational utterances into distinct hierarchical levels: character embeddings, word embeddings, utterance representations, and session-level contextual dependencies. This segmentation allows each layer to specialize in capturing specific types of patterns, enabling the system to handle complex multi-fact queries while maintaining manageable architectural complexity through modular design.
Solution Approach 2:
The DHMN-based model serves multiple functions simultaneously: it performs entity recognition, pronoun resolution, abstract anaphor resolution, and contextual dependency identification within a single unified architecture. This multi-functionality increases productivity by handling diverse query types without requiring separate specialized systems for each task.
3Extent of automation
If manual review of answers is required in memory network systems, then the system can answer user questions, but it cannot operate autonomously and requires continuous human intervention
Solution Approach 1:
The patent implements a self-service system where the DHMN-based model automatically processes incoming utterances, resolves anaphoric references, identifies contextual dependencies, and generates answers without requiring manual review. The system autonomously learns from training data and continuously improves its performance, eliminating the need for continuous human intervention while maintaining high accuracy in resolving complex conversational queries.
4Measurement precision
If conventional systems are used for anaphora resolution, then simple co-reference can be resolved, but the training process is slow and performance on complex queries is poor
Solution Approach 1:
The patent employs pre-trained character embeddings and word embeddings as input representations for the DHMN model. These pre-trained embeddings capture fundamental linguistic patterns from large corpora beforehand, allowing the model to focus training resources on learning task-specific contextual dependencies rather than learning basic language structures from scratch, thereby reducing training time while maintaining high precision in anaphora resolution.
Data Source
AI summary
Conversational systems are required to be capable of handling more sophisticated interactions than providing factual answers only. Such interactions are handled by resolving abstract anaphoric references in conversational systems which includes antecedent fact references and posterior fact references. The present disclosure resolves abstract anaphoric references in conversational systems using hierarchically stacked neural networks. In the present disclosure, a deep hierarchical maxpool network based model is used to obtain a representation of each utterance received from users and a representation of one or more generated sequences of utterances. The obtained representations are further used to identify contextual dependencies with in the one or more generated sequences which helps in resolving abstract anaphoric references in conversational systems. Further, a response for an incoming sequence of utterances is retrieved based on classification of incoming sequence of utterances into one or more pre-created responses. The proposed model takes lesser time to retrain.