Length-Adaptive Neural Network for Question Answering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing question answering systems face challenges in capturing long-term dependencies in sentence interactions due to the neglect of sentence length, leading to low-quality distributed representations and loss of global information.
Innovation Solution
The proposed method employs a length-adaptive neural network structure that uses pre-trained word embeddings, fine-tuned during network training, and switches between BiLSTM and Transformer-based feature extractors based on sentence length to generate contextual representations, followed by attention pooling for correlation measurement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a same feature extractor is used for all sentences regardless of length, then the model structure is simple, but long-term dependency problem occurs and global information is lost
Solution Approach 1:
The patent applies dynamics by making the feature extractor adaptive to sentence length. The system dynamically switches between BiLSTM and Transformer-based extractors based on sentence length thresholds, allowing the model to adjust its processing strategy according to input characteristics. This resolves the contradiction by enabling complex processing for long sentences while maintaining simplicity for short ones.
Solution Approach 2:
The patent changes the parameter of feature extractor selection based on sentence length. By introducing a length threshold parameter, the system selects different extractors (BiLSTM for short sentences, Transformer for long sentences) according to the sentence length parameter, thereby optimizing long-term dependency capture without unnecessarily increasing complexity for all cases.
2Productivity
If BiLSTM is used for all sentences, then the model is simple and efficient for short sentences, but long-term dependency problem occurs in long sentences
Solution Approach 1:
The patent applies local quality by tailoring the feature extractor to the specific characteristics of each sentence based on its length. Short sentences use BiLSTM for efficient processing, while long sentences use Transformer-based extractors for better long-term dependency capture. This localized optimization resolves the contradiction by matching the right tool to the right problem size.
Solution Approach 2:
The system dynamically adapts the feature extractor based on sentence length, switching from BiLSTM to Transformer for longer sentences. This dynamic adjustment maintains high processing efficiency for short sentences while ensuring reliable long-term dependency capture for long sentences, resolving the productivity-reliability contradiction.
3Reliability
If Transformer-based extractor is used for all sentences, then long-term dependency problem is solved, but processing time increases for short sentences
Solution Approach 1:
The patent applies partial action by using the more powerful Transformer-based extractor only when necessary (for long sentences) rather than for all sentences. This selective application ensures reliable long-term dependency capture for long sentences while avoiding the time cost of using Transformers for short sentences, thus resolving the reliability-time contradiction.
Solution Approach 2:
The system changes the extractor selection parameter based on sentence length threshold. By introducing a length parameter that triggers different extractors, the system optimizes processing time by using lighter BiLSTM for short sentences and heavier Transformer for long sentences, balancing reliability and time consumption.
4Device complexity
If sentence length is not considered in feature extraction, then the processing is simplified, but distributed representation quality deteriorates
Solution Approach 1:
The patent applies local quality by adjusting the feature extraction process according to the local characteristic of sentence length. The system selects different extractors based on whether sentences are short or long, thereby improving distributed representation quality for long sentences without unnecessarily complicating processing for short sentences.
Solution Approach 2:
The system dynamically adapts the feature extraction complexity based on sentence length. By making the extractor selection dynamic rather than fixed, the system improves measurement precision for long sentences while maintaining processing simplicity for short sentences, resolving the quality-complexity contradiction.
Data Source
AI summary
Disclosed are an answer selection method for a question answering system. With the method and system, a sentence distributed representation of a question and an answer is generated by adopting a hierarchical length-adaptive neural network structure with a purpose of extracting high-quality sentence features by using different neural feature extractors according to a length of an input sentence. First, one word distributed representation is generated for each word in the input sentence by connecting one fixed word embedding and one finely-adjusted word embedding. Next, a BiLSTM-based feature extractor is used for a short sentence and a Transformer-based feature extractor is used for a long sentence. Finally, a sentence vector representation for measuring a correlation between a question and a candidate answer is generated by using an attention pooling layer considering an interaction between a question sentence and an answer sentence.