Neural Coherence Model via Local Sentence Pair Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing coherence models struggle with domain transferability and overfitting, failing to generalize well to unseen categories of text due to their inability to capture local coherence cues and being prone to overfitting on specific domains, which limits their accuracy and adaptability across different domains.
Innovation Solution
A local discriminative neural model that decomposes global coherence into local coherence scores, using a trained neural network to generate scores for adjacent sentence pairs and aggregate them for global coherence, while leveraging generative pre-training for enhanced performance, allowing for effective cross-domain transfer and improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If discriminative models use negative sampling to generate incoherent passages, then they can distinguish coherent from incoherent passages, but the negative sampling space grows exponentially with passage length, limiting achievable accuracy
Solution Approach 1:
The patent segments the global coherence assessment into local coherence scores for adjacent sentence pairs. Instead of evaluating the entire passage at once (which creates exponential negative sampling space), the model breaks down the problem into smaller, manageable sentence pair comparisons. This segmentation reduces the computational burden while maintaining discrimination accuracy.
Solution Approach 2:
The patent transitions from evaluating coherence at the passage level to evaluating it at the sentence pair level. This dimensional shift from global to local assessment changes the problem space from exponential complexity to polynomial complexity, making negative sampling tractable while preserving the ability to distinguish coherent from incoherent text.
2Measurement precision
If generative models maximize likelihood of training text, then they can model coherent text, but they do not observe incoherent text and cannot produce binary classification decisions about coherence
Solution Approach 1:
The patent merges the strengths of both generative and discriminative approaches. It uses generative pre-training to learn coherent text patterns, then combines this with discriminative fine-tuning using contrastive learning on both coherent and incoherent passages. This hybrid approach enables binary classification capability while maintaining adaptability to both text types.
Solution Approach 2:
The patent applies preliminary generative pre-training to initialize the model with knowledge of coherent text structures before performing discriminative fine-tuning. This preliminary action allows the model to start with strong coherence understanding, which is then refined through contrastive learning on labeled coherent and incoherent pairs, enabling effective binary classification.
3Measurement precision
If coherence models are trained on specific domains, then they achieve high accuracy on those domains, but they overfit and fail to generalize to unseen categories of text
Solution Approach 1:
The patent creates a universal coherence model that can function across multiple domains. By training on diverse domains and using domain-agnostic local coherence features (such as rhetorical structure and linguistic patterns that transcend specific topics), the model achieves both domain-specific accuracy and cross-domain generalization capability.
Solution Approach 2:
The patent changes the training parameters to include multiple domains rather than a single domain. By exposing the model to varied textual domains during training and using contrastive learning with domain-diverse negative samples, the model learns transferable coherence patterns that generalize to unseen categories while maintaining accuracy on specific domains.
Data Source
AI summary
Systems and methods of automatically generating a coherence score for text data is provided. The approach includes receiving a plurality of string tokens representing decomposed portions of the target text data object. A trained neural network is provided that has been trained against a plurality of corpuses of training text across a plurality of topics. The string tokens are arranged to extract string tokens representing adjacent sentence pairs of the target text data object. For each adjacent sentence pair, the neural network generates a local coherence score representing a coherence level of the adjacent sentence pair of the target text data object, which are then aggregated for each adjacent sentence pair of the target text data object to generate a global coherence score for the target text data object.