Phrase Vector Learning via Lattice Candidate Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for vectorizing phrases face issues where similar phrases are indistinguishable and phrases not present in the corpus cannot be vectorized, leading to limitations in estimating nearby word or phrase candidates.
Innovation Solution
A phrase vector learning device and method that constructs a lattice structure from morphological and dependency analysis results, using a neural network to estimate nearby word or phrase candidates, and subsequently learns vector representations for these candidates using a neural network B with the acquired vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If average of word vectors is used to vectorize phrases, then computation is simple, but phrases with similar words become indistinguishable
Solution Approach 1:
The patent segments the phrase vectorization problem into multiple components: constructing a lattice structure from morphological and dependency analysis, training a neural network to estimate nearby word/phrase candidates, and using these candidates to generate refined phrase vectors. This segmentation allows the system to capture contextual relationships while maintaining computational feasibility through staged processing.
Solution Approach 2:
The patent introduces an intermediary neural network that estimates nearby word or phrase candidates as a mediator between the input phrase and the final vector representation. This intermediary layer captures contextual nuances and relationships that simple averaging misses, improving phrase distinction accuracy while keeping the overall system computationally manageable.
2Reliability
If phrase appears in corpus then it can be vectorized, but phrases not in corpus cannot be vectorized
Solution Approach 1:
The patent performs preliminary action by pre-training a neural network on the corpus to learn relationships between words and phrases. This pre-trained model can then estimate nearby word or phrase candidates for any input phrase, even those not present in the corpus, enabling zero-shot vectorization while maintaining reliability for phrases that do appear in the training data.
Solution Approach 2:
The patent creates a universal phrase vectorization system that handles both seen and unseen phrases through a single neural network model. The model learns generalizable patterns from the corpus and applies them to estimate candidates for any phrase, making the system adaptable to out-of-corpus phrases while maintaining reliable vectorization for in-corpus phrases through the unified approach.
3Measurement precision
If lattice structure with multiple candidates is constructed, then nearby word/phrase estimation improves, but computation complexity increases
Solution Approach 1:
The patent segments the complex lattice construction into manageable components: morphological analysis, dependency analysis, and candidate estimation. Each component processes specific aspects of the language structure independently, allowing the system to handle multiple candidates systematically without overwhelming computational complexity.
Solution Approach 2:
The patent applies partial action by focusing the neural network on estimating only the nearby word or phrase candidates that are most relevant to the input phrase, rather than considering all possible combinations. This selective approach maintains high estimation accuracy while reducing computational complexity by concentrating resources on the most promising candidates.
Data Source
AI summary
An appropriate vector of any phrase can be generated. A lattice construction unit 212 constructs a lattice structure formed by links binding adjacent word or phrase candidates based on a morphological analysis result and a dependency analysis result of input text. A first learning unit 213 performs learning of a neural network A for estimating nearby word or phrase candidates from word or phrase candidates based on the lattice structure. A vector generation unit 214 acquires a vector of each of the word or phrase candidates from the neural network A and sets the vector as learning data. A second learning unit performs learning of a neural network B for vectorizing the word or phrase candidates based on the learning data.