Natural Language Learning System Using Syntactic Word Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional natural language processing techniques are hindered by ontological categories and the reliance on 'bags of words,' which limit the accuracy of predicting word similarity, especially for ambiguous terms.
Innovation Solution
A natural language learning system that analyzes sentence corpora to identify internal structures, creates new words combining syntactic features and word embeddings, clusters these new words, filters them, and performs word embedding processing using Recurrent Neural Networks (RNN) to encode syntactic slots and concepts, improving similarity prediction accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If conventional techniques use ontological categories to process natural language, then the processing framework is structured and organized, but the accuracy of word similarity prediction deteriorates
Solution Approach 1:
The patent extracts and removes ontological categories from the processing framework, transitioning from structured category-based processing to unstructured word embedding processing. This extraction eliminates the harmful constraint that ontological categories imposed on word similarity prediction accuracy.
Solution Approach 2:
The patent changes the fundamental parameter of representation from discrete ontological categories to continuous vector embeddings. By transforming words into dense vector representations that capture semantic relationships, the system achieves higher prediction accuracy while maintaining organizational structure through the vector space itself.
2Ease of manufacture
If conventional techniques rely on bags of words to predict meaning, then the implementation is simple and straightforward, but the prediction of ambiguous terms deteriorates
Solution Approach 1:
The patent substitutes the mechanical bag-of-words approach with a neural network-based word embedding system. This replacement uses distributed representations that capture contextual relationships, enabling accurate prediction of ambiguous terms while maintaining computational efficiency through learned vector operations.
3Measurement precision
If Recurrent Neural Networks are used for word similarity prediction, then the model captures sequential relationships, but the processing complexity and computational requirements increase
Solution Approach 1:
The patent extracts and removes the recurrent neural network component from the system, replacing it with simpler word embedding processing. This extraction eliminates the computational complexity and sequential processing requirements while maintaining the ability to capture semantic relationships through static vector representations.
Data Source
AI summary
A natural language learning method, system, and non-transitory computer readable medium include analyzing a corpus of sentences stored in a database to identify an internal structure of words in the corpus of sentences, creating a plurality of new words that are a combination of the internal structure of a word of the words in the corpus of sentences and the word, clustering the plurality of new words created by the creating that match into a plurality of cluster groups, filtering the plurality of cluster groups to create a partial set of each of the plurality of cluster groups, and performing word embedding processing on the partial set of each of the plurality of cluster groups to obtain vectors for new words.