Unsupervised Aspect Extraction via Word Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing unsupervised aspect extraction techniques, such as LDA-based models, fail to directly encode semantic coherence, leading to aspects with loosely related terms, and require large amounts of labeled data for supervised learning, which is time and resource intensive.
Innovation Solution
The approach uses word embeddings, specifically trained using models like Word2vec, and incorporates domain-specific knowledge through graph-based learning objectives to refine word vectors, followed by k-means clustering to infer semantically coherent aspects and calculate association scores for word ranking.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If topic models are used for unsupervised aspect extraction, then labeled data requirement is reduced, but semantic coherence of aspects deteriorates
Solution Approach 1:
The patent introduces word embeddings as an intermediary representation layer between the topic model and the raw text. Word2vec embeddings capture semantic relationships between words, serving as a mediator that guides the topic model to discover semantically coherent aspects without requiring labeled data. The embeddings act as a bridge that transfers semantic knowledge from pre-trained models to the aspect extraction task.
Solution Approach 2:
The patent changes the parameter representation from discrete topic distributions to continuous word embedding vectors. By transforming the aspect representation into the embedding space, the model leverages the semantic structure already captured in the embeddings, thereby improving semantic coherence while maintaining the unsupervised nature of the approach.
2Measurement precision
If supervised learning is used for aspect extraction, then aspect extraction precision is improved, but time and resource consumption increases
Solution Approach 1:
The patent performs preliminary action by pre-training word embeddings on large corpora before applying the model to aspect extraction. This pre-training phase captures general semantic relationships that can be transferred to the specific aspect extraction task, reducing the need for task-specific labeled data and decreasing training time and resource requirements while maintaining precision.
Data Source
AI summary
Methods, systems, and computer-readable storage media for receiving a vocabulary that includes text data that is provided as at least a portion of raw data, the raw data being provided in a computer-readable file, providing word embeddings based on the vocabulary, the word embeddings including word vectors for words included in the vocabulary, clustering word embeddings to provide a plurality of clusters, each cluster representing an aspect inferred from the vocabulary, determining a respective association score between each word in the vocabulary and a respective aspect, and providing a word ranking for each aspect based on the respective association scores.