Topic Model Sentiment Priors Distributed Representations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing topic models for sentiment analysis rely on predefined dictionaries and treat sentiment priors as independent variables, which limits their ability to discover aspect-related sentiment words and improves sentiment classification effectively.
Innovation Solution
Training sentiment priors in the space of word embeddings, where similar words are assigned the same sentiment, allowing for the automatic extension of sentiment dictionaries and improved sentiment classification by using a regularizer to define sentiment for words with similar vectors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If sentiment priors are trained independently for every word using predefined dictionaries, then the model structure remains simple, but the ability to discover aspect-related sentiment words and extend sentiment dictionaries is limited
Solution Approach 1:
The patent combines topic models with distributed word representations (word embeddings) into a unified framework. By merging these two approaches, the model leverages the semantic coherence of word embeddings to automatically extend sentiment dictionaries while maintaining a structured probabilistic framework through topic models, thus improving adaptability without proportionally increasing complexity
Solution Approach 2:
The model enables automatic extension of sentiment dictionaries by utilizing the semantic space of word embeddings. The system self-organizes sentiment information by allowing sentiment priors to be defined in the continuous semantic space, where similar words automatically receive similar sentiment assignments through the regularizer, eliminating the need for manual dictionary curation for each word
2Adaptability or versatility
If separate sentiment priors are trained for each word, then individual word sentiment accuracy may be maintained, but the sentiment dictionary cannot be automatically extended to highly similar, interchangeable words
Solution Approach 1:
The patent changes the parameter space from discrete word-level sentiment priors to continuous sentiment representations in the word embedding space. By transforming sentiment priors into the continuous semantic space of pre-trained word embeddings, the model can automatically generalize sentiment information to similar words through the geometric structure of the embedding space, achieving both automatic extension and maintained precision
Solution Approach 2:
The patent introduces a regularizer as an intermediary mechanism that enforces consistency between sentiment priors of semantically similar words. This regularizer acts as a mediator that transfers sentiment information from known words to similar words through the word embedding space, enabling automatic dictionary extension while preserving sentiment classification accuracy through the intermediate semantic representation
3Loss of information
If a single unified sentiment prediction is provided by LSTM, then the model structure remains straightforward, but the interpretability and detailed aspect-specific sentiment analysis are reduced
Solution Approach 1:
The patent segments the unified sentiment prediction into aspect-specific sentiment analyses through topic modeling. By dividing the sentiment analysis into multiple topic dimensions, each representing different aspects, the model provides detailed aspect-specific sentiment information while maintaining a structured probabilistic framework that is more interpretable than a single unified LSTM prediction
Data Source
AI summary
There are provided a device and method for topic modeling with sentiment priors based on distributed representations. A method for topic modeling with sentiment priors based on distributed representations includes: inputting a review into a topic model; and by the topic model, determining a representation for each word in the review, wherein the representations are word vectors in a semantic space; and estimating the representations using the sentiment priors to determine a topic corresponding to the review, wherein the topic model includes the sentiment priors trained on the basis of the representations distributed by using a regularizer, the regularizer defining the same sentiment to words having similar word vectors, and wherein each sentiment prior is the same for words having similar word vectors.