Probabilistic Word Embeddings for Text Classification Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current text classification systems are limited in their ability to accurately classify text as they only provide coefficient values for known words, ignoring unknown words which diminishes the quality of probability scores and classification results.
Innovation Solution
The use of probabilistic word embeddings, specifically Gaussian distributions, to generate coefficient values for both known and unknown words through a set of mathematical functions, allowing for continuous representations of words in an embedding space.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If text classification systems use predetermined coefficient values for known words, then classification can be performed efficiently, but unknown words are ignored which diminishes the quality of probability scores and classification results
Solution Approach 1:
The patent introduces word embeddings as an intermediary representation that bridges known and unknown words. By mapping words to continuous vector representations based on contextual patterns, the system can infer coefficient values for unknown words through their embedding proximity to known words, thereby improving classification accuracy without sacrificing adaptability to new vocabulary
Solution Approach 2:
The system transforms discrete coefficient values into continuous probability distributions through embedding transformations. This parameter change allows the model to generate meaningful coefficient values for unknown words by leveraging the continuous nature of embedding spaces, where unseen words can be represented as combinations of known word embeddings
2Ease of manufacture
If coefficient values are predetermined for a finite set of known words, then the system is simpler to implement, but the quality of probability scores deteriorates due to ignoring unknown words
Solution Approach 1:
The patent creates copied representations of word meanings through embeddings. Instead of storing explicit coefficient values for every possible word, the system copies semantic information from known words to unknown words through their embedding representations, maintaining reliability while keeping the system implementation relatively simple
Solution Approach 2:
The embedding space serves multiple functions simultaneously: it represents known words, infers unknown words, captures semantic relationships, and generates probability distributions. This universal representation mechanism improves probability score quality without requiring separate mechanisms for different word types
Data Source
AI summary
Disclosed are systems, methods, and non-transitory computer-readable media for probabilistic word embeddings for text classification. A text classification system receives a message including a keyword and determines an embedding probability distribution representing the keyword. The text classification system then determines an embedding value for the keyword based on the embedding probability distribution. The text classification system uses the embedding value as input into a set of mathematical functions, yielding a first set of coefficient values for the keyword. Each respective mathematical function from the set corresponds to a respective classification label from a set of classification labels and defines a continuous surface. Each respective mathematical function is determined from embedding values for a set of known keywords, distribution variance values for the set of known keywords, and a subset of coefficient values for the set of known keywords that corresponds to the respective classification label.