Topic Modeling for Short Text Queries Using Character N-grams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current topic modeling solutions are inefficient for analyzing short search queries, as they are based on algorithms designed for longer documents and fail to recognize misspelled words, categorizing them as noise and misclassifying meaningful queries.
Innovation Solution
Employing character n-grams and word embeddings to generate vectors for search queries, which allows for clustering and theme identification, even with misspelled words, using techniques like Hierarchical Density-Based Spatial Clustering of Applications with Noise (HDBSCAN) to assign topics and reduce noise impact.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If traditional topic modeling algorithms designed for longer documents are used on short search queries, then the system can process search query data automatically, but the algorithms fail to recognize misspelled words and categorize them as noise, reducing measurement precision
Solution Approach 1:
The patent segments words into character n-grams (subword units) rather than treating them as complete words. This segmentation allows the model to recognize partial matches and misspelled words by identifying overlapping character sequences between the query and training library words, thereby improving measurement precision while maintaining automation.
Solution Approach 2:
The patent changes the parameter of word representation from full-word matching to character n-gram based representation. By transforming the input data structure and using subword embeddings, the system can process misspelled words and variations effectively, resolving the contradiction between automated processing and recognition accuracy.
2Productivity
If full-word libraries are used for training topic modeling algorithms, then the model can be trained efficiently, but it becomes less adept at recognizing vocabulary that does not exactly match the training library words such as misspelled words
Solution Approach 1:
By segmenting words into character n-grams, the system creates a more flexible vocabulary representation that maintains training efficiency while improving adaptability. The n-gram approach allows the model to recognize partial overlaps and variations without requiring exhaustive training on every possible word variation.
Solution Approach 2:
The character n-gram representation serves multiple functions: it enables recognition of correctly spelled words, misspelled words, and word variations all through the same mechanism. This universal approach improves both productivity and adaptability by handling diverse vocabulary cases with a single trained model.
3Device complexity
If misspelled words are categorized as noise in traditional topic modeling, then the algorithm maintains simplicity in noise filtering, but meaningful queries with misspelled words are misclassified, reducing reliability
Solution Approach 1:
The patent converts what was previously considered harmful (misspelled words categorized as noise) into beneficial signals for topic identification. By using character n-grams, misspelled words now provide partial matching information that helps identify the intended topic, transforming the harm into a benefit while maintaining reasonable complexity.
Solution Approach 2:
The patent changes the parameter of noise filtering from binary (noise or not) to a gradient based on character n-gram overlap strength. This allows the system to distinguish between actual noise and misspelled words that still contain meaningful character sequences, improving reliability without excessive complexity.
Data Source
AI summary
Systems and methods are directed to a computing device for determining homogeneous groups for search queries. The computing device may generate n-grams for each search query and apply word embedding techniques to calculate representative vectors for each search query. The computing device may train a clustering algorithm using the representative vectors. The computing device may generate topic labels for each cluster and associate each search query in a cluster with the topic label for the cluster.