Incremental LDA Model for Keyword Recommendation Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword recommendation technologies face challenges in providing pertinent recommendations for refined texts, particularly in industries with secondary classifications, such as sports-related topics like race cars, where precision and relevance are difficult to achieve.
Innovation Solution
A keyword recommendation method and system based on the Latent Dirichlet Allocation (LDA) model, which involves calculating a basic and incremental Dirichlet allocation model to obtain probability distributions of words and texts to topics, screening for incremental seed words, and calculating relevance scores to determine relevant keywords for query words.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing keyword recommendation technologies are used, then general topic recommendations can be provided, but precision and relevance for refined texts and secondary classifications cannot be achieved
Solution Approach 1:
The patent segments the keyword recommendation process into two distinct phases: basic LDA model training on general texts and incremental LDA model training on refined texts. This segmentation allows the system to first learn general topic structures and then adapt to specific refined domains, thereby improving precision for secondary classifications while maintaining versatility across different text types.
Solution Approach 2:
The patent performs preliminary action by first training the basic LDA model on general training texts before proceeding to incremental training on refined texts. This preliminary training establishes a foundation of general topic knowledge that enables the incremental model to focus specifically on refining recommendations for secondary classifications, thus improving precision without losing adaptability.
2Adaptability or versatility
If basic LDA model is trained on all training texts, then comprehensive topic coverage is achieved, but precision for specific incremental seed words deteriorates
Solution Approach 1:
The patent segments the training corpus into two parts: general training texts for basic LDA model and refined training texts containing incremental seed words for incremental LDA model. This segmentation allows comprehensive topic coverage from the basic model while maintaining high precision for specific seed words through the incremental model that focuses exclusively on refined texts.
Solution Approach 2:
The patent extracts incremental seed words and their associated refined training texts from the general training corpus to create a separate incremental training set. This extraction enables the incremental LDA model to specialize in specific secondary classifications without being diluted by the broader general text data, thus improving precision while maintaining comprehensive coverage through the basic model.
3Adaptability or versatility
If keyword recommendations are provided for general topics, then broad relevance is achieved, but accuracy for secondary classifications such as race cars deteriorates
Solution Approach 1:
The patent segments the recommendation system into two parallel models: basic LDA for general topics and incremental LDA for refined topics. This segmentation enables the system to provide broad relevance through the basic model while achieving high accuracy for secondary classifications like race cars through the incremental model that is specifically trained on refined texts containing those seed words.
Solution Approach 2:
The patent applies local quality by making the incremental LDA model specialized for specific refined topics while the basic LDA model handles general topics. This local specialization ensures that secondary classifications receive tailored, accurate recommendations rather than generic suggestions, thereby improving accuracy for refined topics while maintaining broad applicability through the basic model.
Data Source
AI summary
Keyword recommendation methods and systems based on a latent Dirichlet allocation (LDA) model. The method comprises: calculating a basic Dirichlet allocation model for training texts; obtaining an incremental seed word, and selecting from the training texts a training text matching the incremental seed word to serve as an incremental training text; calculating an incremental Dirichlet allocation model for the incremental training text; obtaining a probability distribution of complete words to topics and a probability distribution of complete texts to topics; calculating a relevance score between the complete word and any other complete word respectively to obtain a relevance score between every two complete words; and determining, according to an obtained query word and the obtained relevance score between every two complete words, a keyword corresponding to the query word. By employing an incremental training model, the present invention greatly improves the precision of topic clustering and topic diversity, and significantly improves the quality of keywords in the topics.