Microblog Topic Classification via External Link Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing topic detection methodologies for microblogs are unreliable due to the short nature of posts, frequent use of abbreviated terms, and lack of external links or user profile information, making it difficult to classify content effectively.
Innovation Solution
A system and method for supervised training of topic models using User-Labeled Latent Dirichlet Allocation (uL-LDA) and User-Labeled Hierarchical Dirichlet Process (L2-HDP) techniques, which analyze post content and external links to generate trained topic models for microblog classification, allowing for user-specific topic modeling and updating.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing topic detection methodologies (PLSA, LDA) are used for microblog classification, then no manually constructed training data is required, but the models are unreliable due to short post length, abbreviated terms, and lack of external links or user profile information
Solution Approach 1:
The patent introduces an intermediary process that extracts and utilizes contextual information from external links and user profiles to enhance the classification of microblog posts. This intermediary layer bridges the gap between the limited microblog content and the need for reliable topic detection, allowing the system to leverage additional information sources without requiring manual training data construction.
2Measurement precision
If external links and user profile information are required for training models, then model accuracy may improve, but the system becomes unreliable since only 20-25% of tweets contain external links and user profile information is often unavailable due to privacy settings
Solution Approach 1:
The patent applies partial action by utilizing only the portion of information that is available (posts with external links or user profiles) while still functioning effectively with the remaining posts. The system does not require complete information from all posts to achieve effective classification, instead adapting to the partial information available in each case.
Solution Approach 2:
The system dynamically adjusts its information gathering strategy based on availability. When external links or user profiles are present, it leverages these additional parameters for enhanced classification. When they are absent, the system adapts by relying solely on the post content itself, changing its operational parameters to match the available data.
3Ease of manufacture
If the number of topics is predetermined in topic models, then the training procedure can be executed, but the model becomes inflexible and difficult to determine the appropriate number of topics
Solution Approach 1:
The patent implements a dynamic approach where the number of topics is not fixed in advance but can be adjusted based on the characteristics of the microblog posts being classified. The system can adapt the topic count to match the diversity and complexity of the actual data, making the model flexible rather than rigid.
Solution Approach 2:
The system performs preliminary analysis of the microblog posts to determine an appropriate number of topics before executing the full training procedure. This preliminary action allows the model to be configured optimally for the specific dataset at hand, rather than using a generic predetermined number of topics.
Data Source
AI summary
A method for assigning a topic to a collection of microblog posts may include, by an acquisition module, receiving from at least one messaging service server, a plurality of posts, wherein each of the plurality of posts comprise post content; by a generation module, analyzing the posts and extract, from at least one of the posts, a link with an address to an external document; and, by the acquisition module, accessing the external document that is associated with the address and fetch external content associated with the document. The method may also include by the generation module: analyzing the post content to identify at least one label for each post, for each post that includes a link, analyzing the external content to identify a topic, and using a topic modeling technique to generate a trained topic model comprising a plurality of topics and a plurality of associated words.