Hierarchical Topic Model Using Hyperbolic Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing hierarchical topic models fail to capture coherent and distinct topic-subtopic relations due to arbitrary selection of subtopics and have unreasonably long training times, leading to high resource costs.
Innovation Solution
The use of hyperbolic word embeddings to cluster words and determine subtopics based on observed quantities, allowing for an interpretable topic hierarchy with definable relationships between topics and subtopics, and an intelligent algorithm to reduce resource costs in generating hierarchical topic models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If hierarchical topic models are used to capture topic-subtopic relations, then the ability to identify patterns among topics is improved, but the training time becomes unreasonably long and resource costs increase
Solution Approach 1:
The patent segments the topic modeling process into distinct hierarchical levels (first-level topics and second-level subtopics). Each level is processed separately through independent clustering operations, allowing the model to capture coherent topic-subtopic relationships without requiring a single monolithic training process that would be computationally prohibitive.
Solution Approach 2:
The patent performs preliminary clustering of words into first-level topics before proceeding to identify second-level subtopics. This preliminary organization of data into coherent groups enables subsequent subtopic identification to be more efficient and targeted, rather than processing all words simultaneously from scratch.
2Device complexity
If arbitrary selection of subtopics is used in hierarchical topic models, then the model structure is simplified, but the topic-subtopic relations become incoherent and indistinct
Solution Approach 1:
The patent employs a feedback mechanism where the clustering results from the first level inform the subtopic identification at the second level. The system uses the identified first-level topics as input to guide the discovery of second-level subtopics, ensuring that subtopics are coherent and distinct rather than arbitrarily selected. This feedback loop maintains model coherence while preserving interpretability.
3Ease of operation
If traditional topic modeling is used to organize documents, then the documents can be grouped into topics, but the inherent hierarchy of topics is not captured
Solution Approach 1:
The patent implements a nested hierarchical structure where second-level topics (subtopics) are nested within first-level topics. This nested organization captures the inherent hierarchy of topics while maintaining the document grouping capability. Documents can be associated with multiple levels of the hierarchy, preserving both the ease of document organization and the rich hierarchical relationships among topics.
Solution Approach 2:
The patent transitions from flat topic modeling to hierarchical topic modeling by adding a dimensional layer to the topic structure. Instead of organizing documents into a single level of topics, the system creates a two-level hierarchy (topics and subtopics), adding structural dimensionality that captures the inherent organization of subject matter while preserving document grouping capabilities.
Data Source
AI summary
Some techniques described herein relate to generating a hierarchical topic model (HTM), which can be used to generate custom content. In one example, a method includes determining first-level topics in a topic hierarchy related to a corpus of documents. A first-level topic of the first-level topics includes multiple words. The multiple words are grouped into clusters based on word embeddings of the multiple words. The multiple words are then subdivided into second-level topics as subtopics of the first-level topic, such that the number of second-level topics equals the number of clusters. A document of the corpus of documents is assigned to the first-level topic and to a second-level topic of the second-level topics, and an indication is received of access by a user to the document. Custom content is generated for the user based on one or more other documents assigned to the first-level topic and the second-level topic.