Topic Model Compression via Similarity-Based Merging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Large-scale topic models suffer from redundancy and increased model complexity, leading to storage difficulties and reduced efficiency due to excessive trained parameters and repetition between topics, making them cumbersome for interpretation and calculation.
Innovation Solution
A method and apparatus for compressing topic models by converting the format, merging similar topics based on similarity thresholds using Jaccard or Weighted Jaccard similarity, and deleting low-probability words, resulting in a more compact and efficient model.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of topics is increased to grasp long-tailed semantic information, then the model's ability to capture semantic information is improved, but the redundancy between topics increases and the interpretability of single topics deteriorates
Solution Approach 1:
The patent merges similar topics by calculating similarity between topic pairs and combining topics that exceed a similarity threshold. This consolidation reduces redundancy between topics while preserving the semantic information capture capability, directly resolving the contradiction between model adaptability and topic redundancy.
2Adaptability or versatility
If the number of topics is increased to grasp long-tailed semantic information, then the model's ability to capture semantic information is improved, but the efficiency of the entire model deteriorates
Solution Approach 1:
By merging similar topics into consolidated topics, the patent reduces the total number of topics in the model. This decreases the computational burden and improves model efficiency while maintaining the ability to capture semantic information through the merged topic representations.
3Adaptability or versatility
If excessive trained model parameters are used to increase topic number, then the semantic information coverage is improved, but the storage difficulty increases
Solution Approach 1:
The patent merges similar topics to reduce the total number of topics, which directly reduces the quantity of model parameters that need to be stored. This consolidation maintains semantic information coverage while decreasing storage requirements.
4Adaptability or versatility
If excessive trained model parameters are used to increase topic number, then the semantic information coverage is improved, but the calculation efficiency deteriorates
Solution Approach 1:
By merging similar topics and reducing the total topic count, the patent decreases the number of parameters that require calculation during model operations. This improves calculation efficiency while preserving semantic information coverage through the consolidated topic structure.
Data Source
AI summary
A method comprises: acquiring a to-be-compressed topic model, wherein each line of the topic model represents a distribution of a word among respective topics; performing a format conversion on the topic model to obtain a first topic model, wherein each line of the first topic model represents a distribution of a topic among respective words; selecting any two topics from the first topic model to form a topic pair, forming a topic pair set using at least one topic pair, and determining a similarity between the two topics in each topic pair in the topic pair set; merging topic pairs having a similarity greater than a similarity threshold to generate a second topic model; and performing a format conversion on the second topic model to obtain a compressed topic model, so that each line of the compressed topic model represents a distribution of a word among the respective topics.