Topic Extraction Server Using Modular Clustering and Weighted Seed Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional techniques for extracting topics from document groups face challenges in determining accurate parameter values, leading to difficulties in extracting precise topics.
Innovation Solution
A server-based method that includes text preprocessing, keyword extraction, seed selection, initial clustering, and cluster combination units to enhance topic extraction accuracy, using weight calculations and association values to identify core words and combine similar clusters.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional topic extraction techniques are used, then the process is simple, but the accuracy of topic extraction is low due to difficulty in determining parameter values
Solution Approach 1:
The topic extraction process is divided into multiple distinct units: text preprocessing unit, keyword extraction unit, seed selection unit, initial clustering unit, and cluster combination unit. Each unit performs a specific function with defined inputs and outputs, making the complex process manageable and accurate through modular organization
Solution Approach 2:
The text preprocessing unit performs preliminary actions by extracting nouns and removing stopwords before keyword extraction. This preliminary filtering prepares the data in advance, ensuring that only relevant terms proceed to subsequent stages, thereby improving overall extraction accuracy
2Measurement precision
If parameter values are not properly determined, then the extraction process is fast, but the topic extraction accuracy deteriorates
Solution Approach 1:
The system determines parameter values automatically through internal calculations rather than requiring external manual input. The keyword extraction unit calculates weights based on document frequency, the seed selection unit determines parameters from keyword weights, and the clustering units establish their own parameters through association value calculations, enabling self-determination of all critical parameters
Solution Approach 2:
Parameter values are dynamically determined based on the actual data characteristics. Keyword weights change according to document frequency, seed selection parameters adapt based on keyword weights, and clustering thresholds are adjusted based on association values, allowing the system to optimize parameters for each specific extraction task
3Reliability
If keyword frequency is used for selection, then relevant keywords are identified, but the complexity of weight calculation increases
Solution Approach 1:
The text preprocessing unit extracts only nouns from the input text, removing all other word types and stopwords. This extraction of relevant elements in advance simplifies subsequent keyword extraction by focusing only on the part of speech most likely to represent topics, reducing the search space and calculation complexity
Data Source
AI summary
A method and a server for extracting a topic and evaluating suitability of the extracted topic are disclosed. The topic extraction server includes a text preprocessing unit configured to extract noun from a document group and remove stopword from the extracted noun, a keyword extraction unit configured to calculate a weight of a noun and extracting a keyword representing the document group, a seed selection unit configured to calculate a weight of the extracted keyword and select a seed, an initial clustering unit configured to generate one cluster including the selected seed and a keyword shown by several times in a sentence including the selected seed, and a cluster combination unit configured to extract a topic group.