Topic Mining System Using Keyword Extraction and Hierarchical Querying

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing topic mining systems are inefficient and costly, as they require processing large numbers of documents to identify latent topics and often fail to assign human-readable names to clusters, making it difficult to find relevant documents in unindexed and uncategorized data stores.

Innovation Solution

A computer-implemented method that extracts initial keywords from data artifacts, generates contextually relevant keywords, retrieves topic lists from a topic processor, and calculates an ordered list of probable topics using suffix indices and frequency analysis to efficiently categorize documents.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If traditional topic mining systems process large numbers of documents to identify latent topics, then topic identification capability is improved, but processing cost and time consumption increase significantly

Engineering Contradiction:

Improvetopic identification capabilityVSAvoidprocessing time

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system performs preliminary actions by extracting keywords from documents before full topic mining processing. Keywords are extracted and used to query pre-built topic hierarchies, allowing the system to identify topics without processing entire documents through traditional expensive topic mining algorithms.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The invention extracts only the essential elements (keywords) from documents rather than processing the complete document content. By taking out key terms and querying them against topic hierarchies, the system achieves topic identification without the computational burden of processing all document data.

Inventive Principle:

Principle #2Taking out (Extraction)

2Measurement precision

If traditional topic mining systems process large numbers of documents to identify latent topics, then topic identification capability is improved, but processing cost increases

Engineering Contradiction:

Improvetopic identification capabilityVSAvoidprocessing cost

Core Design Contradiction:

Measurement precisionVSLoss of energy

Solution Approach 1:

Topic hierarchies are pre-built and stored in databases before document processing. When a document needs topic identification, the system queries these pre-prepared hierarchies using extracted keywords, avoiding the need to re-process large document corpora and reducing computational energy consumption.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system extracts only keywords from documents rather than performing full document analysis. This extraction approach significantly reduces the amount of data that needs to be processed, thereby lowering computational costs while maintaining topic identification effectiveness.

Inventive Principle:

Principle #2Taking out (Extraction)

3Ease of manufacture

If documents are stored without indexing or categorization, then storage simplicity is maintained, but document retrieval difficulty increases

Engineering Contradiction:

Improvestorage simplicityVSAvoiddocument retrieval difficulty

Core Design Contradiction:

Ease of manufactureVSDifficulty of detecting and measuring

Solution Approach 1:

The system replaces traditional mechanical indexing and categorization mechanisms with an automated keyword-based topic assignment system. Documents are automatically tagged with topics by extracting keywords and querying topic hierarchies, eliminating the need for manual or complex automated indexing while improving retrieval efficiency.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

4Productivity

If coarse-grained techniques are used to group documents into clusters, then processing speed is improved, but topic naming capability deteriorates

Engineering Contradiction:

Improveprocessing speedVSAvoidhuman-readable topic names

Core Design Contradiction:

ProductivityVSLoss of information

Solution Approach 1:

The system adds a new dimension to topic identification by incorporating pre-built topic hierarchies with human-readable names. Instead of relying solely on clustering algorithms that produce abstract topic labels, the system queries structured topic hierarchies to obtain meaningful, human-readable topic names that preserve information while maintaining processing efficiency.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

Data Source

PatentUS10311087B1Systems and methods for determining topics of data artifacts

Publication Date: 2019.06.04 ARCTERA US LLC

AI summary

The disclosed computer-implemented method for determining topics of data artifacts may include (1) extracting at least one initial keyword from a data artifact with an unknown topic, (2) creating a set of keywords by generating a plurality of contextually relevant keywords related to the initial keyword and combining the initial keyword with the contextually relevant keywords to form the set of keywords, (3) retrieving, from a topic processor, at least one list of topics associated with each keyword within the set of keywords, and (4) generating, based on the retrieved topic lists, an ordered list of probable topics of the data artifact. Various other methods, systems, and computer-readable media are also disclosed.