Recurrent Neural Network Topic Trend Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional topic detection and tracking systems fail to accurately capture temporal latent topic dependencies and trends in text document collections over time, as they do not account for the evolution of topics and their relationships across sequential documents.
Innovation Solution
A method and system using a two-layered recurrent neural network-replicated softmax model to generate hidden topic vectors from bag of words vectors, incorporating previous document collections' states to derive topic trends and probabilities, enabling precise topic detection and tracking in temporal ordered text document collections.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional probabilistic topic models (e.g., LDA) are used to extract semantic topics from text documents, then topics can be detected and tracked, but temporal latent topic dependencies between topic collections evolving over time cannot be captured
Solution Approach 1:
The patent transforms the static topic modeling approach into a dynamic sequential model. The recurrent neural network processes text document collections in temporal sequence, where the hidden state at each time step incorporates information from previous time steps. This dynamic approach allows the system to capture evolving topic dependencies over time, resolving the contradiction between detecting topics and preserving temporal dependency information.
Solution Approach 2:
The patent introduces a hidden state vector as an intermediary that carries temporal dependency information between time steps. This hidden state acts as a mediator that preserves and transmits topic dependencies across the sequence of text document collections, enabling the model to capture temporal relationships while maintaining topic detection capability.
2Productivity
If conventional methods compute topic distribution on all text collections, then topics can be detected, but topic trends and keyword trends cannot be captured accurately
Solution Approach 1:
The patent segments the text document collections into temporal sequences and processes them step-by-step through the recurrent neural network. This segmentation allows the model to capture local temporal patterns and trends at each time step while maintaining overall topic detection efficiency, resolving the contradiction between productivity and trend detection accuracy.
Solution Approach 2:
The patent maintains continuous processing of the temporal sequence through the recurrent neural network, where the hidden state is continuously updated at each time step. This continuous action enables the model to track topic trends over time while preserving detection efficiency, as the computational process flows continuously through the sequence without requiring repeated full-document analysis.
3Loss of information
If temporal ordered text document collections are processed to capture evolutional processes, then topic trends can be identified, but the complexity of the model increases
Solution Approach 1:
The patent replaces complex explicit temporal modeling mechanisms with a recurrent neural network that implicitly captures temporal dependencies through its hidden state. This substitution simplifies the model structure while maintaining the ability to capture temporal relationships, as the RNN's inherent sequential processing capability eliminates the need for explicit temporal feature engineering.
Data Source
AI summary
A method and system for automatically performing a discovery of topics within temporal ordered text document collections are provided. The method includes generating a bag of words vector for each text document collection using a predefined dictionary. The method also includes iteratively calculating, based on the generated bag of words vectors, for each text document collection, a hidden topic vector representing topics of the respective text document collection using a calculated hidden state vector memorizing a hidden state of all previous text document collections.