Emerging Trend Detection Using Word-Pair Proximity Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Identifying emerging trends is difficult without prior knowledge of the trend, as existing methods rely heavily on known patterns and are inefficient in detecting new or evolving trends.
Innovation Solution
A system and method for automated identification of emerging trends using a proximity model that analyzes word pairs in documents, tracking their frequency and velocity over time, and utilizing pivot words to provide context and identify trends.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If automated trend identification is implemented, then productivity is improved, but device complexity increases
Solution Approach 1:
The system segments the trend identification process into distinct modules: document collection from multiple sources, proximity model training for word pair extraction, trend identification based on frequency and velocity analysis, and output to downstream systems. This modular segmentation enables automated processing while managing system complexity through organized functional components.
2Measurement precision
If proximity model training is performed to identify word pairs, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The system performs preliminary action by training the proximity model in advance to learn word co-occurrence patterns and relationships. Once trained, the model can quickly identify relevant word pairs in new documents without requiring retraining, thus achieving high measurement precision while minimizing time loss during actual trend identification operations.
3Reliability
If comprehensive document analysis is conducted, then reliability is improved, but quantity of substance increases
Solution Approach 1:
The system extracts only the essential elements needed for trend identification: word pairs within predetermined proximity distances and their frequency/velocity metrics. By taking out only these critical features from the comprehensive document content rather than processing all text data, the system maintains reliable trend detection while reducing the quantity of data that must be processed and stored.
Data Source
AI summary
Systems and methods for automated identification of emerging trends are disclosed. A method may include: (1) receiving, by a computer program executed on an electronic device, a plurality of documents; (2) training, by the computer program, a proximity model to identify word pairs in each of the plurality of documents, wherein the word pairs comprise two words within a predetermined distance of each other at a predetermined frequency in the plurality of documents; (3) identifying, by the computer program and using the proximity model, trends involving the word pairs, wherein the trends are based on a frequency that the word pairs appear in the plurality of documents and/or a velocity at which the word pairs appear in the plurality of documents over a period of time; and (4) outputting, by the computer program, the word pairs and the trends to a downstream system.