Keyword Extraction Models Using Title Attention for Short Texts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword extraction methods for short texts are labor-intensive due to the need for manual data annotation and threshold setting, and they struggle to accurately capture keywords from information entities like mini programs on the WeChat platform.
Innovation Solution
A keyword extraction model trained using an encoding-decoding mechanism that leverages the semantic relationship between a text and its title, performing word and character segmentation to adjust attention weights and determine extraction thresholds adaptively without manual intervention.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If traditional keyword extraction methods (TF-IDF, textrank) are used, then keyword extraction can be performed with simple algorithms, but the text length requirement is relatively long and sufficient information cannot be provided for short texts
Solution Approach 1:
The patent changes the fundamental parameters of the extraction approach by shifting from frequency-based metrics (TF-IDF) and graph-based voting (textrank) to semantic representation learning. This involves transforming text into dense vector embeddings that capture semantic meaning, allowing accurate keyword extraction from short texts where traditional frequency-based methods fail due to insufficient token counts.
Solution Approach 2:
The patent replaces mechanical counting and voting mechanisms with neural network-based semantic understanding. Instead of mechanically counting term frequencies or performing graph traversals, the system uses deep learning models to understand semantic relationships, enabling accurate keyword identification even with limited text input.
2Extent of automation
If sequence-to-sequence model is used for keyword extraction, then automated extraction can be achieved, but a large quantity of data annotation and manual threshold presetting is required
Solution Approach 1:
The patent implements self-service by enabling the model to automatically determine extraction thresholds through self-attention mechanisms. The attention weights computed during encoding naturally provide relevance scores that eliminate the need for manual threshold presetting. The system serves itself by using its own internal representations to guide the extraction process without external intervention.
Solution Approach 2:
The patent incorporates feedback loops where the decoding process continuously references the encoded text representations and adjusts keyword generation based on attention weights. The model receives feedback from its own attention mechanisms to dynamically determine which words to extract, replacing the need for external threshold configuration and reducing annotation requirements.
3Measurement precision
If manual annotation and threshold setting are performed, then extraction accuracy can be controlled, but labor costs increase significantly
Solution Approach 1:
The model performs self-calibration by using attention mechanisms to automatically determine which words are most relevant. The attention weights serve as intrinsic relevance scores that eliminate the need for manual threshold setting, achieving both accuracy and efficiency without human intervention in the extraction process.
Solution Approach 2:
The patent changes the parameter determination approach from external manual setting to internal model-generated parameters. Attention weights dynamically computed during inference replace static manually-set thresholds, allowing the system to adapt to different texts automatically while maintaining accuracy and reducing labor costs.
Data Source
AI summary
This application provide a keyword extraction method. The method includes: performing word segmentation on a text, to obtain a plurality of candidate words; selecting, from the plurality of candidate words, a candidate word that appears in the title; determining an extraction threshold according to an attention weight of the selected candidate word relative to the title; and determining a keyword of the text from the candidate words according to the extraction threshold.