Keyword Extraction Using Probabilistic Feature Weighting
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing keyword extraction methods are overly complex and suffer from low precision due to reliance on manually formulated rules and inadequate consideration of multiple features such as term frequency, location, semantic, and linguistic features.
Innovation Solution
A method and apparatus that extract candidate keywords from text by obtaining effective features and calculating probabilities using weighting coefficients, eliminating the need for manual rules and reducing complexity by focusing on relevant features for determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If manually formulated rules are used for keyword extraction, then the extraction process can be structured, but the process becomes excessively complex and accuracy decreases
Solution Approach 1:
The patent replaces manually formulated rules (mechanical system) with a machine learning model that automatically learns feature weightings from data. The model substitutes complex rule-based logic with probabilistic calculations based on learned features, thereby reducing process complexity while improving extraction accuracy through adaptive learning rather than static rules
Solution Approach 2:
The patent transforms the keyword extraction approach by changing from fixed manual rules to dynamic parameter weightings learned from training data. The system adjusts feature importance parameters automatically based on statistical analysis of the training corpus, allowing the extraction process to adapt to different domains and text types without manual rule reconfiguration
2Measurement precision
If multiple features are considered for keyword extraction, then extraction precision can be improved, but the calculation amount increases
Solution Approach 1:
The patent extracts and retains only the most discriminative features for keyword extraction by using the machine learning model to identify and weight significant features while discarding irrelevant ones. This selective feature extraction maintains high precision by focusing on informative features while reducing calculation overhead by eliminating redundant features from the processing pipeline
Solution Approach 2:
The patent applies partial action by selecting a subset of the most important features rather than processing all possible features. The machine learning model identifies and weights only the critical features needed for accurate extraction, performing sufficient (but not excessive) analysis to achieve high precision without the computational cost of evaluating every possible feature
Data Source
AI summary
A keyword extraction method is provided. A candidate keyword from target text is extracted by a server. For each candidate keyword, each effective feature corresponding to the candidate keyword is obtained by the server. Calculation is performed by the server according to each effective feature corresponding to the candidate keyword and a weighting coefficient respectively corresponding to each effective feature, to obtain a probability that the candidate keyword belongs to a target keyword, and the candidate keyword is determined as the target keyword of the target text based on the probability.