Key Phrase Extraction for Topical Authority Ranking
Solution Overview
Problem
Conventional methods for calculating topical authority of websites are inadequate for large-scale internet applications, failing to accurately and efficiently assess the authority of webpages across numerous topics and new content, leading to issues with disinformation and spam in search results.
Innovation Solution
A system that extracts key phrase candidates from documents using machine learning to compute a site's topical authority score, incorporating features like Term Frequency, Inverse Document Frequency, and user clicks to rank websites based on their authority within specific topics, thereby improving search engine results and trustworthiness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional global ranking methods are used to assess website authority, then a single ranking score is produced for each website, but the resolution is insufficient to reflect authority on granular or specific topics
Solution Approach 1:
The patent segments the global ranking problem into topic-specific rankings by introducing topic models that decompose website authority into multiple topic dimensions. Each website is evaluated on individual topics rather than receiving a single aggregate score, enabling granular authority assessment while managing complexity through modular topic modeling components.
2Adaptability or versatility
If fixed sets of topics are used for ranking, then existing topics can be evaluated, but new pages and sites appearing daily cannot be adequately handled
Solution Approach 1:
The patent implements dynamic topic extraction that adapts to new content by using unsupervised topic modeling algorithms that automatically discover and incorporate emerging topics from incoming pages and sites. The system continuously updates its topic vocabulary without requiring manual reconfiguration, enabling it to handle daily new content while maintaining extraction precision through probabilistic topic assignment.
3Quantity of substance
If machine learning models process billions of pages to extract topics, then comprehensive topic coverage is achieved, but computational efficiency and processing time are reduced
Solution Approach 1:
The patent applies partial processing by focusing computational resources on the most informative features and topics for each page rather than analyzing all content uniformly. The system extracts only key topical features needed for authority assessment, skipping redundant processing steps, which maintains comprehensive topic coverage across billions of pages while improving processing throughput through selective feature extraction.
Data Source
AI summary
A system for extracting key phrase candidates from a corpus of documents, including a processor, a memory, and a program executing on the processor. The system is configured to run a key phrase model to extract one or more key phrase candidates from each document in the corpus and convert each extracted key phrase candidate into a feature vector. The key phrase model also filters the feature vectors to remove duplicates using a classifier that was trained on a set of key phrase pairs with manual labels indicating whether two key phrases are duplicates of each other, to produce remaining key phrase candidates. The system uses the remaining key phrase candidates in a computer-implemented application.