Salient Item Identification Engine for Web Search Relevance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Commercial web search engines face challenges in accurately identifying salient entities or items within web pages, leading to irrelevant links and decreased relevance in entity-triggered experiences, as many entities mentioned in documents are not central to the content.
Innovation Solution
A system that includes a salient item identification engine using a log data acquisition component to obtain query and click data, a soft labeling component to determine salience annotation values based on user query visits, and model training to generate salience scores for items within web pages, leveraging user behavior and document features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If all entities or items in web pages are extracted and linked, then the coverage of entity links is increased, but the precision and relevance of salient entity identification deteriorates
Solution Approach 1:
The patent segments entities into different salience levels (salient vs. non-salient) based on their importance to the document content. This segmentation allows the system to process all entities while distinguishing and prioritizing only the salient ones for linking and recommendation, thus maintaining high coverage while improving precision.
Solution Approach 2:
The patent applies different quality standards to different entities within the same document. Instead of treating all entities uniformly, it assigns varying levels of salience annotations based on local context and importance, enabling precise identification of salient entities while maintaining comprehensive entity extraction.
2Measurement precision
If manual annotation methods are used to identify salient entities, then the precision of salience identification is improved, but the productivity and scalability deteriorates
Solution Approach 1:
The patent implements a self-service annotation system where the software automatically generates salience annotations using machine learning models trained on manually annotated data. This allows the system to annotate entities at scale without requiring continuous manual intervention, thereby maintaining high precision while dramatically improving productivity and scalability.
Solution Approach 2:
The patent performs preliminary manual annotation to create training datasets, then uses these datasets to train automated models that can perform salience identification at scale. This preliminary action enables the system to achieve both high precision (from trained models) and high productivity (from automated processing).
Data Source
AI summary
A set of representations of item-page pairs of items and respective web pages that include the respective items is obtained, each representation including feature function values indicating weights associated with features of associated web pages, the features including page classification features. An annotated set of labeled training data that is annotated with salience annotation values of items for respective web pages that include the items is obtained. The salience annotation values are determined based on a soft function, by determining a first count of a total number of user queries associated with corresponding visits to the respective web pages, and determining a ratio of a second count to the first count, the second count determined as a cardinality of a subset of the corresponding visits that are associated with user queries that include the item, the subset included in the corresponding visits. Models are trained using the annotated set.