Bridge-Query Labeling for False Negatives in ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine-learning models face inefficiencies in training due to the lack of effective generation of negative training examples, leading to false negatives, which degrade the model's performance in predicting user interactions with content.
Innovation Solution
An online system updates negative training examples by identifying bridge queries and computing similarity scores between initial query and bridge queries to generate updated labels, improving the training process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If automated measurement is used to generate training labels based on user interactions, then positive training examples are effectively generated, but negative training examples become inaccurate due to false negatives
Solution Approach 1:
The patent introduces bridge queries as an intermediary mechanism to connect negative examples with potential positive associations. By computing similarity scores between original queries and bridge queries, the system identifies false negatives and updates labels accordingly, resolving the contradiction between automated label generation accuracy and negative example validity
Solution Approach 2:
The system implements a feedback loop where similarity scores computed from bridge queries feed back into label updates. This feedback mechanism allows the system to iteratively refine negative example labels by comparing against bridge query results, thereby improving both measurement precision and negative example reliability
2Ease of manufacture
If all unselected search results are treated as negative examples, then label generation is simplified, but model training efficacy decreases due to false negatives
Solution Approach 1:
Instead of applying a blanket rule to all unselected results, the patent applies partial action by selectively updating only those negative examples that show high similarity to bridge queries. This approach maintains the simplicity of automated label generation while improving training efficacy by focusing computational effort on ambiguous cases
Solution Approach 2:
The system changes the label parameter from a static binary classification to a dynamic value that incorporates similarity scores. By computing updated labels based on bridge query similarity, the system transforms rigid negative labels into nuanced representations that reflect potential positive associations, thereby improving model training efficacy without significantly complicating the label generation process
Data Source
AI summary
An online system updates the labels on negative examples to account for the possibility that the example is a false negative. The system generates a set of initial training examples that each include a query input by the user and item data for an item presented as a result to the user's query. Each training example also includes an initial label, which represents whether the user interacted with the item presented as a search result. The online system updates the initial label for a negative training example by identifying a set of bridge queries and computing a similarity score between the query for the training example and the bridge queries. The online system computes an updated label for the negative example based on the similarity scores and updates the training example with the updated label.