Entity Ranking via Distribution Sampling for ML Data Quality
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional methods for selecting and ranking entities in machine learning models tend to favor entities with higher interaction levels, leading to lower-quality entities with fewer interactions being overlooked, which limits their visibility and future ranking opportunities.
Innovation Solution
The use of distribution sampling methods, including rate and probability distributions, to adjust entity selection and ranking, allowing entities with lower interaction levels to be more prominently featured by generating and sampling these distributions to expand the range of values considered in the selection and ranking processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional ranking methods are used that favor entities with higher interaction levels, then entities with high interactions are ranked higher and receive more visibility, but entities with lower interactions are overlooked and have reduced visibility and future ranking opportunities
Solution Approach 1:
The patent changes the ranking parameter from raw interaction counts to a distribution-based metric that considers both interaction level and entity selection frequency. By sampling from a distribution that incorporates uncertainty and selection history, the system adjusts rankings to provide opportunities for lesser-known entities while maintaining reliability for well-performing entities.
Solution Approach 2:
The ranking system becomes dynamic by continuously updating the distribution parameters based on observed interactions and selection frequencies. The distribution parameters (mean and standard deviation) are adjusted over time, allowing the system to adaptively balance between exploiting known high-performing entities and exploring potentially valuable lesser-known entities.
2Adaptability or versatility
If distribution sampling methods are used to expand the range of values considered in entity selection, then entities with lower interaction levels gain visibility and ranking opportunities, but the selection process becomes more complex
Solution Approach 1:
The system uses the observed interaction data and selection history itself to generate the distribution parameters, making the complex sampling process self-justifying. The data that is already being collected for ranking purposes is repurposed to create the distribution, so the additional complexity is paid for by utilizing existing data infrastructure rather than requiring separate complex measurement systems.
Data Source
AI summary
Embodiments of the disclosed technologies include generating a reward score for an entity. A rate distribution is determined using the reward score and a number of times the entity has been selected for ranking. A sampled rate value is generated by sampling the rate distribution. A probability score is generated for a pair of the entity and a user based on the sampled rate value. A probability distribution is determined using the probability score. A sampled probability value is generated by sampling the probability distribution. A machine learning model is trained using the sampled probability value.