Machine Learning Ranking Training with Noise-Smoothed Policy Gradients
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing search engine ranking algorithms face challenges in optimizing ranking quality due to the non-convex and non-smooth nature of ranking quality metric functions, leading to inefficiencies in training Machine Learning Algorithms (MLAs) for ranking objects in Search Engine Results Pages (SERPs).
Innovation Solution
A method and server are developed to train MLAs using a ranking quality metric function that incorporates noise-induced relevance scores and policy gradient values, allowing for direct optimization of ranking quality through techniques like coordinate-wise antithetic sampling and decision-tree-based algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional ranking algorithms are used to rank search results, then the ranking process can be performed, but the training efficiency and optimization capability are insufficient due to the non-convex and non-smooth nature of ranking quality metric functions
Solution Approach 1:
The patent transforms the non-convex and non-smooth ranking quality metric function into a smooth approximation by introducing a temperature parameter. This parameter transformation allows the use of gradient-based optimization methods, significantly improving training efficiency while maintaining the essential ranking quality assessment capability.
Solution Approach 2:
The patent introduces a policy gradient as an intermediary mechanism that bridges the ranking quality metric function and the training process. This intermediary enables the transmission of optimization signals from the quality metric to the model parameters, solving the optimization problem caused by the non-differentiable nature of traditional ranking metrics.
2Reliability
If noise-induced relevance scores are introduced to smooth the ranking quality metric function, then training optimization is improved, but the computational complexity increases
Solution Approach 1:
The patent applies partial noise induction rather than complete randomization, using a controlled temperature parameter to introduce just enough noise to smooth the metric function while maintaining its discriminative power. This partial action achieves the smoothing effect without excessive computational overhead.
Solution Approach 2:
The patent performs smoothing of the ranking quality metric function in advance during the training process, rather than computing exact gradients during backpropagation. This preliminary smoothing action allows the use of standard gradient descent methods without requiring complex computational procedures during the main training loop.
3Manufacturing precision
If policy gradient methods are used for training the machine learning algorithm, then direct optimization of ranking quality is achieved, but the training process becomes more complex
Solution Approach 1:
The patent replaces complex policy gradient computation with a simplified gradient estimation approach using temperature-based smoothing. This substitution maintains the direct optimization capability of policy gradient methods while eliminating the computational complexity of exact gradient calculations for non-differentiable ranking metrics.
Data Source
AI summary
Method and server for training a Machine Learning Algorithm (MLA) for ranking objects in response to a query are disclosed. The training includes use of a ranking quality metric function that is one of a flat and a discontinuous function to determine a performance score of the MLA. The method includes generating relevance scores for a set of training objects based on data associated with the set of training objects and a training query, generating noise-induced relevance scores for the set of training objects by combining the relevance scores and noise values, generating the performance score for the MLA based on the noise-induced relevance scores, determining a policy gradient value for adjusting relevance scores to be generated by the MLA for the in-use objects in response to the in-use query, and applying the policy gradient value for training the MLA to rank in-use objects in response to an in-use query.