L2 Normalized Embeddings for Popularity Bias in Recommendations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning-based item recommendation systems suffer from popularity bias, favoring popular items and poorly recommending less popular or new items due to their skewed distribution and radial property of softmax loss, which leads to poor performance in online settings.
Innovation Solution
Normalizing item and session embeddings using L2 norm and optimizing for cosine similarity to reduce the influence of embedding norms, incorporating position embeddings, and using a Graph Neural Network (GNN) to model session-graph representations, thereby addressing popularity bias.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If neural network-based recommendation models are used to predict item clicks, then the model can learn useful representations for session-graphs, but the model becomes biased towards recommending popular items and fails to recommend relevant less popular items
Solution Approach 1:
The patent applies L2 normalization to item embeddings, transforming the parameter space of embedding vectors. This normalization constrains embedding norms to be equal, preventing popular items with larger norms from dominating the recommendation scores. The parameter change from raw embeddings to normalized embeddings directly addresses the popularity bias while maintaining recommendation accuracy.
2Productivity
If the model relies on past item clicks to predict likely clicks, then the model can leverage historical data, but the model performs poorly for less popular new items arriving daily
Solution Approach 1:
L2 normalization creates equipotentiality by ensuring all item embeddings have equal norm. This eliminates the advantage that popular items inherently have due to their accumulated click data, which results in larger embedding norms. New and less popular items are placed on the same mathematical footing as popular items, allowing them to be recommended based on relevance rather than popularity.
3Measurement precision
If softmax loss with radial property is used for training, then the model can optimize click prediction, but the radial property inherently favors popular items with larger embedding norms
Solution Approach 1:
The patent applies L2 normalization as a preliminary anti-action before computing recommendation scores. By normalizing embeddings beforehand, the system preemptively counteracts the harmful radial property of softmax loss that would otherwise favor items with larger norms. This preliminary intervention prevents popularity bias from manifesting in the recommendation outputs.
Data Source
AI summary
This disclosure relates generally to method and system for handling popularity bias in item recommendations. In an embodiment the method includes initializing an item embedding look-up matrix corresponding to items in a sequence of item-clicks with respect to a training data. L2 norm is applied to the item embedding look-up matrix to learn a normalized item embeddings. Using a neural network, a session embeddings corresponding to the sequences of item-clicks is modeled and L2 norm is applied to the session embeddings to obtain a normalized session embeddings. Relevance scores corresponding to each of the plurality of items are obtained based on similarity between the normalized item embeddings and the normalized session embeddings. A multi-dimensional probability vector corresponding to the relevance scores for the items to be clicked in the sequence is obtained. A list of the items ordered based on the multi-dimensional probability vector is provided as recommendation.