Recommendation Model Training With Impact-Weighted Negative Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In large-scale recommendation systems, issues such as data imbalance, excessive resource consumption, slow training, and reduced prediction accuracy due to overfitting and data homogeneity lead to suboptimal model performance, particularly affecting top N ranking.
Innovation Solution
A recommendation model training method that involves selecting positive and negative samples based on specific rules, calculating sampling probabilities, and incorporating these samples into a training set to enhance model performance, using algorithms that consider score differences, vector distances, and ranking variations to prioritize impactful negative samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all available data is used for training, then model coverage is improved, but training time and resource consumption increase excessively
Solution Approach 1:
The patent segments the training data into positive samples and negative samples, and further segments negative samples into different groups based on their characteristics (e.g., hard negative samples, easy negative samples). This segmentation allows the model to focus computational resources on more important samples while still maintaining comprehensive coverage through selective sampling.
Solution Approach 2:
Instead of using all available data equally, the patent applies partial action by selectively sampling only a portion of negative samples based on calculated sampling probabilities. Hard negative samples are sampled with higher probability while easy negative samples are sampled with lower probability, achieving efficient training without exhaustive data processing.
2Reliability
If more negative samples are included in training, then model robustness is improved, but data imbalance worsens when negative samples far outnumber positive samples
Solution Approach 1:
The patent changes the parameter of sample selection by introducing sampling probabilities that dynamically adjust the effective quantity of negative samples. By calculating sampling probabilities based on sample characteristics (e.g., score differences, vector distances), the method transforms the fixed data imbalance into a flexible sampling strategy that maintains robustness while balancing the training distribution.
Solution Approach 2:
The patent applies local quality by treating different negative samples differently based on their individual characteristics. Hard negative samples (which are more informative) are sampled with higher probability, while easy negative samples are sampled with lower probability. This local differentiation allows the model to focus on critical samples that improve robustness without being overwhelmed by redundant data.
3Stability of the object's composition
If homogeneous data is used for training, then data consistency is improved, but model overfitting increases
Solution Approach 1:
The patent introduces asymmetry in the sampling strategy by treating positive samples and negative samples differently, and further differentiating among negative samples based on their characteristics. This asymmetric sampling ensures that the training set includes diverse samples with varying degrees of difficulty and relevance, preventing the model from learning only homogeneous patterns and reducing overfitting.
4Device complexity
If sampling probability is based on score difference only, then sampling simplicity is improved, but sampling accuracy decreases when other factors like vector distance matter
Solution Approach 1:
The patent merges multiple sampling criteria (score difference, vector distance, and other relevant factors) into a unified sampling probability calculation. By combining these different dimensions of sample quality assessment, the method achieves comprehensive sampling accuracy that considers both the relevance (score difference) and the semantic distance (vector distance) of samples, while maintaining reasonable algorithmic complexity through efficient computation.
Data Source
AI summary
A recommendation model training method includes selecting a positive sample in a sample set, and adding the positive sample to a training set, where the sample set includes the positive sample and negative samples, each sample includes n sample features, n≥1, and the sample features of each sample include a feature used to represent whether the sample is a positive sample or a negative sample, calculating sampling probabilities of the negative samples in the sample set by using a preset algorithm, selecting a negative sample from the sample set based on the sampling probability, and adding the negative sample to the training set, and performing training by using the samples in the training set, to obtain a recommendation model.