Cross-model score normalization for content ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing content management systems face challenges in effectively normalizing relevancy scores generated by different machine learning models for various types of content, making it impractical to implement a single prediction pipeline or model.
Innovation Solution
The use of distinct machine learning sub-models to score different types of content, with a normalizing mapping function represented as a cubic spline to map relevancy scores to expected end-user interaction scores, allowing for cross-model score normalization and ranking of content suggestions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If distinct machine learning sub-models are used to score different types of content, then the system can handle disparate content types more effectively, but the complexity of the prediction pipeline increases
Solution Approach 1:
The patent divides the content scoring system into distinct machine learning sub-models, each specialized for specific content types (e.g., files, folders, cloud documents). This segmentation allows each sub-model to be optimized for its particular content type while maintaining overall system versatility. The normalization layer then integrates these segmented models into a unified ranking system.
2Device complexity
If a single machine learning model is used to score all content types, then the prediction pipeline is simpler, but the model cannot effectively handle the diversity of content types
Solution Approach 1:
The system segments the content scoring task into multiple specialized sub-models based on content type. Each sub-model processes a specific category of content (files, folders, cloud documents) with tailored features and algorithms, thereby achieving effective handling of content diversity while maintaining manageable complexity through modular design.
3Ease of operation
If relevancy scores from different sub-models are used directly for ranking, then the scoring process is straightforward, but the scores cannot be meaningfully compared across different content types
Solution Approach 1:
The patent applies parameter transformation through normalization mapping functions that convert relevancy scores from different sub-models into a common scale. This transformation preserves the relative ranking within each content type while enabling meaningful comparison across different content types, thereby achieving both score comparability and operational simplicity.
4Measurement precision
If a normalization mapping function is introduced to map relevancy scores to expected end-user interaction scores, then cross-model score comparison becomes possible, but the system complexity increases
Solution Approach 1:
The system introduces normalization mapping functions that transform relevancy scores into expected end-user interaction scores through parameter changes. These functions learn the relationship between raw scores and actual user interactions from training data, enabling accurate cross-model comparison while maintaining a relatively simple system architecture through efficient mathematical transformations.
Data Source
AI summary
Computer-implemented techniques encompass using distinct machine learning sub-models to score respective types of candidate content for the purpose of providing personalized content suggestions to end-users of a content management system. The relevancy scores generated by the distinct sub-models are mapped to expected end-user interaction scores of the candidate content scored. Content suggestions are provided at end-users' computing devices where the suggested content is selected from the candidate content based on the expected end-user interaction scores of the candidate content. For each distinct sub-model, a normalizing mapping function is solved using an optimizer that maps the relevancy scores generated by the sub-model for the candidate content to expected end-user interaction scores for the candidate content. The expected end-user interaction scores are comparable across the distinct sub-models and can be used to rank content suggestions across the distinct sub-models.