Multi-phase ML Training for Search Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current web search systems face challenges in effectively ranking search results using transformer-based machine learning models, particularly when dealing with non-text data and the need for large datasets that include user click information for training.
Innovation Solution
The method involves training transformer models using query metadata and search relevance data, including click data, in a pretraining phase, followed by refinement with smaller crowd-sourced datasets to improve search result ranking performance, utilizing multiple training phases with different datasets and assessor-generated labels to enhance relevance determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If transformer models are trained on large datasets including non-text click data, then the model's ability to handle diverse search data improves, but the training complexity and computational resources required increase
Solution Approach 1:
The training process is divided into two distinct phases: pre-training on large-scale click data to learn basic patterns, and fine-tuning on smaller labeled relevance datasets to specialize for search ranking. This segmentation allows the model to handle diverse data types while managing training complexity through staged specialization.
Solution Approach 2:
The model performs preliminary pre-training on abundant click data before the actual fine-tuning stage. This preliminary action enables the model to acquire general search behavior patterns from non-text data, reducing the burden on subsequent training with limited labeled relevance data.
2Measurement precision
If transformer models are used for search ranking, then language comprehension and semantic understanding improve, but the computational resources and training time required increase
Solution Approach 1:
Training is segmented into pre-training and fine-tuning phases, allowing semantic understanding to be developed efficiently on large unlabeled data during pre-training, then refined quickly on smaller labeled datasets during fine-tuning, reducing total training time while maintaining accuracy.
Solution Approach 2:
Semantic understanding capabilities are preliminarily established through pre-training on large corpora before fine-tuning for specific search tasks. This preliminary semantic foundation reduces the time required for task-specific training while preserving comprehension accuracy.
3Measurement precision
If large datasets with user click information are used for training, then the model's prediction accuracy improves, but the data processing and storage requirements increase
Solution Approach 1:
The method extracts and utilizes only the essential click data features needed for training, separating useful interaction patterns from the bulk of raw click data. This extraction approach maintains prediction accuracy while reducing the effective data volume that needs to be processed and stored during training.
4Measurement precision
If multiple training phases with different datasets are implemented, then the model's relevance determination performance improves, but the training process complexity increases
Solution Approach 1:
The training process is segmented into pre-training and fine-tuning phases with clearly defined objectives and datasets. This segmentation improves relevance determination by exposing the model to diverse data patterns first, then specializing on labeled relevance data, while managing complexity through phase separation and standardized workflows.
Data Source
AI summary
A method and system for training a machine learning model to rank digital objects generated using a search query are described. The method includes training the machine learning model in a first phase to determine a predicted user interaction parameter, based on a first plurality of training digital objects associated with past user interaction parameters. The machine learning model is then trained in a second phase to determine a synthetic assessor-generated label, based on a second plurality of training digital objects associated with search queries and labeled with human-assigned assessor-generated labels indicative of a relevance of the training digital objects to the queries. The machine learning model may be applied to the first plurality of training digital objects to generate a first augmented plurality of training digital objects, which may then be used to train the machine learning model to determine a relevance parameter for a digital object.