Multi-Task Conversion Prediction for Sparse Ad Interaction Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional recommender systems face challenges in accurately predicting user conversions due to discrepancies between first-party and third-party reporting, high inference costs, data sparsity, negative transfer, and inability to capture complex user interactions, leading to inconsistent performance metrics and reduced return on investment for advertisers.
Innovation Solution
A multi-task model integrating click-through and view-through conversions using a modified Deep & Cross Network (DCN) v2 with layer normalization and Multi-gate Mixture-of-Experts (MMoE) architecture, along with Progressive Layered Extraction (PLE), to enhance feature interaction and capture diverse user behaviors.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional recommender systems are used to predict user conversions, then the system is simple to implement, but the prediction accuracy is low due to discrepancies between first-party and third-party reporting
Solution Approach 1:
The patent merges first-party and third-party conversion reporting systems into a unified multi-task model that processes both data sources simultaneously. This integration resolves the discrepancies between the two reporting methods by learning their relationships through shared neural network layers and task-specific output heads, thereby improving conversion prediction accuracy while managing complexity through architectural design.
Solution Approach 2:
The multi-task model serves multiple functions: it processes first-party conversion data, third-party conversion data, and user interaction data simultaneously through a single unified architecture. This multi-functional approach eliminates the need for separate prediction systems for different data sources, improving overall prediction accuracy while reducing the complexity of maintaining multiple separate models.
2Reliability
If separate models are used for first-party and third-party conversions, then each model can be optimized for its specific data source, but the inference costs are high
Solution Approach 1:
The patent combines first-party and third-party conversion prediction into a single multi-task model with shared neural network layers. This merging allows the system to maintain specialized prediction capabilities for each data source while sharing computational resources, thereby reducing inference costs compared to running separate models while preserving prediction reliability through task-specific output layers.
Solution Approach 2:
The unified model performs multiple prediction tasks (first-party conversion, third-party conversion, and user interaction prediction) simultaneously through a single inference pass. This multi-functionality reduces the total computational overhead and inference costs compared to executing multiple separate models, while maintaining reliable predictions through specialized loss functions and output heads for each task.
3Measurement precision
If simple models are used for conversion prediction, then the model is fast to train, but the model cannot capture complex user interactions
Solution Approach 1:
The patent segments the model architecture into shared layers for learning common user interaction patterns and task-specific layers for capturing complex conversion behaviors. This segmentation allows the model to capture complex user interactions through specialized components while keeping the overall training process efficient by sharing computational work across multiple tasks through the common layers.
Solution Approach 2:
The multi-task model simultaneously learns from multiple data sources and prediction targets, making more efficient use of training data and reducing training time compared to separate models. The shared architecture captures complex user interactions across different tasks while the unified training process leverages common patterns, reducing overall training time despite the increased model capacity.
4Measurement precision
If conventional single-task models are used, then the model architecture is simple, but the model suffers from data sparsity and negative transfer
Solution Approach 1:
The patent merges multiple single-task models into a unified multi-task model that shares neural network layers. This merging addresses data sparsity by pooling data from multiple sources (first-party conversions, third-party conversions, user interactions) to train a more robust model. It prevents negative transfer through shared representation learning that captures common patterns across tasks while maintaining task-specific prediction capabilities through separate output heads.
Data Source
AI summary
The systems and techniques described herein relate to predicting user conversions in online advertising. Input data associated with user and advertisement features may be processed through neural networks to generate embedding representations or feature cross representations. A multi-task layer calculates probabilities associated with multiple user actions like clicks, page views, sign-ups, or purchases. Click-through and view-through conversion probabilities may be calculated to generate a score. The systems and techniques described herein perform predictions on multiple types of user actions despite data sparsity and negative transfer challenges, enhancing advertisement targeting and improving conversion metrics.