Attention Layers for Real-Time Bidding Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning models for predicting online user activity and determining bid amounts in advertising inventory auctions struggle with accurately capturing sequential context and are costly to train, with limitations including short lookback periods and lack of explainability.
Innovation Solution
The use of attention layers in machine learning models to analyze historical user activity data and bid response data, enabling real-time bidding algorithms to effectively target consumers likely to convert by understanding correlations between previous actions and click or conversion likelihoods.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional machine learning models are used to predict user activity and determine bid amounts, then the system can process advertising inventory auctions, but the models struggle to accurately capture sequential context and have limited lookback periods
Solution Approach 1:
The patent transforms the model architecture by incorporating attention mechanisms that dynamically adjust parameter weights based on sequential importance. The attention layers compute context-dependent weights for different historical time steps, allowing the model to focus on relevant past events while maintaining a long lookback period. This resolves the contradiction by changing how parameters are utilized rather than increasing overall model complexity.
Solution Approach 2:
The patent segments the sequential processing into multiple attention layers that independently analyze different aspects of historical user behavior. Each attention layer focuses on specific temporal patterns or feature interactions, dividing the complex sequential modeling task into manageable segments. This enables accurate capture of sequential context without requiring a monolithic complex model.
2Measurement precision
If complex machine learning models with long lookback periods are used to improve prediction accuracy, then conversion attribution improves, but computational training costs increase significantly
Solution Approach 1:
The patent implements partial attention mechanisms that selectively process only the most relevant historical time steps rather than uniformly processing all past data. The attention scores identify and focus computational resources on key moments in user journey that most influence conversion likelihood, performing partial processing of the full historical sequence. This reduces computational burden while maintaining prediction accuracy by concentrating resources on critical information.
3Ease of operation
If traditional models are used for real-time bidding, then the system operates in real-time, but the models lack explainability regarding which historical events drove predictions
Solution Approach 1:
The patent introduces attention weights as intermediary variables that bridge the black-box prediction process and interpretable historical event attribution. These attention scores serve as mediators that quantify the influence of each historical time step on the current prediction, providing explainability without slowing down real-time inference. The attention mechanisms act as interpretable layers that reveal which past events drove the prediction while maintaining real-time operational speed.
Data Source
AI summary
The subject technology identifies a series of journey event types in an online user journey, the event types including an impression event, an email event, a click event, and a website visit, and assigns an encoder to each event type. Using an assigned encoder, the technology encodes each event type to generate an encoded vector for each event type. The encoded vector is representative of at least a portion of the online user journey relating to that event type. The technology generates an encoded vector for each event type to create a set of encoded vectors, the set of encoded vectors including one or more of an impression event encoded vector, an email event encoded vector, a click event encoded vector, and a website visit encoded vector. The technology aggregates the set of encoded vectors to generate an output of the online user journey encoder, the output including a composite encoded user journey vector for training one or more attention layers to make a prediction.