Clickstream Embeddings for Sparse Real-Time Session Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
User interactions with websites or mobile applications are primarily analyzed in batches, making real-time interaction data modeling difficult due to its sparseness, which hinders timely insights and predictions.
Innovation Solution
A self-supervised learning process trains an artificial intelligence model, such as a transformer model, to generate embeddings representing real-time user interactions, enabling real-time analysis and prediction by reconstructing interactions using an encoder and decoder model without labeled data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If batch analysis is used for user interactions, then processing stability is maintained, but real-time insights and predictions are delayed
Solution Approach 1:
The system pre-trains an embedding model offline using historical user interaction data and self-supervised learning techniques. This pre-trained model captures fundamental user behavior patterns and is then deployed for real-time inference, allowing the system to maintain modeling complexity at a manageable level while achieving real-time analysis capabilities through the separation of offline training and online prediction.
Solution Approach 2:
The patent replaces traditional batch processing mechanical systems with a neural embedding model that operates on sparse real-time data. Instead of requiring large volumes of historical data for accurate modeling, the system uses a learned embedding representation that can process individual real-time interactions efficiently, substituting the mechanical batch-processing approach with an intelligent data representation system.
2Measurement precision
If real-time interaction data is modeled, then timely insights are achieved, but data sparseness makes modeling difficult
Solution Approach 1:
The system employs self-supervised learning where the model generates its own training data by predicting future user interactions based on historical patterns. The embedding model is trained to reconstruct user interaction sequences, allowing it to learn meaningful representations from sparse real-time data without requiring external labeled data or large historical datasets, thus serving itself to overcome data sparseness.
Solution Approach 2:
The patent transforms the data representation by changing from raw interaction logs to embedded vector representations. This parameter transformation compresses and enriches sparse interaction data into dense embedding vectors that capture essential user behavior patterns, enabling accurate predictions even when the original data quantity is insufficient.
3Ease of manufacture
If existing user data is used for analysis, then modeling is simplified, but accuracy is reduced when data is sparse or unavailable
Solution Approach 1:
The embedding model serves as an intermediary layer between raw user interaction data and downstream prediction models. It transforms sparse and unstructured interaction data into structured embedding representations that capture user behavior patterns, thereby simplifying the modeling process for downstream tasks while improving prediction reliability by providing enriched feature representations even when original data is sparse.
Data Source
AI summary
Methods and systems are described herein for generating embeddings representing real-time interactions of a user device with a server to be used by a downstream model. For example, the downstream model may be trained to select and provide data to a user of the user device based on the embedding. In some embodiments, an artificial intelligence model may be trained using self-supervised learning to reconstruct real-time interactions of the user during a current user session. The artificial intelligence model, for example, an auto-encoder, may be trained using reference real-time interactions of the user to generate embeddings that can be mapped to predicted reconstructions of the real-time interactions. The artificial intelligence model can be trained by minimizing a loss computed from the reference real-time interactions and the reconstructions of the real-time interactions.