Few-Shot Multi-Task Recommender System for B2B Data Sparsity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional recommender systems in B2B contexts face challenges such as static and non-holistic data sources, data sparseness, the cold start problem, and inability to incorporate dynamic context and varied optimization goals.
Innovation Solution
A recommender system that enables web crawling, utilizes third-party data connectors, executes machine learning model training, and builds models for various B2B recommendation use cases, incorporating features like intelligent data enrichment, few-shot learning, multi-modal embedding, context integration, and multi-task evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional recommender systems use static third-party data sources, then the system structure is simple, but the recommendation accuracy is diminished and data is not holistic
Solution Approach 1:
The patent combines multiple data sources including structured data from third-party connectors, unstructured data from web crawling, and internal enterprise data into a unified data lake. This merging of diverse data sources provides a holistic view of enterprises, transforming static limited data into dynamic comprehensive data, thereby improving recommendation accuracy while accepting increased system complexity.
Solution Approach 2:
The system implements a universal data lake architecture that serves multiple recommendation tasks and use cases. The same data infrastructure supports various recommendation scenarios (product recommendations, supplier recommendations, etc.), allowing the system to handle diverse data types and tasks through a single multi-functional platform, improving accuracy across multiple applications.
2Measurement precision
If traditional recommender systems rely on large amounts of historical training data, then model performance may be improved, but the system cannot handle cold start problems and data sparseness
Solution Approach 1:
The system performs preliminary actions by continuously crawling and storing enterprise data, product data, and interaction data in the data lake before actual recommendation tasks occur. This pre-collection and pre-processing of data ensures that when cold start scenarios occur (new enterprises or products with no interaction history), the system already has baseline data to work with, enabling it to provide recommendations without requiring extensive historical interaction data.
Solution Approach 2:
The patent introduces an intermediary embedding layer that transforms raw data from diverse sources into unified vector representations. This embedding space acts as a mediator that can represent entities even with limited data, allowing the system to handle cold start problems by mapping new entities into the existing embedding space using their basic attributes and crawled information, rather than requiring extensive historical interaction data.
3Ease of manufacture
If traditional ML models use incomplete and noisy training data, then training is simplified, but the models easily overfit and predictions become sub-optimal
Solution Approach 1:
The system performs preliminary data cleaning, validation, and enrichment actions before training models. The data lake architecture pre-processes and stores cleaned, validated, and enriched data with proper schemas and quality checks. This preliminary processing ensures that when models are trained, they receive high-quality data, reducing overfitting and improving prediction reliability while maintaining training simplicity through the pre-processed data structure.
Solution Approach 2:
The system implements feedback mechanisms where model performance is continuously monitored and used to improve data quality and processing. The data lake includes feedback loops that refine data collection, cleaning, and feature engineering based on model performance metrics. This feedback ensures that training data quality continuously improves, preventing overfitting and enhancing prediction reliability while keeping the training process manageable.
4Measurement precision
If traditional recommender systems do not incorporate dynamic context, then the system is simpler to implement, but ML models become sub-optimal in changing B2B contexts
Solution Approach 1:
The patent implements dynamic context incorporation by continuously updating the data lake with fresh data from web crawling, third-party connectors, and enterprise interactions. The context integration module dynamically adjusts recommendations based on changing contexts such as enterprise needs, market conditions, and technological factors. This dynamic approach improves prediction accuracy by ensuring recommendations reflect current realities, while the modular architecture manages system complexity through organized data flow and processing stages.
5Adaptability or versatility
If traditional recommender systems are designed for single optimization goal, then the system architecture is simpler, but it cannot provide recommendations based on different optimization goals and scenarios
Solution Approach 1:
The system implements a universal recommendation engine that can handle multiple optimization goals (contribution margin, coverage, revenue, etc.) through a single multi-functional architecture. The data lake and model framework are designed to support various recommendation scenarios and optimization objectives simultaneously, allowing the system to adapt to different business requirements without requiring separate systems for each goal, thus achieving versatility while controlling complexity through shared infrastructure.
Data Source
AI summary
Implementations are directed to receiving a set of tuples, each tuple including an entity and a product from a set of products, for each tuple: generating, by an embedding module, a total latent vector as input to a recommender network, the total latent vector generated based on a structural vector, a textual vector, and a categorical vector, each generated based on a product profile of a respective product and an entity profile of the entity, generating, by a context integration module, a latent context vector based on a context vector representative of a context of the entity, and inputting the total latent vector and the latent context vector to the recommender network, the recommender network being trained by few-shot learning using a multi-task loss function, and generating, by the recommender network, a prediction including a set of recommendations specific to the entity.