Multi-Domain User Profiling via Pre-Trained Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for user profiling often rely on single-domain models, which fail to provide a holistic understanding of users due to the lack of consideration for interactions between different characteristics and limited datasets, leading to poor performance in tasks such as behavior prediction.
Innovation Solution
A method that combines raw data from multiple domains using pre-trained embeddings and fine-tuned models, leveraging self-attention mechanisms to generate multi-domain behavior records and improve task performance by considering interactions between different domains.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If single-domain models are used for user profiling, then the model complexity is reduced and training is simpler, but the understanding of user characteristics becomes incomplete and task performance deteriorates
Solution Approach 1:
The patent merges multiple single-domain models into a unified multi-domain model that processes data from different domains (e.g., social media, e-commerce, mobility) simultaneously. This combination allows the model to capture holistic user characteristics and interactions between different user attributes, resolving the contradiction by achieving complete user understanding without proportionally increasing complexity through integrated architecture design.
Solution Approach 2:
The patent creates a universal user profiling model that can handle multiple domains and tasks through a shared embedding layer and common model structure. This multi-functional model serves different domains (social, shopping, mobility) while maintaining domain-specific capabilities, thereby improving user understanding accuracy without requiring separate complex models for each domain.
2Reliability
If multiple domains are integrated for holistic user modelling, then the understanding of user characteristics improves, but the model complexity and data processing requirements increase
Solution Approach 1:
The patent segments the multi-domain model into distinct domain-specific modules (e.g., social domain module, e-commerce domain module, mobility domain module) that each process data from their respective domains. These segmented modules feed into a shared embedding layer, allowing holistic user understanding while maintaining manageable complexity through modular architecture that can be developed and trained independently.
Solution Approach 2:
The patent introduces an embedding layer as an intermediary that transforms data from multiple domains into a unified representation space. This embedding layer acts as a mediator that reconciles different domain data formats and features, enabling holistic user understanding without directly complexifying the overall model structure by providing a standardized interface between domains and the core model.
3Quantity of substance
If datasets are limited for a specific domain, then data collection costs are reduced, but the model performance for that domain deteriorates
Solution Approach 1:
The patent performs preliminary training of the multi-domain model using abundant data from domains with rich data availability (e.g., social media data) before fine-tuning on the target domain with limited data. This preliminary action allows the model to learn general user representation patterns from data-rich domains, which then transfers to improve performance in data-scarce domains without requiring large volumes of target domain data.
Solution Approach 2:
The patent changes the training parameters and data distribution across domains during the training process, using weighted sampling or focal loss to emphasize the target domain with limited data while still benefiting from data-rich domains. This parameter adjustment allows the model to adapt to data volume differences and maintain high performance even when the target domain has limited training data.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An embodiment of a method (800) for learning a first task in a first domain comprises receiving (801) a first raw data of the first domain and a second raw data of a second domain characterizing a behavior of a user. The method further comprises determining (802) a pre-trained embedding using the first raw data and the second raw data and using (803) the pre-trained embedding for learning a first fine-tuned model to perform the first task.