Personalized LLM Fine-Tuning Using User Activity Representations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) lack personalization due to general-purpose training on vast user-agnostic datasets, making it difficult to tailor responses to individual user contexts and preferences.
Innovation Solution
A personalized response system is developed by finetuning a machine-learned language model using user-specific training examples, including user requests, activities, and satisfaction scores, and applying an evaluation model to generate personalized responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a large language model is trained on vast user-agnostic training datasets for general purposes, then the model achieves comprehensive knowledge coverage and general-purpose capability, but the model loses the ability to personalize responses to individual user contexts and preferences
Solution Approach 1:
The patent segments the training process into two distinct phases: (1) pre-training on vast user-agnostic datasets to build comprehensive knowledge, and (2) fine-tuning on user-specific data to add personalization capability. This segmentation allows the model to maintain general knowledge while acquiring personalized response abilities without requiring complete retraining.
Solution Approach 2:
The patent applies preliminary action by first establishing a robust base model through extensive pre-training on general datasets before introducing user-specific personalization. This preliminary foundation enables the model to efficiently adapt to individual users during the fine-tuning phase, as the core language understanding and generation capabilities are already in place.
2Reliability
If a concierge system uses a general-purpose large language model, then the system can handle diverse customer service tasks, but the system cannot effectively comprehend and meet individual customer requests and preferences
Solution Approach 1:
The patent applies local quality by integrating user-specific representations into the general-purpose LLM framework. Each user receives personalized attention through learned user embeddings and context-specific fine-tuning, allowing the system to maintain high service quality across diverse tasks while adapting to individual user preferences and contexts locally.
Solution Approach 2:
The patent implements feedback mechanisms where user interactions and responses are used to continuously refine and personalize the model's behavior. The system learns from user feedback during fine-tuning, adjusting its responses to better match individual user preferences while maintaining overall service reliability across different customer scenarios.
3Adaptability or versatility
If the training dataset includes different data sources without incorporating user context, then the model achieves broad knowledge coverage, but the model cannot generate personalized responses tailored to individual user preferences
Solution Approach 1:
The patent applies dynamics by making the training process adaptive and flexible. Instead of requiring static, exhaustive training data for every possible user scenario, the system dynamically fine-tunes on user-specific data after initial pre-training. This dynamic approach allows the model to personalize responses effectively while keeping the base training dataset manageable in size.
Data Source
AI summary
A computer system finetunes a machine-learned language model to generate a personalized response to a user request. The system may generate a user representation for each of a plurality of users by applying a transformer model to a sequence of tokens representing a sequence of activities of the user. The system may train an evaluation model coupled to receive a user representation and a response to a user request and generate an estimated evaluation score indicating a level of personalization of the response to the user. The system may finetune a first machine-learned language model to generate a second machine-learned language model. The finetuned machine-learned language model is configured to provide personalized responses for customer services at an online concierge system.