Dynamic Aspect Prompting Framework for Language Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional language models face limitations in prompt-based learning due to finite and context-independent prompt templates, which restrict their ability to capture domain-specific knowledge and can lead to biased outputs, especially when dealing with complex tasks like information extraction and text analysis.
Innovation Solution
The development of a dynamic aspect-based prompting framework that extracts relevant aspects from training data to generate contextual prompts, allowing language models to learn and perform tasks more accurately by formulating prompts as masked language modeling problems, thereby improving in-context learning and reducing training time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If conventional prompt templates are used, then the prompt structure is simple and easy to implement, but the ability to capture domain-specific knowledge is limited and outputs may be biased
Solution Approach 1:
The patent transforms static, fixed prompt templates into dynamic prompts by extracting aspects from training data and generating prompts that adapt to the specific context. The prompting function dynamically selects and generates prompts based on the input text and learned aspects, enabling the model to capture domain-specific knowledge while maintaining flexibility and reducing bias.
Solution Approach 2:
The patent changes the parameters of prompt generation by using aspect extraction to create contextualized prompts. Instead of using fixed templates, the system generates prompts based on extracted aspects (subject, topic, entity) which serve as parameters that vary with the input data, thereby improving adaptability to different domains and contexts.
2Measurement precision
If more training data is used to improve accuracy, then the model performance improves, but the training time and computational resources increase
Solution Approach 1:
The patent extracts relevant aspects from training data (subject, topic, entity) and uses these extracted aspects to generate prompts that encapsulate domain-specific knowledge. This extraction allows the model to learn from a smaller amount of data by focusing on the most important contextual information, thereby reducing training time while maintaining or improving accuracy.
Solution Approach 2:
The system performs preliminary aspect extraction and prompt generation before the main training process. By pre-processing the training data to extract aspects and generate contextualized prompts, the model can more efficiently learn from the data during training, reducing overall training time while improving performance.
3Adaptability or versatility
If fixed prompt templates are used, then the implementation is straightforward, but the prompts cannot adapt to different contexts and may introduce biases
Solution Approach 1:
The system enables self-service prompt generation by automatically extracting aspects from the input text and generating appropriate prompts without requiring manual intervention. The model uses its own learned aspects to dynamically generate context-adapted prompts, making the process autonomous and reducing the need for manual prompt engineering while improving context adaptability.
Data Source
AI summary
Techniques for dynamically developing a contextual set of prompts based on relevant aspects extracted from s set of training data. One technique includes obtaining training data comprising text examples and associated labels, extracting aspects from the training data, generating prompting templates based on the training data and the extracted aspects, concatenating each of the text examples with the respective generated prompting template to create prompting functions, training a machine learning language model on the prompting functions to predict a solution for a task, where the training is formulated as a masked language modeling problem with blanks of the prompting templates being set as text labels and expected output for the task being set as specified solution labels, and the training learns or updates model parameters of the machine learning language model for performing the task. The machine learning language model is provided with the learned or updated model parameters.