Clustering Prompts for Language Model Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for constructing training data for language models are inefficient, leading to high annotation workloads, costs, and randomness in prompt sampling, which can result in language model outputs that do not meet user preferences.
Innovation Solution
The method involves clustering prompts in a training dataset to determine multiple clusters, calculating cohesion levels to set sampling probabilities, and selecting a target cluster for sampling to construct valuable target training data, thereby reducing annotation workload and improving language model outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If random prompt sampling is used to construct training data, then the process is simple and fast, but the sampled prompts are random and may not represent diverse or valuable topics, leading to insufficient annotation quality
Solution Approach 1:
The patent segments the training data into multiple clusters based on prompt topics or themes. By dividing the dataset into distinct clusters, the system can then selectively sample from specific clusters rather than performing random sampling across the entire dataset, ensuring both efficiency and representativeness.
Solution Approach 2:
The patent applies local quality by differentiating sampling strategies for different clusters. Instead of uniform random sampling, the system uses targeted sampling within specific clusters that have higher cohesion levels or are more relevant to the training objectives, thereby improving the representativeness and quality of sampled prompts.
2Reliability
If all prompts in the training dataset are annotated, then the training data is comprehensive, but the annotation workload and costs are high
Solution Approach 1:
The patent extracts and selects only the most valuable prompts for annotation by identifying clusters with high cohesion levels or specific topic relevance. This extraction approach allows the system to annotate a subset of high-quality prompts that represent the diverse and valuable topics, reducing annotation workload while maintaining training data comprehensiveness.
Solution Approach 2:
The patent performs preliminary clustering and cohesion level calculation before the annotation process. This preliminary action identifies and prioritizes which prompts should be annotated first, allowing annotators to focus on the most valuable prompts and reducing overall annotation time and costs.
3Measurement precision
If prompts are clustered and sampled based on cohesion levels, then the sampled prompts are more diverse and valuable, but the process complexity increases
Solution Approach 1:
The patent implements self-service by using the language model itself to perform clustering and cohesion level calculation. The model processes the training data autonomously to identify clusters and their cohesion levels, eliminating the need for complex external processing systems and reducing overall process complexity while maintaining high sampling quality.
Data Source
AI summary
Embodiments of the present disclosure relate to a method, an electronic device, and a computer program product for constructing training data. The method includes determining multiple clusters by clustering prompts in a training dataset; and determining, based on multiple cohesion levels of the multiple clusters, multiple sampling probabilities corresponding to the multiple clusters, where the cohesion levels indicate intra-cluster distances in the clusters. The method further includes determining, according to the multiple sampling probabilities, a target cluster for sampling. The method further includes constructing target training data by sampling target prompts from the target cluster. According to embodiments of the present disclosure, when fine-tuning a language model, prompts can be screened according to a clustering result of the prompts, so as to make the determined prompts more valuable for annotation, thereby ensuring output results of the language model obtained by training to be comprehensive and diverse.