Instruction-Tuning Data Set Selection for Zero-Shot Task Relevance

The patent applies preliminary action by evaluating task similarity before the actual training process. The system calculates similarity scores between target tasks and candidate training tasks using instruction embeddings and cosine similarity, then pre-selects relevant training tasks. This preliminary selection prevents negative transfer from irrelevant tasks, ensuring that only high-similarity tasks are used for training, thus maintaining both training efficiency and zero-shot learning performance.

The patent applies the extraction principle by isolating and selecting only the relevant components (training tasks) from the entire task pool. The system extracts instructions from training tasks that have similarity scores above a threshold, discarding irrelevant tasks. This selective extraction maintains model coverage on relevant tasks while significantly reducing computing resource consumption by excluding unrelated tasks from the training process.

The patent applies partial action by implementing a two-stage similarity evaluation process. First, a quick cosine similarity calculation is performed on instruction embeddings to filter out obviously irrelevant tasks. Then, the computationally intensive model transfer method is applied only to the remaining candidate tasks that passed the initial filter. This partial application of the expensive evaluation method maintains high task relevance accuracy while minimizing additional training time.