Large language model training method and target classification model generation method and device

By generating a set of synthetic instances and processing them using a target classification model, the problem of forgetting old knowledge in large language models during continuous learning is solved. This enables new task learning and old task memorization in the absence of real data, improving the flexibility and adaptability of the model.

CN121660025APending Publication Date: 2026-03-13ALIBABA CLOUD COMPUTING CO LTD
0 Cites 0 Cited by

Patent Information

Application Number
CN202411282852.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-12
Publication Date
2026-03-13

AI Technical Summary

Technical Problem

Large language models are prone to catastrophic forgetting during continuous learning and cannot keep up with both new and old knowledge when real training data is unavailable.

Method used

The model is trained by generating a set of synthetic instances, which are then processed and classified using a target classification model. This results in a high-quality and diverse set of synthetic instances, reducing reliance on real training data. The model is trained by combining synthetic instances from historical training and target training tasks.

Benefits of technology

It improves the flexibility and adaptability of the model in continuous learning, maintains memory of old tasks, improves data utilization efficiency and model generalization ability, and solves the problem of catastrophic forgetting.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention discloses a large language model training method and device and a target classification model generating method and device. The method comprises the steps of determining a target training task and at least one historical training task to be subjected to memory training; obtaining a first synthetic instance set corresponding to the target training task; obtaining a second synthetic instance set corresponding to the historical training tasks in the at least one historical training task, and determining a target synthetic instance set according to the first synthetic instance set and the second synthetic instance set; and training a to-be-trained large language model according to the target synthesis instance set to obtain a target large language model. The technical problem that old knowledge of the model is maintained depending on real training data of a large language model in related technologies, and new and old knowledge of the large language model cannot be considered when the real training data is unavailable is solved.
Need to check novelty before this filing date? Find Prior Art