Large-scale continuous learning methods and related devices based on multi-granularity knowledge transfer

Abstract

This invention discloses a method and related apparatus for continuous learning of large models based on multi-granularity knowledge transfer, belonging to the field of artificial intelligence and deep learning technology. It constructs a set of fine-grained knowledge units and a set of coarse-grained knowledge prototypes. The coarse-grained knowledge prototypes guide the selection and replay of fine-grained knowledge units. An expanded training set is constructed using the replay sample set, and then the large model is fine-tuned. During the fine-tuning process, prototype consistency regularization and parameter stability regularization are jointly applied. The finely-tuned large model is then used to perform prediction tasks. Through multi-granularity knowledge transfer of fine and coarse-grained knowledge and dual regularization constraints, the problem of knowledge forgetting caused by dynamic changes in data distribution can be effectively alleviated, avoiding damage to the model's existing predictive capabilities, thereby ensuring the adaptability, stability, and predictive accuracy of the large model during continuous learning.

Images