Prompt learning method and device based on unsupervised knowledge distillation

By employing a two-stage training method based on unsupervised knowledge distillation and utilizing a multi-level alignment loss function to transfer discriminative knowledge from the teacher model to the student model, the problem of insufficient generalization ability of visual language models in new domains and unseen categories is solved, achieving efficient model adaptation and generalization performance improvement.

CN122154979APending Publication Date: 2026-06-05INST OF COMPUTING TECH CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
INST OF COMPUTING TECH CHINESE ACAD OF SCI
Filing Date
2026-03-19
Publication Date
2026-06-05

AI Technical Summary

Technical Problem

Existing visual language model adaptation methods based on cue learning rely on a small number of labeled samples, which are prone to overfitting and have insufficient generalization ability. Furthermore, the introduction of complex structures or external models leads to high computational costs and fails to fully utilize the semantic information in unlabeled samples.

Method used

A two-stage training strategy based on unsupervised knowledge distillation is adopted. The supervised fine-tuning stage optimizes the learnable cue parameters of the teacher model, and the discriminative knowledge of the teacher model is transferred to the learnable cue parameters of the student model through the unsupervised distillation stage. Multi-level alignment loss functions are used to constrain the consistency of the prediction distribution, including instance-level, batch-level and relation-level alignment loss functions.

Benefits of technology

Without increasing computational costs, it significantly improves the generalization performance of visual language models in new domains and unseen categories, enhancing the model's adaptability and generalization ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122154979A_ABST
    Figure CN122154979A_ABST
Patent Text Reader

Abstract

The application provides a prompt learning method based on unsupervised knowledge distillation, comprising: a supervised fine-tuning stage, taking a first visual language model as a teacher model, freezing first pre-training parameters of the teacher model, and performing supervised fine-tuning on the teacher model through labeled samples to optimize first learnable prompt parameters of the teacher model; an unsupervised distillation stage, taking a second visual language model as a student model, freezing second pre-training parameters of the student model, aligning inference results of the student model and the teacher model on unlabeled samples, and migrating discriminative knowledge of the teacher model to second learnable prompt parameters of the student model. The application also provides a prompt learning device based on unsupervised knowledge distillation, a storage medium and an electronic device. Therefore, the application significantly improves the adaptation effect of the visual language model on downstream tasks, improves the generalization performance of the visual language model, and has low training and inference costs.
Need to check novelty before this filing date? Find Prior Art