Layered self-adaptive freezing fine tuning method and device for multi-modal large model

Through the hierarchical adaptive frozen fine-tuning method, the parameter importance is dynamically evaluated and binary masks are generated, which solves the problems of knowledge degradation and computational overhead in model fine-tuning and realizes efficient, lightweight and privacy-safe continuous learning of multi-task capabilities.

CN120781879AInactive Publication Date: 2025-10-14SHANGHAI ARTIFICIAL INTELLIGENCE INNOVATION CENT
0 Cites 1 Cited by

Patent Information

Application Number
CN202510877655.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-06-27
Publication Date
2025-10-14
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Existing technologies suffer from significant degradation of the original pre-trained knowledge due to overfitting to new task data during model fine-tuning, making it difficult to effectively apply them in multi-task and dynamic incremental learning scenarios. Traditional methods also rely on old task data or generate pseudo data, which results in privacy leakage and computational overhead.

Method used

A hierarchical adaptive freezing and fine-tuning method for multimodal large models is adopted to obtain a domain expert model through training on the full dataset. The global unfreezing parameter quota is allocated hierarchically, the importance of parameters is dynamically evaluated and binary masks are generated, and key training parameters are sparsely trained to avoid dependence on old task data and noise generation.

Benefits of technology

It achieves efficient, lightweight, and privacy-safe continuous learning in open task scenarios, reduces computational overhead, improves the model's adaptability and generalization ability in new tasks, and prevents catastrophic forgetting.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence, discloses a hierarchical adaptive freezing fine tuning method and device for a multi-modal large model, and particularly discloses the method comprising the following steps: training a pre-trained multi-modal large model according to a total data set subset to obtain a field expert model; dividing the domain expert model into a plurality of structural layers, and distributing global unfreezing parameter quotas for each structural layer according to contribution degrees; and in each structural layer, performing adaptive importance sorting on each parameter, screening out parameters consistent with the corresponding global unfreezing parameter quota quantity according to an importance sorting sequence to generate a binary mask, and freezing other parameters. And finally, performing sparse training on the domain expert model according to the binary mask. According to the method, the dependence of traditional data playback on old task data or pseudo data generation is overcome, task boundary presetting or manual intervention is not needed, and efficient, lightweight and privacy-safe continuous learning is realized while the generalization ability of the pre-training model is protected.
Need to check novelty before this filing date? Find Prior Art