A multi-subtask decoupling-oriented large model cooperative fine-tuning method and device and storage medium

By introducing a shared-dedicated parameter hierarchical design and output control token into the large model, the problems of parameter competition and uncontrollable response in multi-task fine-tuning in vertical domains are solved, achieving decoupled learning and output controllability of multi-tasks, and improving the stability and adaptability of the model.

CN122154827APending Publication Date: 2026-06-05ZHEJIANG UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
ZHEJIANG UNIV
Filing Date
2026-01-14
Publication Date
2026-06-05

AI Technical Summary

Technical Problem

Existing fine-tuning of large vertical domain models suffers from issues such as parameter competition and knowledge conflict between tasks, as well as uncontrollable response behavior, leading to performance fluctuations and unstable output in multi-task scenarios.

Method used

A multi-level parameter structure is adopted to separate shared parameters from task-specific parameters, and an output compliance control token is introduced. The task routing module enables independent parameter updates and controllable switching of output modes, respectively learning to follow the model's inherent knowledge and user prompts.

Benefits of technology

It improves the stability and scalability of the model in multi-task scenarios, reduces training overhead and deployment costs, and achieves controllability and interpretability of output behavior.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122154827A_ABST
    Figure CN122154827A_ABST
Patent Text Reader

Abstract

The application discloses a multi-subtask decoupling-oriented large model cooperative fine-tuning method and device and a storage medium, and comprises the following steps: constructing a multi-level parameter structure on the basis of a pre-trained large model; the multi-level parameter structure comprises shared parameters and a plurality of task-specific parameters, wherein each task-specific parameter is attached to a network layer corresponding to the shared parameters in a low-rank adaptive form; performing multi-task joint training, wherein the data of different inference tasks respectively drive the update of the corresponding task-specific parameters, and the shared parameters remain frozen or are only slightly fine-tuned; introducing an output compliance control Token during the training process; in the inference stage, the corresponding task-specific parameters are selected according to the task category of the input sample through a task routing module, and the output preference is determined according to the control Token carried in the input, so that the model generates inference results meeting the target constraints. The application can improve the stability, scalability and controllable generation capability of the model in the multi-task inference scene.
Need to check novelty before this filing date? Find Prior Art