The invention provides a multi-
modal model training method and device. The method comprises the following steps: training each
decomposition matrix in a multi-
modal model by using training data until the loss value of a preset
loss function is smaller than a first preset threshold value, and obtaining a trained multi-
modal model; wherein the parameter matrix of the multi-modal model comprises a basic matrix and an increment matrix of each layer, the
decomposition matrix is obtained by decomposing the increment matrix, one
decomposition matrix corresponding to the increment matrix is a same-order matrix, other decomposition matrixes are non-same-order matrixes, and the total parameter quantity of the decomposition matrixes is smaller than the parameter quantity of the increment matrix. According to the method, the decomposition matrix with a small parameter quantity in the multi-modal model is trained, the
fine tuning training time is shortened, the training efficiency of the model is improved, the demand of hardware
video memory resources is reduced, the training cost is reduced, the precision improvement of downstream tasks of the multi-modal model is completed by using a small amount of
video memory resources and
training time, and the training efficiency of the multi-modal model is improved. And the method has relatively strong generalization ability.