The invention provides a CDVFT
fine tuning method based on cyclic matrix inversion. The CDVFT
fine tuning method is efficient in memory. The CDVFT
fine tuning method comprises the following steps: acquiring an input
column vector and a weight increment; performing forward
processing based on the input
column vector and the weight increment to obtain an intermediate calculation result and forward output, and only storing the forward output in the memory in the model training stage; during the period of back propagation during training, carrying out
inverse operation on
forward propagation on forward output to obtain a
tensor required by gradient calculation; carrying out
inverse operation on element-by-element multiplication of a product of an
intermediate state vector and a
diagonal matrix vector by using an element-by-element division method; the calculation of the
intermediate state vector and the cyclic matrix vector is based on
inverse operation of cyclic matrix inversion. According to the method, the
tensor stored in the
forward propagation period of fine tuning is reduced by using the characteristic that the cyclic matrix can be used for rapid inversion, and the
tensor is recovered in the reverse propagation period, so that the peak memory occupation in the fine tuning process is reduced.