The invention discloses a
large model training method based on second-order
matrix optimization, and belongs to the field of
deep learning model training technology optimization, and the
large model training method based on second-order
matrix optimization comprises the following steps: S1, decomposing a second-order matrix into row and column vectors, and carrying out sliding average and distributed block reduction storage; s2, generating a statistical
row vector by combining row gradient aggregation with a historical
attenuation factor; s3, carrying out block distributed statistics in the column direction and synchronously generating column vectors across equipment; s4, a low-rank matrix is constructed through outer products of row and column vectors, and
estimation precision is improved through
noise suppression; s5, performing dynamic sparse sampling, performing initial high-density focusing, and stabilizing the sampling rate of a key layer; s6, the sampling points execute
time sequence attenuation updating, and asynchronous calculation is carried out to improve the
resource utilization rate; s7, performing
Gaussian kernel
smoothing on a neighborhood value compensation coverage gap in an unsampled region; s8, fusing low-rank
estimation and sparse data, and balancing global precision by self-adaptive weight; the method has the beneficial effects of reducing
video memory occupation, and improving
distributed computing efficiency and
balance training precision and speed.