The invention provides a
large model calculation method and device oriented to a low GPU resource card group calculation
system. The method comprises the steps that parameters of a part of a large
language model are unloaded into a CPU memory in the GPU resource card group calculation
system; obtaining a reasoning request, wherein the reasoning request comprises a
batch processing size and a sequence length; in the cue
word processing stage, whether the sequence length is larger than two times of the dimension of a
hidden layer of the large
language model or not is judged, and if yes, the calculation intensity of all attention
layers and all multi-layer
perceptron of the large
language model is calculated; judging whether the calculation intensity of all the attention
layers is greater than that of the multi-layer
perceptron or not, if yes, unloading all the attention
layers in a GPU
video memory and unloading all the multi-layer
perceptron in a CPU memory, and if not, unloading all the multi-layer perceptron in the GPU
video memory and unloading all the attention layers in the CPU memory; and using the unloaded GPU resource card group computing
system to run the large language model for reasoning to obtain a reasoning result of the reasoning request.