The application discloses a large
language model inference acceleration method for real-time voice interaction and
electronic equipment, and relates to the technical field of
artificial intelligence. The method comprises the following steps: obtaining optimal recognition text output by an
automatic speech recognition engine as a draft sequence; inputting input data containing the draft sequence into a target large
language model, and calculating
posterior probability distribution of all token positions in the draft sequence through single
forward propagation parallel calculation; determining a model output candidate sequence according to the
posterior probability distribution of each position, comparing the model output candidate sequence with the draft sequence, and completing
inference output based on the comparison result. The application does not need to additionally
train, deploy and maintain an independent draft model, effectively reducing
system engineering complexity and memory occupation. Meanwhile, the application scheme is executed based on the original architecture of the target large
language model throughout the whole process, without the need to modify the
model network structure, and without the need to carry out retraining or fine-tuning, so that the original generality and recognition accuracy of the pre-trained large language model are completely retained.