The invention provides a multi-model
speech recognition method based on a C-Adapter adapter, and belongs to the technical field of
speech recognition, and the method comprises the steps: carrying out the
feature extraction of an original speech
signal through an acoustic
encoder, and obtaining the speech feature information; the C-Adapter adapter carries out
processing according to the voice feature information, generates
modal adaptation feature information and maps the
modal adaptation feature information to a voice recognition space; mapping the voice recognition task characters to a voice recognition space through a text vector model, and constructing text task representation information; the
modal adaptation feature information and the text task representation information in the
speech recognition space are spliced, and speech information to be recognized is constructed; and recognizing the voice information to be recognized through the large
language model to obtain a voice recognition result. According to the method, the combination of the speech recognition model and the large
language model is realized, the accuracy of speech recognition is effectively improved, the speed of speech recognition is improved, and the requirement of converting speech into characters in real time is met while the speech recognition efficiency is improved.