The invention relates to the technical field of
artificial intelligence, and discloses a
Bluetooth communication intelligent
speech translation method and
system based on multi-mode enhancement, and the method comprises the steps: collecting a multi-channel
audio signal through a built-in multi-
microphone array of a
Bluetooth device, carrying out the dynamic direction self-adaptive beam forming of the multi-channel
audio signal, and carrying out the self-adaptive beam forming of the multi-channel
audio signal; extracting a Mel
spectrogram feature of the direction enhancement
signal, identifying lip regions of a plurality of candidate speakers in each frame of real-time speaking video captured by a camera, performing
time sequence convolution on the lip regions to obtain a lip movement
time sequence embedded vector, calculating a correlation
score with the Mel
spectrogram feature, separating the direction enhancement
signal, and obtaining a lip movement
time sequence embedded vector; and performing text transcription and conversion on the high-confidence separation voice to obtain a
translation language text, and sending the synthesized target translation voice to a preset mobile terminal through the
Bluetooth device to obtain a target translation result. According to the method, the real-time performance and accuracy of
speech translation are improved in a multi-person scene, far-field speech,
noise interference and accent difference.