End-to-end bone and air conduction voice combined recognition method

Abstract

The invention discloses an end-to-end bone and air conduction voice combined recognition method, which comprises the following steps of: firstly, acquiring synchronous air conduction and bone conduction voice data to construct a data set, and outputting a corresponding text; performing data enhancement and acoustic feature extraction on the air conduction and bone conduction voice signals; then, a Conformer-based end-to-end deep neural network model is built, and the Conformer-based end-to-end deep neural network model is composed of three parts, namely two branch networks for processing air conduction voice and bone conduction voice, and a fusion network based on multi-mode Transducer; and then training the neural network, and finally obtaining a corresponding recognition result through the trained network. Compared with a traditional method that voice recognition is carried out only through air conduction voice signals, the combined recognition method can remarkably reduce the error rate of voice recognition, and the overall recognition performance of the system is improved.

Description

technical field [0001] The invention belongs to the technical field of speech recognition, and in particular relates to a bone-air conduction speech joint recognition method. Background technique [0002] In the past decade, thanks to the rise and progress of deep learning, robust automatic speech recognition has achieved remarkable development and has been applied in various fields such as smartphones, smart home appliances, and automobiles. Robust speech recognition algorithms based on deep learning can be mainly divided into two types, one is to remove noise at the front end of the system, including speech enhancement, extracting noise-robust features, etc., and the other is to design an automatic Robust recognition models adapted to different noisy scenarios. However, so far, these deep learning-based speech recognition methods are based on air-conduction speech. Due to the conduction characteristics of speech in the air, speech is easily disturbed by environmental noi...

AI Technical Summary

Problems solved by technology

Since there is no publicly available large-scale bone-air-conduction speech database that can be used for deep learning speech recognition, so far, there is no work on deep learning-based end-to-end bone-air conduction joint speech recognition.

Images