A dialect speech recognition method in a vehicle-mounted multi-source noise environment based on ASR

Abstract

The present application relates to the technical field of intelligent automobile voice recognition, and particularly relates to a dialect voice recognition method in a vehicle-mounted multi-source noise environment based on ASR. The present application collects mandarin, Cantonese, Sichuanese voice data and three types of noise data of whistle, engine and wind sound in a vehicle environment; pre-processes the collected voice, extracts effective voice features and suppresses noise interference through a noise reduction algorithm based on frequency domain filtering; inputs the enhanced voice into a Dolphin-asr model for recognition, and realizes accurate recognition of multi-dialect voice in a complex vehicle-mounted environment. The present application can improve the recognition accuracy and environmental adaptability of a vehicle intelligent cabin voice interaction system in a multi-noise and multi-dialect scene.

Description

Technical Field

[0001] This invention relates to the field of intelligent vehicle voice interaction technology, specifically to a dialect speech recognition method based on ASR in a multi-source noise environment in a vehicle. Background Technology

[0002] With the development of smart cockpit technology, in-vehicle voice interaction systems have become an important way for humans and machines to interact in automobiles. However, during vehicle operation, there are complex noises such as horn honking, engine running, and wind noise, which severely interfere with voice signals. At the same time, users in different regions have different dialect needs, such as Mandarin, Cantonese, and Sichuanese. Traditional speech recognition models have low recognition accuracy in low signal-to-noise ratio and dialect scenarios, making it difficult to meet the needs of smart cockpit applications. Summary of the Invention

[0003] The purpose of this invention is to provide a dialect speech recognition method based on ASR in a vehicle multi-source noise environment. By fusing a frequency domain filtering noise reduction algorithm with the Dolphin-asr model, the accuracy of dialect speech recognition in complex vehicle noise environments can be improved.

[0004] The specific solution includes: first, collecting various types of voice data from the vehicle environment, including Mandarin, Cantonese, Sichuan dialect, as well as noise data such as horn, engine, and wind noise; second, using frequency domain analysis methods to obtain the spectral features of voice and noise, and reducing the impact of noise through adaptive filtering; and finally, enhancing the voice input to the Dolphin-asr model to achieve voice feature extraction, recognition, and dialect classification.

[0005] This invention combines frequency domain noise reduction with a deep learning ASR model to effectively improve speech clarity and recognition performance in complex vehicle noise environments, meeting the multi-dialect voice interaction needs of smart cockpits and improving the adaptability and reliability of in-vehicle voice systems. Attached Figure Description

[0006] Figure 1 This is a flowchart of the dialect speech recognition method based on ASR in a vehicle-mounted multi-source noise environment according to the present invention. Detailed Implementation

[0007] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of an ASR-based dialect speech recognition method and system for in-vehicle multi-source noise environments proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0008] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0009] The following description, in conjunction with the accompanying drawings, details the specific scheme of a dialect speech recognition method and system based on ASR in a vehicle-mounted multi-source noise environment provided by the present invention.

[0010] Please see Figure 1 The diagram illustrates a flowchart of a dialect speech recognition method based on ASR in a vehicle-mounted multi-source noise environment according to the present invention. The method includes:

[0011] Step S1: Collect dialect speech data in a multi-source noise environment in the vehicle and construct an in-vehicle dialect speech dataset.

[0012] The embodiments of the present invention are mainly applied to the voice interaction scenario of intelligent cockpit. In order to solve the problem of reduced voice recognition accuracy caused by complex noise environment during vehicle driving, voice data in real vehicle environment is collected first.

[0013] Specifically, the system collects voice signals from the driver and passengers using a microphone array installed inside the vehicle cabin, with a sampling frequency set to 16kHz. It collects voice data under different driving conditions, including when the vehicle is stationary, when driving on urban roads, and when driving at high speeds.

[0014] The voice data includes three dialects: Mandarin, Cantonese, and Sichuanese. It also collects typical noise data from the vehicle environment, including vehicle horn noise, engine noise, and wind noise generated at high speeds.

[0015] Preferably, in one embodiment of the present invention, a multi-source noise database is established for different noise types, wherein horn noise is mainly concentrated in the high-frequency region, engine noise is mainly manifested as low-frequency periodic interference, and wind noise is mainly manifested as broadband random disturbance.

[0016] Based on the collected speech data under different signal-to-noise ratio conditions, an in-vehicle multi-source noise dialect speech dataset containing dialect category labels, noise category labels, and text labels is constructed to provide a data foundation for subsequent model training.

[0017] Step S2: Perform frequency domain analysis on the input speech signal and use a noise reduction algorithm based on frequency domain filtering to obtain an enhanced speech signal.

[0018] Preferably, in one embodiment of the present invention, the acquired raw speech signal is preprocessed, including pre-emphasis, framing, and windowing.

[0019] First, perform a Fast Fourier Transform on the input time-domain speech signal x(t) to convert the time-domain signal into a frequency-domain representation: X(k) = FFT(x(t)), where X(k) represents the spectral amplitude of the speech signal at the k-th frequency point.

[0020] Subsequently, based on the spectral characteristics of vehicle environmental noise, noise estimation is performed in different frequency regions to obtain the noise spectrum N(k). Based on the energy relationship between the speech spectrum and the noise spectrum, an adaptive frequency domain filter weight is designed: H(k) = |X(k)|² / (|X(k)|² + λ|N(k)|²). This frequency domain filter weight corrects the original speech spectrum, and the enhanced speech signal is recovered using an inverse fast Fourier transform.

[0021] Step S3: Input the enhanced speech into the Dolphin-asr model to complete speech feature extraction and dialect recognition.

[0022] Preferably, in one embodiment of the present invention, the enhanced speech, after frequency domain filtering, is input into the Dolphin-asr speech recognition model. The Dolphin-asr model includes a speech feature extraction module, a deep encoding module, a speech decoding module, and a dialect classification module. The input speech undergoes acoustic feature transformation to obtain two-dimensional acoustic features, which are then subjected to deep feature encoding by the Dolphin-asr model, ultimately outputting the recognized text and its corresponding dialect category.

[0023] Step S4: Classify dialects based on the recognition results and output the speech recognition results.

[0024] Preferably, in one embodiment of the present invention, the dialect category is determined based on the output probability of the model. The model outputs three probabilities: Mandarin, Cantonese, and Sichuanese: P=[Pm,Py,Pc]. The category corresponding to the highest probability is selected as the final dialect recognition result: Class=argmax(P). When the recognition confidence is greater than a preset threshold, the system outputs the corresponding text and dialect type; when the confidence is lower than the threshold, the speech enhancement and recognition process is re-executed.

[0025] This completes the dialect speech recognition process in a multi-source noise environment in a vehicle.

[0026] In summary, this invention first collects in-vehicle voice data including three dialects (Mandarin, Cantonese, and Sichuanese) and three noise environments (horn, engine, and wind noise); then, it performs spectral analysis and noise suppression on the input voice using a frequency domain filtering and noise reduction algorithm to obtain an enhanced voice signal; the enhanced voice is then input into the Dolphin-asr model, and dialect voice recognition is completed through deep feature extraction and voice decoding.

[0027] An embodiment of the present invention also provides a dialect speech recognition system based on ASR in a vehicle-mounted multi-source noise environment. The system includes: a speech acquisition module for acquiring various types of speech data in the vehicle environment; a frequency domain noise reduction module for performing spectrum analysis on the input speech and suppressing noise interference; an ASR recognition module for using the Dolphin-asr model to complete speech feature extraction and text conversion; a dialect classification module for recognizing Mandarin, Cantonese, and Sichuan dialect; a data storage module for storing speech data, model parameters, and recognition results; and a communication output module for sending the recognition results to the vehicle's intelligent cockpit control system.

[0028] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0029] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A dialect speech recognition method based on ASR in a vehicle-mounted multi-source noise environment, characterized in that, The method includes: collecting voice data under vehicle driving conditions, the voice data including three dialects (Mandarin, Cantonese, and Sichuanese) and three types of vehicle noise (horn, engine, and wind noise); preprocessing the collected voice signal and using a noise reduction algorithm based on frequency domain filtering to suppress the noise frequency band to obtain an enhanced voice signal; inputting the enhanced voice signal into a Dolphin-asr speech recognition model for feature extraction and speech decoding, and outputting the corresponding dialect recognition result.

2. The dialect speech recognition method based on ASR in a vehicle-mounted multi-source noise environment according to claim 1, characterized in that, The noise reduction algorithm based on frequency domain filtering includes: performing a fast Fourier transform on the input speech signal to convert the time domain signal into a frequency domain signal; determining the noise frequency band based on the spectral characteristics of horn, engine, and wind noise; designing adaptive frequency domain filtering weights to attenuate the noise frequency region; and recovering the enhanced speech through inverse Fourier transform.

3. The dialect speech recognition method based on ASR in a vehicle-mounted multi-source noise environment according to claim 1, characterized in that, The Dolphin-asr model includes an acoustic feature extraction module, a deep neural network encoding module, and a speech decoding module, which are used to extract acoustic features of different dialects and complete the speech-to-text conversion.

4. The dialect speech recognition method based on ASR in a vehicle-mounted multi-source noise environment according to claim 1, characterized in that, The speech recognition process includes: establishing a dialect speech database containing Mandarin, Cantonese, and Sichuanese; training and testing speech data under different signal-to-noise ratio conditions; and outputting the final dialect recognition result based on the recognition confidence level.

5. A dialect speech recognition system based on ASR in a vehicle-mounted multi-source noise environment, characterized in that, include: The system includes a voice acquisition module, a frequency domain noise reduction module, an ASR recognition module, a dialect classification module, a result output module, and a storage and communication module. The system executes the method to achieve accurate dialect speech recognition in complex vehicle noise environments.

Images