Noise disease recognition method based on MFCC and CNN

Abstract

A voice disease recognition method based on MFCC and CNN comprises a patient audio acquisition module, an audio data preprocessing module, an MFCC feature extraction module, a classification model construction module and a patient voice disease category recognition module which are connected in sequence. The audio data preprocessing module uses a VAD endpoint detection algorithm to cut and unify recorded effective segments, and the MFCC feature extraction module performs operations such as weighting, windowing, transformation and filtering on the cut audio to obtain an exclusive MFCC feature map as a training object of a downstream classification task; the classification model construction module feeds a feature map containing pathological information to the voice disease recognition module. According to the method, the voice information of the vowel pronunciation of the Chinese pinyin of the patient is utilized to realize the classification and identification of four voice diseases (spasmodic dysphonia, vocal cord paralysis, vocal cord nodules and vocal cord polyps) with similar pathological expressions.

Description

Technical Field

[0001] The invention belongs to the field of computer speech recognition and relates to a voice disease recognition method using MFCC and CNN. Background Art

[0002] According to statistics, 3%-9% of the world's population has varying degrees of voice disease or related disease symptoms, especially in groups such as teachers, hosts, and speakers who need to use their vocal cords frequently to make sounds. The pathological manifestations of most voice diseases are mainly hoarseness, dysphonia, and aphonia. Voice disease problems affect people's normal communication, life, and work quality.

[0003] Clinically, electronic laryngoscope is often used to determine the location of laryngeal lesions, and laryngeal electromyography is used to diagnose laryngeal muscle and vocal cord lesions. These two clinical voice disease detection methods rely on the doctor's clinical experience and subjective judgment, and invasive diagnostic methods are prone to cause secondary damage to patients.

[0004] The voice disease identification method based on acoustic sensors can provide a low-cost, convenient and non-invasive way to discover and screen voice diseases. Compared with traditional doctor auscultation and laryngoscopy, the use of acoustic sensors can collect the audio of the patient's vocal cord vibration without human intervention, and assist doctors in the identification and diagnosis of diseases with more objective data.

[0005] In the field of computer speech recognition, audio signal analysis methods based on traditional digital signal processing technology have achieved good performance in analyzing audio, but the corresponding time-frequency domain analysis mainly starts from correlation, and many potential semantic expressions are difficult to discover, especially when the similarity of the audio itself is high, the traditional processing methods can only capture limited information.

[0006] The algorithm model based on machine learning / deep learning methods of data science has shown great advantages in audio encoding and decoding and downstream classification and prediction tasks. In particular, it is used to find information about pathological features in audio data. The present invention collects vowel audio based on an acoustic sensor and constructs a method for identifying voice diseases based on MFCC and CNN. Summary of the invention

[0007] The present invention mainly overcomes the lack of methods for identifying the types of voice diseases based on speech audio features at home and abroad. Starting from the reconstruction of sound sensor and MFCC feature construction, a new voice disease identification method based on MFCC and CNN is proposed.

[0008] The present invention selects 6 phonemes of Chinese pinyin vowels, " / a / , / o / , / e / , / i / , / u / , / ü / ". These vowel phonemes are all voiced sounds, and are accompanied by the vibration of the vocal cords when they are uttered. The short-time energy of the audio signal is large, the average amplitude is large, the short-time zero-crossing rate is low, the pathological characteristics carried are more obvious, and the information is richer. The MFCC coefficient is used to extract features of the above audio information, thereby realizing a high-accuracy, high-efficiency and convenient voice disease recognition method.

[0009] The voice disease recognition method based on MFCC and CNN described in the present invention includes a patient audio acquisition module, an audio data preprocessing module, an MFCC feature extraction module, a classification model construction module, and a patient voice disease category recognition module which are connected in sequence. The patient audio acquisition module uses an acoustic sensor to record the patient's designated phoneme utterances, the audio data preprocessing module uses a VAD endpoint detection algorithm to crop and unify the recorded valid segments, the MFCC feature extraction module performs operations such as emphasis, windowing, transformation, and filtering on the cropped audio to obtain an exclusive MFCC feature map as a training object for downstream classification tasks, and the classification model construction module feeds the feature map containing pathological information to the voice disease recognition module.

[0010] Patient audio collection module, including:

[0011] (1.1) Select the six vowels in Chinese Pinyin: / a / , / o / , / e / , / i / , / u / , / ü / as the recording objects;

[0012] (1.2) Use a high-quality microphone and digital amplifier as the acquisition equipment, the distance from the microphone to the mouth is about 15-20 cm, the background noise level is 40-45 dB, the sampling frequency is 44100 Hz, the resolution is 16 bits, and it is saved in uncompressed WAV format;

[0013] Audio data preprocessing module, specifically including:

[0014] (2.1) Use the VAD endpoint detection algorithm to mark the starting and ending points of the valid parts of the collected audio clips;

[0015] (2.2) The effective segments are cut into short audio data with a time length of 0.5s, and the trimmed audio is spliced ​​with silent segments to unify the time length of the audio;

[0016] MFCC feature extraction module, specifically including:

[0017] (3.1) Pre-emphasize the standard patient voice audio clipped in (2.2) with a pre-emphasis coefficient of 0.97 to increase its high-frequency resolution;

[0018] (3.2) Frame division, windowing, and fast Fourier transform are performed to obtain an array. The elements in the array represent the energy of the speech signal in one frame:

[0019]

[0020] (3.3) The spectrum is smoothed by a Mel-scale triangular bandpass filter to highlight the formant of the original speech;

[0021] (3.4) Perform logarithmic operation on the signal data after filtering in (3.3), perform coordinate scaling, and map the signal to a low-dimensional space through discrete cosine transform (DCT) to obtain MFCC feature coefficients of size 128×47;

[0022] (3.4) Perform logarithmic operation on the signal data after filtering in (3.3), perform coordinate scaling, and map the signal to a low-dimensional space through discrete cosine transform (DCT) to obtain MFCC feature coefficients of size 128×47;

[0023]

[0024]

[0025] (3.5) Compress and reorganize the MFCC feature coefficient array obtained in (3.4), and average the values ​​on 47 time frames on each order of MFCC to obtain a 128×1 MFCC two-dimensional matrix;

[0026] (3.6) The two-dimensional matrix obtained in (3.5) is resized, and each 0.5s unit audio is extracted by the MFCC feature extraction module to obtain a 16×8 feature map;

[0027] Classification model building module, including:

[0028] (4.1) A classification model is constructed based on a convolutional neural network. The input of the model is the MFCC feature map of size 16×8 in (3.6), and the stack of 6 vowel phonemes is used as the final input feature.

[0029] (4.2) Input the feature map in (4.1) into a convolutional layer with a kernel of 3×3 and an activation function of tanh, and pass it through a dropout layer with a probability of 0.1 to prevent overfitting, and flatten the output feature map into a single row tensor;

[0030] (4.3) The single row tensor is passed through three fully connected layers in sequence, continuously reducing its size until the number of classification categories is 4, and finally the output is activated by the softmax function to obtain the probability of the audio being judged as one of the four diseases;

[0031] Output=s0ftmax(ReLU(ReLU(Z1W2+b2)W3+b3)W4+b4) (4-1)

[0032] (4.4) Integrate and input all the collected audio data features, conduct training and learning, and save the classification model;

[0033] The patient's voice disease category identification module includes:

[0034] (5.1) The patient provides or collects the patient's reading segments of the specified phonemes on the outpatient site, and converts them into feature units according to steps 2 and 3;

[0035] (5.2) Record the classification model saved in step 4, input the audio MFCC feature unit obtained in (5.1), and obtain the probability scores of the four types of diseases;

[0036] (5.3) Based on the probability score obtained in (5.2), weighted processing is performed and compared with the set threshold to output the patient's voice disease category.

[0037] The present invention utilizes the audio information of the pronunciation of Chinese Pinyin vowels of the patients to realize the classification and identification of four voice diseases with similar pathological manifestations (spasmodic dysphonia, vocal cord paralysis, vocal cord nodules, and vocal cord polyps).

[0038] The advantages of the present invention are: the method of the present invention is easy to operate, the model has a shallow number of layers, the preprocessing time is short, the reasoning time is short, and the diagnosis speed is fast. In the current diagnosis of voice diseases, patients need to go to professional medical institutions to register, schedule diagnosis, and use invasive medical instruments for observation and diagnosis, which has high costs and risks. Therefore, the present invention can effectively collect audio information through acoustic sensors to extract MFCC features to realize the diagnosis of the disease type of patients with voice diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A data processing flow diagram of the present invention is shown in FIG.

[0040] Figure 2 This is a schematic diagram of the MFCC feature extraction process of the present invention.

[0041] Figure 3 A schematic diagram of a feature map recasting process of the present invention

[0042] Figure 4 Schematic diagram of the classification model structure of the present invention

[0043] Figure 5 The system structure diagram of the present invention is DETAILED DESCRIPTION

[0044] The present invention is further described below in conjunction with the accompanying drawings.

[0045] Reference Figure 1-Figure 5 :

[0046] The voice disease recognition method based on MFCC and CNN includes a patient audio acquisition module, an audio data preprocessing module, an MFCC feature extraction module, a classification model construction module, and a patient voice disease category recognition module, which are connected in sequence. The patient audio acquisition module uses an acoustic sensor to record the patient's designated phoneme utterances, and the audio data preprocessing module uses a VAD endpoint detection algorithm to crop and unify the recorded valid segments. The MFCC feature extraction module performs operations such as emphasis, windowing, transformation, and filtering on the cropped audio to obtain an exclusive MFCC feature map as a training object for downstream classification tasks. The classification model construction module feeds the feature map containing pathological information to the voice disease recognition module.

[0047] Patient audio collection module, including:

[0048] (1.1) Select the six vowels in Chinese Pinyin: / a / , / o / , / e / , / i / , / u / , / ü / as the recording objects;

[0049] (1.2) Use a high-quality microphone and digital amplifier as the acquisition equipment, the distance from the microphone to the mouth is about 15-20 cm, the background noise level is 40-45 dB, the sampling frequency is 44100 Hz, the resolution is 16 bits, and it is saved in uncompressed WAV format;

[0050] Audio data preprocessing module, specifically including:

[0051] (2.1) Use the VAD endpoint detection algorithm to mark the starting and ending points of the valid parts of the collected audio clips;

[0052] (2.2) The effective segments are cut into short audio data with a time length of 0.5s, and the trimmed audio is spliced ​​with silent segments to unify the time length of the audio;

[0053] MFCC feature extraction module, specifically including:

[0054] (3.1) Pre-emphasize the standard patient voice audio clipped in (2.2) with a pre-emphasis coefficient of 0.97 to increase its high-frequency resolution;

[0055] (3.2) Frame division, windowing, and fast Fourier transform are performed to obtain an array. The elements in the array represent the energy of the speech signal in one frame:

[0056]

[0057] (3.3) The spectrum is smoothed by a Mel-scale triangular bandpass filter to highlight the formant of the original speech;

[0058] (3.4) Perform logarithmic operation on the signal data after filtering in (3.3), perform coordinate scaling, and map the signal to a low-dimensional space through discrete cosine transform (DCT) to obtain MFCC feature coefficients of size 128×47;

[0059]

[0060]

[0061] (3.5) Compress and reorganize the MFCC feature coefficient array obtained in (3.4), and average the values ​​on 47 time frames on each order of MFCC to obtain a 128×1 MFCC two-dimensional matrix;

[0062] (3.6) The two-dimensional matrix obtained in (3.5) is resized, and each 0.5s unit audio is extracted by the MFCC feature extraction module to obtain a 16×8 feature map;

[0063] Classification model building module, including:

[0064] (4.1) A classification model is constructed based on a convolutional neural network. The input of the model is the MFCC feature map of size 16×8 in (3.6), and the stack of 6 vowel phonemes is used as the final input feature.

[0065] (4.2) Input the feature map in (4.1) into a convolutional layer with a kernel of 3×3 and an activation function of tanh, and pass it through a dropout layer with a probability of 0.1 to prevent overfitting, and flatten the output feature map into a single row tensor;

[0066] (4.3) The single row tensor is passed through three fully connected layers in sequence, continuously reducing its size until the number of classification categories is 4, and finally the output is activated by the softmax function to obtain the probability of the audio being judged as one of the four diseases;

[0067] Output=softmax(ReLU(ReLU(Z1W2+b2)W3+b3)W4+b4) (4-1)

[0068] (4.4) Integrate and input all the collected audio data features, conduct training and learning, and save the classification model;

[0069] The patient's voice disease category identification module includes:

[0070] (5.1) The patient provides or collects the patient's reading segments of the specified phonemes on the outpatient site, and converts them into feature units according to steps 2 and 3;

[0071] (5.2) Record the classification model saved in step 4, input the audio MFCC feature unit obtained in (5.1), and obtain the probability scores of the four types of diseases;

[0072] (5.3) Based on the probability score obtained in (5.2), weighted processing is performed and compared with the set threshold to output the patient's voice disease category.

Claims

1. A voice disease recognition method based on MFCC and CNN includes a patient audio acquisition module, an audio data preprocessing module, an MFCC feature extraction module, a classification model construction module, and a patient voice disease category recognition module, which are connected in sequence. The patient audio acquisition module uses an acoustic sensor to record the patient's designated phoneme utterances, and the audio data preprocessing module uses a VAD endpoint detection algorithm to crop and unify the recorded valid segments. The MFCC feature extraction module performs operations such as emphasis, windowing, transformation, and filtering on the cropped audio to obtain an exclusive MFCC feature map as a training object for downstream classification tasks. The classification model construction module feeds the feature map containing pathological information to the voice disease recognition module. Patient audio collection module, including: (1.1) Select the six vowels in Chinese Pinyin: / a / , / o / , / e / , / i / , / u / , / ü / as the recording objects; (1.2) Use a high-quality microphone and digital amplifier as the acquisition equipment, the distance from the microphone to the mouth is about 15-20 cm, the background noise level is 40-45 dB, the sampling frequency is 44100 Hz, the resolution is 16 bits, and it is saved in uncompressed WAV format; Step 2), audio data preprocessing module, specifically including: (2.1) Use the VAD endpoint detection algorithm to mark the starting and ending points of the valid parts of the collected audio clips; (2.2) The effective segments are cut into short audio data with a time length of 0.5s, and the trimmed audio is spliced ​​with silent segments to unify the time length of the audio; Step 3), MFCC feature extraction module, specifically including: (3.1) Pre-emphasize the standard patient voice audio clipped in (2.2) with a pre-emphasis coefficient of 0.97 to increase its high-frequency resolution; (3.2) Frame division, windowing, and fast Fourier transform are performed to obtain an array. The elements in the array represent the energy of the speech signal in one frame: (3.3) The spectrum is smoothed by a Mel-scale triangular bandpass filter to highlight the formant of the original speech; (3.4) Perform logarithmic operation on the signal data after filtering in (3.3), perform coordinate scaling, and map the signal to a low-dimensional space through discrete cosine transform (DCT) to obtain MFCC feature coefficients of size 128×47; DCT: MFCC: (3.5) Compress and reorganize the MFCC feature coefficient array obtained in (3.4), and average the values ​​on 47 time frames on each order of MFCC to obtain a 128×1 MFCC two-dimensional matrix; (3.6) The two-dimensional matrix obtained in (3.5) is resized, and each 0.5s unit audio is extracted by the MFCC feature extraction module to obtain a 16×8 feature map; Step 4), the classification model construction module includes: (4.1) A classification model is constructed based on a convolutional neural network. The input of the model is the MFCC feature map of size 16×8 in (3.6), and the stack of 6 vowel phonemes is used as the final input feature. (4.2) Input the feature map in (4.1) into a convolutional layer with a kernel of 3×3 and an activation function of tanh, and pass it through a dropout layer with a probability of 0.1 to prevent overfitting, and flatten the output feature map into a single row tensor; (4.3) The single row tensor is passed through three fully connected layers in sequence, continuously reducing its size until the number of classification categories is 4, and finally the output is activated by the softmax function to obtain the probability of the audio being judged as one of the four diseases; Output=softmax(ReLU(ReLU(Z1W2+b2)W3+b3)W4+b4) (4-1) (4.4) Integrate and input all the collected audio data features, conduct training and learning, and save the classification model; Step 5), the patient's voice disease category identification module specifically includes: (5.1) The patient provides or collects the patient's reading segments of the specified phonemes on the outpatient site, and converts them into feature units according to steps 2 and 3; (5.2) Record the classification model saved in step 4, input the audio MFCC feature unit obtained in (5.1), and obtain the probability scores of the four types of diseases; (5.3) Based on the probability score obtained in (5.2), weighted processing is performed and compared with the set threshold to output the patient's voice disease category.

Images