Parkinson's speech detection method based on end-to-end deep learning

Abstract

The present invention discloses a Parkinson's speech detection method based on end-to-end deep learning, comprising the following steps: S1: collecting the speech of a Parkinson's patient and preprocessing the speech to obtain a speech sample; S2: substituting the speech sample into a log-Mel spectrum to perform dynamic feature conversion to obtain a eigenvalue; S3: substituting the eigenvalue into a detection model based on Time-distributed 2D-CNNs and 1D-CNN for calculation, and performing parameter optimization of the model to obtain a calculation model; S4: importing the user's speech information into the calculation model to obtain a calculation result. The present application converts the speech into time-series dynamic features by using the log-Mel spectrum, and then substitutes them into the corresponding model algorithm, thereby effectively improving the accuracy of the entire model.

Description

Technical Field

[0001] The present invention relates to the field of big data computing, and in particular to a Parkinson's speech detection method based on end-to-end deep learning. Background Art

[0002] Parkinson's disease (PD) is a chronic, progressive neurodegenerative disorder whose diagnosis is typically based on specific clinical features found during medical history and neurological examination. Clinical data show that approximately 60% to 90% of PD patients experience speech impairment, typically characterized by low voice, lack of pitch, inaccurate pronunciation, weak sounds (breathing sounds), and creaky sounds (rough speech). Speech signal analysis is considered an important non-invasive method for PD identification.

[0003] Existing speech recognition systems for Parkinson's disease patients typically utilize traditional machine learning algorithms based on static features, including K-nearest neighbor (KNN), Gaussian naive Bayes classifiers (GNB), decision trees (DT), and support vector machines (SVM). These algorithms have low accuracy and provide limited support to doctors during screening. Therefore, it is crucial to develop a more accurate computational model for Parkinson's disease speech, providing doctors with an accurate basis for determining whether a patient is a PD patient. Summary of the Invention

[0004] In response to the shortcomings of the above-mentioned technologies, the present invention provides a Parkinson's speech detection method based on end-to-end deep learning. By using logarithmic Mel-scale maps to convert speech into temporal dynamic features, and then substituting them into the corresponding model algorithm, the accuracy of the entire model can be effectively improved.

[0005] To achieve the above objectives, the present invention discloses a Parkinson's speech detection method based on end-to-end deep learning, comprising the following steps:

[0006] S1: Collect the speech of Parkinson's patients and pre-process the speech to obtain speech samples;

[0007] S2: Substitute the speech sample into the logarithmic Mel spectrogram for dynamic feature conversion to obtain the eigenvalue;

[0008] S3: Substitute the feature values ​​into the detection model based on Time-distributed 2D-CNNs and 1D-CNN for calculation, and optimize the model parameters to obtain the calculation model;

[0009] S4: Import the user's voice information into the calculation model to obtain the calculation results.

[0010] Preferably, in step S1, it is necessary to collect voice in a quiet environment, record voice for a certain length of time and save it, and resample the voice with a resampling frequency of 22050HZ, and save the resampled voice.

[0011] Preferably, in step S2, when converting the temporal dynamic features of the speech log-Mel spectrogram, the speech is first converted into a log-Mel spectrogram, and then the log-Mel spectrogram is divided into a series of overlapping segments, and these segments are obtained by applying a sliding window along the time axis. The main parameters include: window size of 2048, jump length of 512, Mel bandwidth of 65, maximum frequency of 8000, and sampling frequency of 22050.

[0012] Preferably, in step S3, a series of overlapping segments on the log-mel spectrogram are input into the Time-distributed 2D-CNNs model, and these segments are detected to obtain local features, i.e., dynamic features of the time series.

[0013] Preferably, the local features are substituted into the 1D-CNN model, the dynamic features of the time series are flattened into a sequence, and the time dependency is learned. After multiple learning and training, a computational model is finally obtained to output the result information.

[0014] Preferably, in step S4, the user's voice information is substituted into the calculation model, and result information is output. According to the displayed content of the result information, professionals determine whether the user is a Parkinson's patient.

[0015] The beneficial effects of the present invention are as follows: compared with the prior art, the end-to-end deep learning-based Parkinson's speech detection method provided by the present invention adopts the dynamic features of the PD speech log-mel spectrogram with time series signals. In addition to being able to capture characteristics similar to the human ear's perception of frequency without manual extraction, that is, being sensitive to changes in the low-frequency band and insensitive to speech changes in the high-frequency band, it can also automatically obtain and enhance speech fluctuations caused by PD during the pronunciation process that cannot be detected by general static features from continuous speech through operations such as framing and windowing, thereby greatly improving the accuracy of system detection; at the same time, a Time-distributed 2D-CNNs combined with 1D-CNN based Parkinson's speech detection model is proposed, which fully utilizes 2D-CNNs (two-dimensional convolutional neural networks) to efficiently capture the spatial information of local spectrograms, while effectively maintaining the time-dependent information structure of speech on the time axis. The entire recognition process does not require the feature engineering steps in traditional machine learning algorithms, and can be effectively modeled on a smaller training data set, thereby achieving efficient PD detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the steps of the present invention;

[0017] Figure 2 It is the model processing diagram of the present invention;

[0018] Figures 3 to 8 Learning curves for Parkinson's speech detection using existing models of different depths;

[0019] Figure 9 Learning curve for Parkinson's speech detection in this application. DETAILED DESCRIPTION

[0020] In order to more clearly describe the present invention, the present invention is further described below in conjunction with the accompanying drawings and embodiments. Of course, the protection scope of the present invention is not limited to this. Without paying any creative work, simple replacements that can be made by those skilled in the art are all within the scope of protection of this application.

[0021] See also Figure 1 and Figure 2 The present invention discloses a Parkinson's speech detection method based on end-to-end deep learning, comprising the following steps: S1: collecting and preprocessing speech from a Parkinson's patient to obtain a speech sample; S2: subjecting the speech sample to a log-mel spectrogram for dynamic feature conversion to obtain eigenvalues; S3: subjecting the eigenvalues ​​to calculations within a detection model based on Time-distributed 2D-CNNs and 1D-CNNs, and optimizing the model parameters to obtain a computational model; S4: importing the user's speech information into the computational model to obtain a computational result. In this embodiment, the purpose of this application is to obtain a computational model, which can then be used to obtain a computational result by simply importing the processed user's speech. A professional doctor uses this computational result as one of the criteria for determining whether the user is a Parkinson's patient. Furthermore, this application utilizes speech temporal features and spectrum-based local spatial features to effectively capture the dynamics of PD speech signals. Compared to existing machine learning algorithms that rely on static features, this application achieves higher accuracy.

[0022] To achieve the above purpose, in step S1, it is necessary to collect speech in a quiet environment, record a certain length of speech and save it, and resample the speech, saving it at a sampling frequency of 22050 Hz. In this embodiment, it is necessary to collect the speech of Parkinson's patients to obtain basic features. As mentioned in the background art, PD patients may have language disorders, typical features of which include low voices, lack of pitch, inaccurate pronunciation, weak sounds, and creaking sounds. Therefore, these sounds are collected in advance and used as indicators for subsequent judgment of whether Parkinson's disease exists. Since the subsequent process requires feature conversion using logarithmic mel spectrograms, it is necessary to convert the frequency of the speech into corresponding parameters.

[0023] In step S2, when converting the temporal dynamic features of the speech logarithmic Mel spectrogram, the speech is first converted into a logarithmic Mel spectrogram, and then the logarithmic Mel spectrogram is divided into a series of overlapping segments, and these segments are obtained by applying a sliding window along the time axis. The main parameters include: window size of 2048, jump length of 512, Mel bandwidth of 65, maximum frequency of 8000, and sampling frequency of 22050; in this embodiment, since the input feature requirements of the subsequent model are dynamic and temporal; therefore, it is necessary to obtain the temporal dynamic features in the process of feature conversion. After adjusting the characteristic parameters, the speech outputs the required dynamic features through the logarithmic Mel spectrogram.

[0024] In step S3, a series of overlapping segments on the log-mel spectrogram are first input into the Time-distributed 2D-CNNs model. These segments are detected to obtain local features, namely the dynamic features of the time series. After obtaining the local features, the local features are substituted into the 1D-CNN model to flatten the dynamic features of the time series into a sequence and learn temporal dependencies. After multiple learning and training steps, a computational model is finally obtained and the result information is output. In this process, the Time-distributed 2D-CNNs includes 2D convolutional layers, batch normalization layers, 2D average pooling layers, and dropout layers. The 2D convolutional layers are used to capture information about the local spatial spectrogram. The batch normalization layers increase the convergence speed and help improve generalization. The 2D average pooling layers and dropout layers are used for dimensionality reduction and to prevent overfitting, respectively. The 1D-CNN model consists of a one-dimensional convolutional neural network layer and a one-dimensional average pooling layer. The dynamic features of the time series obtained by the Time-distributed 2D-CNNs module are flattened into a sequence and then fed into the 1D-CNN model to learn temporal dependencies. The one-dimensional convolution operation is performed only on the time axis.

[0025] In step S4, the user's voice information is substituted into the calculation model, and the result information is output. According to the display content of the result information, it is assisted in determining whether the user is a Parkinson's patient.

[0026] See also Figures 3 to 9 In the specific implementation process, this application selected 45 experimental subjects (15 HC and 30 PD cases), and the experimental results on a total of 268 short sentence speech samples (training data: test data = 6:4) showed that: among the traditional machine learning algorithms using static features, including K-nearest neighbor (KNN), Gaussian naive Bayes classifier (GNB), decision tree (DT), and support vector machine (SVM) models, the KNN model achieved the best accuracy of 73.33% when using the feature set Surfboard subset and GNB achieved the best accuracy of 73.33% when using the feature set Phonation. When using commonly used deep learning models, including MLP, FCN, ResNet, Time-CNN, Encoder and CNNs-LSTM (with Figures 3 to 8 Among the models with a one-to-one correspondence, the CNNs-LSTM model achieved the best accuracy of 76.22%. The proposed Time-distributed 2D-CNNs and 1D-CNN based on temporal distribution achieved an accuracy of 81.56%. Experimental results on speech input data of a selection of simple and complex sentences showed that the proposed model achieved an accuracy rate of 92%, demonstrating that this method can significantly improve system detection accuracy. Furthermore, even with a smaller training dataset, the model exhibits better convergence than commonly used deep learning models.

[0027] The above disclosures are only a few specific embodiments of the present invention, but the present invention is not limited thereto, and any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. A Parkinson's speech detection method based on end-to-end deep learning, characterized in that: The following steps are involved: S1: Collect the speech of Parkinson's patients and pre-process the speech to obtain speech samples; S2: Substitute the speech sample into the logarithmic Mel spectrogram for dynamic feature conversion to obtain the eigenvalue; S3: Substitute the feature values ​​into the detection model based on Time-distributed 2D-CNNs and 1D-CNN for calculation, and optimize the model parameters to obtain the calculation model; S4: Importing the user's voice information into the calculation model to obtain calculation results; In step S1, it is necessary to collect voice in a quiet environment, record a certain length of voice and save it, and resample the voice with a resampling frequency of 22050HZ, and save the resampled voice; In step S2, when converting the temporal dynamic features of speech log-Mel spectrogram, the speech is first converted into a log-Mel spectrogram, and then the log-Mel spectrogram is divided into a series of overlapping segments. These segments are obtained by applying a sliding window along the time axis. The main parameters include: window size of 2048, jump length of 512, Mel bandwidth of 65, maximum frequency of 8000, and sampling frequency of 22050.

2. The Parkinson's speech detection method based on end-to-end deep learning according to claim 1 is characterized in that In step S3, a series of overlapping segments on the log-mel spectrogram are input into the Time-distributed 2D-CNNs model, and these segments are detected to obtain local features, that is, the dynamic features of the time series.

3. The Parkinson's speech detection method based on end-to-end deep learning according to claim 2, characterized in that Substitute the local features into the 1D-CNN model, flatten the dynamic features of the time series into a sequence, and learn the time dependency. After multiple learning and training, the computational model is finally obtained and the result information is output.

4. The Parkinson's speech detection method based on end-to-end deep learning according to claim 1, characterized in that In step S4, the user's voice information is substituted into the calculation model, and the result information is output. According to the display content of the result information, it is assisted in determining whether the user is a Parkinson's patient.

Images