Parkinson's disease detection model construction method and platform based on speech features

Abstract

The present invention discloses a method and platform for constructing a Parkinson's disease detection model based on speech features. The method includes normalizing the encoded questionnaire results and extracted speech features to obtain normalized questionnaire results and speech features; converting the normalized questionnaire results and speech features into high-dimensional representation vectors through an embedding layer; obtaining an attention score between each embedding vector and the questionnaire embedding, and normalizing the scores through a softmax function; performing a weighted fusion of the questionnaire result embedding vector and the speech feature embedding vector based on the attention score to obtain a fused feature vector; and inputting the fused feature vector into a risk assessment model to obtain a Parkinson's disease risk prediction. The technical solution provided by the present invention can provide convenient and accurate prediction services for Parkinson's disease patients.

Description

Technical Field

[0001] The present invention relates to the field of speech processing, and in particular to a method and platform for constructing a Parkinson's disease detection model based on speech features. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, speech detection has gradually attracted attention as a non-invasive, cost-effective method. Humans produce speech by generating airflow through the lungs, which causes the vocal cords to vibrate. This process involves complex vocal tract structure and neural regulation. Neurodegeneration in patients with Parkinson's disease can lead to degeneration of central and autonomic nervous system nerves related to voice production, as well as muscle stiffness, which impairs vocal muscle control. Therefore, analyzing the speech characteristics of patients with Parkinson's disease may provide a new approach to predicting Parkinson's disease.

[0003] Although earlier studies, such as those by Little et al., have revealed some correlations between PD and speech parameters, existing speech detection models still suffer from insufficient generalization. These models often rely on specific datasets for training and validation, resulting in poor performance when used with diverse populations or in different speech environments. Furthermore, speech detection in real-world scenarios faces challenges such as irregular pronunciation and inconsistent content, requiring models to be highly error-tolerant and independent of spoken content.

[0004] Internationally, research using speech data combined with artificial intelligence to predict Parkinson's disease has made some progress, but much of this research remains at the research stage and has yet to be widely translated into practical applications. While research in China started relatively late, it has developed rapidly, with significant breakthroughs particularly in the application of deep learning technologies. However, current research findings remain primarily limited to laboratories or medical professionals, lacking public service applications for the general public. Summary of the Invention

[0005] The present invention aims to overcome the shortcomings of the prior art and provide a method for constructing a Parkinson's disease detection model based on speech features, comprising the following steps:

[0006] Step 1: extracting user voice features and obtaining questionnaire results, encoding the questionnaire results, and normalizing the encoded questionnaire results and extracted voice features to obtain normalized questionnaire results and voice features;

[0007] Step 2: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors through the embedding layer. The speech features are mapped into an embedding vector of fixed dimension through the fully connected layer. Each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and then spliced ​​into a single questionnaire embedding vector.

[0008] Step 3: Get the attention score between each embedding vector and the questionnaire embedding, and normalize the score through the softmax function;

[0009] Step 4: Based on the attention score, the embedding vector of the questionnaire results and the embedding vector of the speech features are weightedly fused to obtain a fused feature vector;

[0010] Step 5: Input the obtained fusion feature vector into the risk assessment model to obtain the risk prediction of Parkinson's disease.

[0011] Furthermore, the normalized questionnaire results and speech features are converted into high-dimensional representation vectors respectively through the embedding layer, wherein the speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and spliced ​​into a single questionnaire embedding vector, including:

[0012] Each speech feature is mapped to an embedding vector of fixed dimension through a fully connected layer. For each question in the questionnaire, it is converted to the same embedding dimension as the mapped speech feature through another fully connected layer, and the embedding vectors of all questions in the questionnaire are spliced ​​into a single questionnaire embedding vector.

[0013] Furthermore, obtaining the attention score between each embedding vector and the questionnaire embedding and normalizing the score through the softmax function includes:

[0014] Calculate the attention score of each embedding vector, and calculate the correlation between the speech feature embedding vector and the questionnaire embedding vector through the dot product or cosine similarity function; for each speech feature embedding vector, obtain the attention score between it and the questionnaire embedding vector, and normalize the attention score through the softmax function so that the sum of the normalized attention scores is 1.

[0015] Furthermore, the embedding vector of the questionnaire results and the embedding vector of the speech features are weightedly fused according to the attention score to obtain a fused feature vector, including:

[0016] The speech feature embedding vectors are weighted and summed using the weights output by the attention layer to obtain a weighted speech feature embedding vector; the weighted speech feature embedding vector is concatenated with the questionnaire embedding vector to obtain a fused feature vector.

[0017] Furthermore, the cross entropy loss function is used to measure the difference between the predicted probability distribution and the true probability distribution. The cross entropy loss is calculated by calculating the predicted cross entropy loss of the questionnaire results and speech features, and weighting them according to the importance of the features. The calculation formula of the cross entropy loss is:

[0018] L = α·CEsurvey + β·CEspeech

[0019] Among them, CEsurvey is the cross entropy loss of the questionnaire results, CEspeech is the cross entropy loss of the speech features, and α and β are weights assigned according to the importance of the features.

[0020] A Parkinson's disease detection model construction platform based on speech features, which applies the Parkinson's disease detection model construction method based on speech features, includes a database module, a data acquisition module, a data preprocessing module, a model selection module, an output module, and a performance evaluation module;

[0021] The data acquisition module is connected to the database module, the database module is connected to the data preprocessing module, the data preprocessing module is connected to the model selection module, and the output module and performance evaluation module are respectively connected to the model selection module.

[0022] The beneficial effects of the present invention are: by comprehensively utilizing the A.Little dataset, the MDVR-KCL dataset, and the Saarbruecken Voice dataset, this product expands the age range of voice samples and improves the generalization ability of the model. In terms of voice feature selection, after medical analysis and optimization of the random forest algorithm, 15 key features were selected from 22 acoustic parameters to enhance the accuracy of the model. By developing a lightweight and convenient application platform, WeChat Mini Program, we have innovated the dataset training and mini program collection mechanism, and provided a user-friendly and highly interactive application platform, allowing users to easily access and use this diagnostic tool, and maximize the use of user-input voice information to enhance the system's voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of the method for building a Parkinson's disease detection model based on speech features;

[0024] Figure 2 Flowchart for MLP model training;

[0025] Figure 3 Use flowcharts for users;

[0026] Figure 4 Rank the association of 15 characteristics with Parkinson's disease;

[0027] Figure 5 Schematic diagram of MLP model small-scale data cross-dataset training and detection;

[0028] Figure 6 Schematic diagram of small-scale cross-dataset training and detection for DNN models;

[0029] Figure 7 Schematic diagram of small-scale data cross-dataset training and detection for regression models;

[0030] Figure 8 Schematic diagram of small-scale data cross-dataset training and detection for random forest models;

[0031] Figure 9 Schematic diagram of ROC curve for small-scale training of MLP model;

[0032] Figure 10 This is a schematic diagram of the discrimination between the patient group and the control group samples after integrating the 15 features in the three data sets;

[0033] Figure 11 This is a schematic diagram of the MFCC images of the 70-year-old and 80-year-old healthy control group and Parkinson's disease patient group synthesized by Pydub based on the MDVR-KCL dataset;

[0034] Figure 12 This is a diagram of the confusion matrix obtained after the model was trained on the Saarbruecken Voice Database and MDVR-KCL datasets and then tested on the Little dataset;

[0035] Figure 13 Schematic diagram of the confusion matrix obtained by randomly selecting 50 PD samples and 39 HC samples from the three datasets as the test set and all the remaining samples as the training set;

[0036] Figure 14 A diagram showing the confusion matrix of the model from the MIR, Anik, and Ahmed teams, trained on the MDVR-KCL dataset and tested on the Little dataset.

[0037] Figure 15 Schematic diagram of the MLP neural network used to detect Parkinson's disease. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0039] The features and performance of the present invention are further described in detail below with reference to the embodiments.

[0040] like Figure 1 As shown in FIG, the method for constructing a Parkinson's disease detection model based on speech features includes the following steps:

[0041] Step 1: extracting user voice features and obtaining questionnaire results, encoding the questionnaire results, and normalizing the encoded questionnaire results and extracted voice features to obtain normalized questionnaire results and voice features;

[0042] Step 2: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors through the embedding layer. The speech features are mapped into an embedding vector of fixed dimension through the fully connected layer. Each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and then spliced ​​into a single questionnaire embedding vector.

[0043] Step 3: Get the attention score between each embedding vector and the questionnaire embedding, and normalize the score through the softmax function;

[0044] Step 4: Based on the attention score, the embedding vector of the questionnaire results and the embedding vector of the speech features are weightedly fused to obtain a fused feature vector;

[0045] Step 5: Input the obtained fusion feature vector into the risk assessment model to obtain the risk prediction of Parkinson's disease.

[0046] The embedding layer converts the normalized questionnaire results and speech features into high-dimensional representation vectors, wherein the speech features are mapped into fixed-dimensional embedding vectors through a fully connected layer, and each question in the questionnaire is converted into an embedding vector of the same dimension through a fully connected layer, and then concatenated into a single questionnaire embedding vector, including:

[0047] Each speech feature is mapped to an embedding vector of fixed dimension through a fully connected layer. For each question in the questionnaire, it is converted to the same embedding dimension as the mapped speech feature through another fully connected layer, and the embedding vectors of all questions in the questionnaire are spliced ​​into a single questionnaire embedding vector.

[0048] The above method obtains the attention score between each embedding vector and the questionnaire embedding, and normalizes the score through the softmax function, including:

[0049] Calculate the attention score of each embedding vector, and calculate the correlation between the speech feature embedding vector and the questionnaire embedding vector through the dot product or cosine similarity function; for each speech feature embedding vector, obtain the attention score between it and the questionnaire embedding vector, and normalize the attention score through the softmax function so that the sum of the normalized attention scores is 1.

[0050] The method of weightedly fusing the embedding vector of the questionnaire results and the embedding vector of the speech features according to the attention score to obtain a fused feature vector includes:

[0051] The speech feature embedding vectors are weighted and summed using the weights output by the attention layer to obtain a weighted speech feature embedding vector; the weighted speech feature embedding vector is concatenated with the questionnaire embedding vector to obtain a fused feature vector.

[0052] It also includes using the cross-entropy loss function to measure the difference between the predicted probability distribution and the true probability distribution. The cross-entropy loss is calculated by calculating the predicted cross-entropy loss of the questionnaire results and speech features, and performing weighted summation based on the weights assigned according to the feature importance. The calculation formula for the cross-entropy loss is:

[0053] L = α·CEsurvey + β·CEspeech

[0054] Among them, CEsurvey is the cross entropy loss of the questionnaire results, CEspeech is the cross entropy loss of the speech features, and α and β are weights assigned according to the importance of the features.

[0055] A Parkinson's disease detection model construction platform based on speech features, applying any of the above-mentioned Parkinson's disease detection model construction methods based on speech features, including a database module, a data acquisition module, a data preprocessing module, a model selection module, an output module and a performance evaluation module;

[0056] The data acquisition module is connected to the database module, the database module is connected to the data preprocessing module, the data preprocessing module is connected to the model selection module, and the output module and performance evaluation module are respectively connected to the model selection module.

[0057] Specifically, this system uses an MLP model to predict Parkinson's disease. Its unique feature is the use of a joint dataset, including speech samples from 70- and 80-year-olds, for model training to improve generalization. Simultaneously, user speech information is collected through a mini-program, and 15 parameter features are extracted. These features are then labeled based on user feedback about their physical condition and added to the training set, enabling the system to automatically learn.

[0058] Dataset integration and age range processing:

[0059] The model used in this product primarily references three datasets for training: the A.Little dataset, the MDVR-KCL dataset, and the Saarbruecken Voice dataset. Furthermore, by synthesizing age ranges based on the MDVR-KCL dataset, the age diversity of voice samples is expanded. Equal amounts of voice samples from the MDVR-KCL dataset for 70- and 80-year-olds are synthesized to improve the model's generalization across age groups.

[0060] Sound feature selection and optimization:

[0061] During the research and development process, the team analyzed and selected 22 acoustic parameters from a medical perspective for neural network model training. Using a random forest ensemble algorithm in controlled experiments, they ultimately selected 15 key features. This feature compression aims to enhance the model's average accuracy.

[0062] Model selection and training:

[0063] In the process of selecting the model, our team tried a variety of machine learning algorithms and deep learning algorithms, such as support vector machines (SVM), deep neural networks (DNN), multi-layer perceptrons (MLP), etc. Ultimately, MLP was selected by our team due to its ability to handle complex nonlinear features and good robustness. We trained it and obtained the core model of the product. Figure 5 To the attached Figure 8 These are four common machine learning algorithms: MLP, DNN, regression algorithm, and random forest. For the detection confusion matrix of small data sets (the detection set is 12 control group samples and 10 Parkinson's group samples randomly selected from three data sets), it can be seen that the MLP model has outstanding performance in small-scale cross-dataset training and detection. Figure 9 This is the ROC curve during the MLP model training process.

[0064] Strong generalization ability:

[0065] Based on this information and data, it is clear that our prediction system has strong generalization capabilities and maintains stable detection accuracy across data sets. Figure 12 and attached Figure 13 In , we show the confusion matrices of two testing methods to evaluate the performance of the model on different datasets.

[0066] In the attached Figure 12 In the detection experiment, our model was trained on the Saarbruecken Voice Database and MDVR-KCL datasets, and then tested on the Little dataset, with an accuracy of 87.64%, a recall of 84.00%, a precision of 93.33%, and an F1 score of 88.42%.

[0067] In the attached Figure 13 In the detection experiment, our model randomly selected 50 PD samples and 39 HC samples from 3 datasets and retained them as the test set. All the remaining samples were used as the training set. The obtained model achieved an accuracy of 94.38%, a recall rate of 92.00%, a precision of 97.87%, and an F1 score of 94.85%.

[0068] We replicated three high-accuracy models from cutting-edge research. Although these models achieved over 90% detection accuracy when trained on the MDVR-KCL dataset and tested on the same dataset, in cross-dataset testing, after testing on the Little dataset, the test results of each team are as follows:

[0069] Anik team's test results:

[0070] Accuracy:0.7538461538461538

[0071] Precision:0.7538461538461538

[0072] Recall:1.0

[0073] F1 Score:0.8596491228070176

[0074] AUC-ROC:0.4421060090702948

[0075] Ahmed team's test results:

[0076] Accuracy:0.7282051282051282

[0077] Precision:0.7582417582417582

[0078] Recall:0.9387755102040817

[0079] F1 Score:0.8389057750759877

[0080] AUC-ROC:0.5110544217687075

[0081] The test results of the MIR team are:

[0082] Accuracy:0.40512820512820513

[0083] Precision:0.6597938144329897

[0084] Recall:0.43537414965986393

[0085] F1 Score:0.5245901639344263

[0086] AUC-ROC:0.32029478458049887

[0087] It can be seen that the accuracy of other models drops significantly when detecting across datasets. Our model significantly exceeds the average accuracy of other models with an average accuracy of 90.075%, demonstrating the significant improvement of our model in processing speech data and content collected in different scenarios. Figure 12 , attached Figure 13 With attached Figure 14 The comparison confirms the robustness and stability of our model in early warning of individuals who may have PD, because we focus more on recall and accuracy of detection across datasets to predict the risk of disease under different sampling conditions as much as possible.

[0088] Application development:

[0089] Our team also developed a lightweight and convenient application platform with the help of WeChat Mini Program, which allows users to use the application without downloading and installing. Using this platform, our team developed a user-friendly and interactive application terminal, providing users with an easy-to-access and easy-to-use Parkinson's disease prediction tool. The backend stores 15 features of the user's voice and uses the user's physical condition form (attached) to calculate the patient's disease risk. Figure 3 The data is labeled and automatically transferred from the database to the backend for training every month.

[0090] Table 1. True Positives (TP): correctly classified Parkinson’s disease samples, True Negatives (TN): correctly classified healthy samples, False Negatives (FN): incorrectly classified Parkinson’s disease samples as healthy, False Positives (FP): incorrectly classified healthy samples as Parkinson’s disease samples.

[0091] DETAILED DESCRIPTION

[0093] Feature extraction:

[0094] Since we wanted to find some features with strong universality to ensure the generalization performance of the model, we selected 22 common features from 10 IEEE papers and used the control variable method to select 15 of them with stable differences in cross-dataset tests.

[0095] The following are the feature metrics used in our experiments for training and prediction:

[0096] MDVP:Fo(Hz)-average fundamental frequency of sound

[0097] MDVP:Fhi(Hz)-maximum sound fundamental frequency

[0098] MDVP:Flo(Hz)-minimum sound fundamental frequency

[0099] MDVP:Jitter(%), MDVP:Jitter(Abs), MDVP:RAP, MDVP:PPQ, Jitter:DDP - Several ways to measure fundamental frequency variation

[0100] MDVP:Shimmer (%), MDVP:Shimmer (dB), Shimmer:APQ3, Shimmer:APQ5, Shimmer:DDA - Several ways to measure amplitude changes

[0101] NHR, HNR: Two ways to measure the ratio of noise to pitch in sound

[0102] Calculate MDVP:Fo(Hz), MDVP:Fhi(Hz) and MDVP:Flo(Hz)

[0103] Read audio:

[0104] Given an audio path audio_path, we get the sound signal s(t) of the audio.

[0105] s(t)=Sound(audio_path)

[0106] Calculate the fundamental frequency trajectory:

[0107] We extract the fundamental frequency trajectory P(t) from the sound signal s(t).

[0108] P(t)=Pitch(s(t))

[0109] Extract non-zero fundamental frequency values:

[0110] From the pitch trace P(t), we extract the non-zero pitch values ​​and store them as a sequence P′.

[0111] P′={p∈P(t)|p≠0}

[0112] Calculate MDVP_Fo_Hz (mean fundamental frequency):

[0113]

[0114] where N is the number of elements in P′.

[0115] Calculate MDVP_Fhi_Hz (highest fundamental frequency):

[0116] MDVP_Fhi_Hz=max(P′)

[0117] Calculate MDVP_Flo_Hz (minimum fundamental frequency):

[0118] MDVP_Flo_Hz=min(P′)

[0119] Calculation of NHR and HNR

[0120] Calculate the spectrum of a sound:

[0121] S(f)=FFT{s(t)}

[0122] Where S(f) is the spectrum of the sound signal s(t).

[0123] Calculate the fundamental frequency F0:

[0124] F0=mean(Pitch(s(t)))

[0125] Here, Pitch(s(t)) is the fundamental frequency trajectory of the sound signal.

[0126] Define the frequencies of the first 4 harmonics:

[0127] f h =n×F0,n∈{1,2,3,4}

[0128] For each harmonic f h , calculate its energy:

[0129]

[0130] Where, bandwidth = 50Hz.

[0131] Calculate the total energy of all harmonics:

[0132]

[0133] Calculate the total energy of the sound signal:

[0134]

[0135] Calculate the noise energy:

[0136] E noise =E total -E harmonics

[0137] Calculate NHR:

[0138]

[0139] Calculate HNR:

[0140]

[0141] Calculate MDVP:Jitter(%), MDVP:Jitter(Abs), MDVP:RAP, MDVP:PPQ, Jitter:DDP

[0142] Extract the fundamental frequency trajectory:

[0143] P(t)=Pitch(s(t))

[0144] Where P(t) is the fundamental frequency trajectory of the sound signal s(t).

[0145] Calculate the jitter percentage (Jitter Percent):

[0146]

[0147] Where N is the number of fundamental frequency values.

[0148] Calculate the absolute jitter (Jitter Absolute):

[0149] Jitter Absolute=mean(|ΔP|)

[0150] where ΔP is the difference between adjacent fundamental frequency values.

[0151] Calculate RAP (Relative Average Perturbation):

[0152]

[0153] Calculate PPQ (Pitch Perturbation Quotient):

[0154]

[0155] Calculate MDVP:Shimmer(%), MDVP:Shimmer(dB), Shimmer:APQ3, Shimmer:APQ5, Shimmer:DDA

[0156] Extract amplitude trace:

[0157] A(t)=Amplitude(s(t))

[0158] Calculate the Shimmer Percent:

[0159]

[0160] Calculate Shimmer in dB:

[0161] Assuming we use a small offset for slight amplitude adjustments to avoid taking the logarithm of 0, then:

[0162] Shimmer dB=20×mean(log 10 (|A|+offset) i+1 -log 10 (|A|+offset) i )

[0163] Calculate APQ3:

[0164]

[0165] Calculate APQ5:

[0166]

[0167] Calculate DDA:

[0168] DDA=3×mean(|A(i)-mean(A(i-1),A(i),A(i+1))|)

[0169] 2. Dataset training: Combine the speech samples of 70-year-olds and 80-year-olds with other data sets and use the MLP model for training to improve the generalization ability of the model. Figure 11 These are MFCC images of the 70-year-old and 80-year-old healthy control group and Parkinson's disease patient group synthesized using Pydub based on the MDVR-KCL dataset.

[0170] 2. System Overview:

[0171] This paper presents a highly generalizable, automated learning Parkinson's disease prediction system based on speech processing. It aims to assess a user's risk of Parkinson's disease based on five different speech samples. The system, presented as a WeChat mini-program called "SoundHealth," stores 15 features of the user's speech on the backend. The system primarily includes the following components:

[0172] In the above-mentioned Parkinson's disease identification scheme, in order to use feature selection techniques to reduce irrelevant or redundant features, we can adopt a variety of specific feature selection algorithms. The following is a brief description of several commonly used feature selection algorithms and how they are applied:

[0173] Filter:

[0174] Mutual Information: Calculate the mutual information between each feature and the target variable, sort them by mutual information value, and select features with higher mutual information values. Mutual information directly measures the correlation between features and the target variable.

[0175] Chi-squared Test: Applicable to classification problems, it evaluates the correlation between each feature and the target variable by calculating the chi-square statistic between them, and selects the feature with the higher statistic.

[0176] Correlation coefficient: For continuous features, you can use the correlation coefficient (such as the Pearson correlation coefficient) to measure the linear relationship between the feature and the target variable, and select features with stronger correlation.

[0177] Wrapper (wrapping method):

[0178] Recursive Feature Elimination (RFE): This technique combines machine learning models (such as support vector machines and decision trees) with feature selection. It selects features by recursively considering smaller and smaller sets of features. First, the model is trained on the original feature set, and the importance of each feature is evaluated using metrics such as coefficients and feature importance. Then, the least important features are removed, and the model is retrained on the remaining feature set. This process is repeated until the desired number of features is reached.

[0179] Embedded (embedded method):

[0180] Model-based feature selection: Some machine learning algorithms (such as random forests and gradient boosted trees) automatically evaluate the importance of each feature during training. We can use these algorithms to train the model and perform feature selection based on the feature importance output by the model. For example, in random forests, the importance of each feature can be measured by calculating the average reduction in impurity across the trees.

[0181] When applying these feature selection algorithms, the general steps are as follows:

[0182] Preprocess the data, including data cleaning, feature extraction, etc.

[0183] Select a suitable feature selection algorithm and set the corresponding parameters.

[0184] Run the feature selection algorithm on the training data to obtain the feature importance ranking or the selected feature subset. The appropriate selection algorithm can be determined based on the final model output structure.

[0185] Based on the feature selection results, irrelevant or redundant features are eliminated, and features that contribute more to model performance are retained.

[0186] Retrain the model using the selected subset of features and evaluate the model's performance.

[0187] Voice Collection and Input: Users input voice input through the WeChat mini-program "Sound Health." The system receives five different voice samples, covering the user's daily pronunciation, to more comprehensively and accurately assess Parkinson's disease risk.

[0188] Feature extraction: The system extracts 15 features from user speech, including but not limited to MDVP:Fo (average fundamental frequency), MDVP:Jitter (vibration rate), MDVP:Shimmer (voice instability), etc. These features have been shown to have strong universality on multiple datasets.

[0189] Questionnaire assessment: Users complete a questionnaire, and the system uses this information to further assess Parkinson's disease risk. By integrating questionnaire results with speech features, the system's accuracy is improved.

[0190] Automatic Learning and Model Updates: The system automatically retrieves user voice samples and relevant questionnaire results from the database monthly to update the training model. This system uses a joint training approach, integrating the Saarbruecken Voice Database, Oxford Parkinson's Disease Detection Dataset, and the Mobile Device Voice Recordings at King's College London dataset, to improve the model's generalization capabilities.

[0191] Specifically:

[0192] The core idea of ​​this algorithm is to assign different weights to different features based on their contribution to Parkinson's disease risk assessment. Through the attention mechanism, the model can automatically learn and adjust these weights, thereby effectively integrating questionnaire results and speech features.

[0193] The specific implementation steps are as follows:

[0194] 1. Data preprocessing:

[0195] oEncode and normalize the questionnaire results and convert them into numerical data.

[0196] o Perform the same normalization process on the extracted speech features to ensure that the data are at the same scale.

[0197] 2. Feature Embedding:

[0198] Use embedding layers (such as fully connected layers or convolutional layers) to convert the questionnaire results and speech features into high-dimensional representation vectors respectively: Speech feature embedding: Use a fully connected layer to convert each speech feature into an embedding vector of a fixed dimension. For example, each feature can be mapped to a 128-dimensional vector. Questionnaire embedding: For each question in the questionnaire, use another fully connected layer to convert it to the same embedding dimension (for example, 128 dimensions). Then, the embedding vectors of all questions can be concatenated or average-pooled into a single questionnaire embedding vector.

[0199] 3. Attention Mechanism:

[0200] An attention mechanism is introduced to dynamically adjust the weights of different features. By calculating an attention score, the model can assign a weight to each feature to reflect its importance in Parkinson's disease risk assessment. The calculation of the attention score can be based on the correlation, importance, or other metrics between features. Specifically: In this layer, we calculate the attention score for each embedding vector. The correlation between the speech feature embedding and the questionnaire embedding can be calculated using dot product, cosine similarity, or more complex functions.

[0201] For each embedding vector, we calculate the attention score between it and the questionnaire embedding and normalize these scores through the softmax function to ensure that their sum is 1.

[0202] 4. Feature Fusion:

[0203] According to the attention score, the embedding vector of the questionnaire results and the embedding vector of the speech features are weightedly fused.

[0204] 5. Risk Assessment:

[0205] o Input the fused feature vector into a risk assessment model (such as a classifier or regression model).

[0206] o Model predicts or estimates Parkinson's disease risk based on fusion features

[0207] Assume we have 15 speech features and a questionnaire result containing multiple questions. We can design a network structure that includes an embedding layer, an attention layer, and a fusion layer.

[0208] Embedding layer:

[0209] Speech feature embedding: Use a fully connected layer to convert each speech feature into an embedding vector of fixed dimension. For example, each feature can be mapped to a 128-dimensional vector.

[0210] Questionnaire embedding: For each question in the questionnaire, another fully connected layer is used to convert it to the same embedding dimension (e.g. 128). The embedding vectors of all questions can then be concatenated or average-pooled into a single questionnaire embedding vector.

[0211] Attention layer:

[0212] In this layer, we calculate the attention score for each embedding vector. We can use dot product, cosine similarity, or more complex functions to calculate the correlation between the speech feature embedding and the questionnaire embedding.

[0213] For each embedding vector, we compute the attention score between it and the questionnaire embedding and normalize these scores through the softmax function to ensure that their sum is 1.

[0214] Fusion layer:

[0215] Using the weights output by the attention layer, the speech feature embeddings are weighted and summed to obtain a weighted speech feature embedding vector.

[0216] The weighted speech feature embedding vector is concatenated or added to the questionnaire embedding vector to obtain the final fused feature vector.

[0217] In the above scheme, for the task of fusing questionnaire results and speech features, a cross-entropy loss function can be used. The cross-entropy loss function is commonly used in classification problems and can measure the difference between the predicted probability distribution and the true probability distribution. Specifically, the cross-entropy loss is calculated for the questionnaire results and the speech feature predictions respectively, and then they are weighted and summed according to their importance:

[0218] [L=\alpha\cdot CE_{survey}+\beta\cdot CE_{speech}]

[0219] Where (CE_{survey}) is the cross entropy loss of the survey results, (CE_{speech}) is the cross entropy loss of the speech features, and (\alpha) and (\beta) are weights assigned according to feature importance.

[0220] In summary, the MLP neural network is suitable for detecting Parkinson's disease.

[0221] 3. System implementation details:

[0222] Database Construction: The system was jointly trained using the MDVR-KCL dataset, the Saarbruecken Voice Database, and the Little dataset. The MDVR-KCL dataset contains samples from 16 Parkinson's patients and 21 healthy controls, while the Saarbruecken Voice Database contains 1,532 samples from healthy individuals and others, and 13 samples from Parkinson's patients. The Little dataset consists of 195 samples recorded from 23 Parkinson's patients and 8 healthy controls.

[0223] Data preprocessing: To address data imbalance, we used Synthetic Minority Over-sampling Technique (SMOTE) for data augmentation. Furthermore, to simulate age-specific characteristics, we synthesized samples by adjusting the playback speed of the original audio.

[0224] Model Selection and Training: Among various machine learning algorithms, a neural network model based on a multilayer perceptron (MLP) was selected for the speech prediction problem. The MLP model demonstrated good generalization capabilities with limited labeled data in multiple experiments. Stochastic gradient descent (SGD) was used to optimize the model, and the ReLU activation function was applied to improve learning performance.

[0225] Since there are few samples of Parkinson's disease patients in the dataset, SMOTE (Synthetic Minority Over-sampling Technique) can be used for oversampling to generate synthetic samples to balance the dataset.

[0226] Another approach is to employ resampling techniques such as undersampling the majority class or using class weights to give more attention to the minority class in the loss function.

[0227] Extract meaningful features from raw audio data, such as Mel-frequency cepstral coefficients (MFCCs), spectrogram features, etc.

[0228] Consider using feature selection techniques to reduce irrelevant or redundant features to improve the generalization ability of the model.

[0229] Performance Evaluation: Cross-dataset testing demonstrates the model's performance under different environments and speech acquisition conditions. Metrics such as ROC curves, confusion matrices, accuracy, recall, and precision are used to comprehensively evaluate system performance.

[0230] 4. Implementation process:

[0231] After listening to the sample demonstration in the WeChat mini-program "Sound Health", users enter the recording interface, follow the prompts to record 5 audios, and fill out the relevant questionnaire.

[0232] The system receives and stores 15 features extracted from the user's voice sample.

[0233] The questionnaire information filled out by users is combined with voice features to conduct a comprehensive assessment of Parkinson's disease risk.

[0234] The system automatically updates the model every month and uses newly acquired data for training to continuously improve the prediction accuracy for Asians.

[0235] Users can obtain risk assessment results and relevant medical advice at any time through the WeChat mini program.

[0236] This implementation plan not only considers feature extraction of user voice input, but also combines multi-source data for joint training to improve the generalization ability of the model, making it more suitable for users of different ages, contexts and regions.

[0237] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A method for constructing a Parkinson's disease detection model based on speech features, comprising the following steps: Step 1: extracting user voice features and obtaining questionnaire results, encoding the questionnaire results, and normalizing the encoded questionnaire results and extracted voice features to obtain normalized questionnaire results and voice features; Step 2: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors through the embedding layer. The speech features are mapped into an embedding vector of fixed dimension through the fully connected layer. Each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and then spliced ​​into a single questionnaire embedding vector. Each speech feature is mapped to an embedding vector of fixed dimension through a fully connected layer. For each question in the questionnaire, another fully connected layer converts it to the same embedding dimension as the mapped speech feature, and the embedding vectors of all questions in the questionnaire are concatenated into a single questionnaire embedding vector. Step 3: Get the attention score between each embedding vector and the questionnaire embedding, and normalize the score through the softmax function; The attention score of each embedding vector is calculated, and the correlation between the speech feature embedding vector and the questionnaire embedding vector is calculated by the dot product or cosine similarity function. For each speech feature embedding vector, the attention score between it and the questionnaire embedding vector is obtained, and the attention score is normalized by the softmax function so that the sum of the normalized attention scores is 1. Step 4: Based on the attention score, the embedding vector of the questionnaire results and the embedding vector of the speech features are weightedly fused to obtain a fused feature vector; Step 5: Input the obtained fusion feature vector into the risk assessment model to obtain the risk prediction of Parkinson's disease.

2. The method for constructing a Parkinson's disease detection model based on speech features according to claim 1, wherein: The method of weightedly fusing the embedding vector of the questionnaire results and the embedding vector of the speech features according to the attention score to obtain a fused feature vector includes: The speech feature embedding vectors are weighted and summed using the weights output by the attention layer to obtain a weighted speech feature embedding vector; the weighted speech feature embedding vector is concatenated with the questionnaire embedding vector to obtain a fused feature vector.

3. The method for constructing a Parkinson's disease detection model based on speech features according to claim 2, characterized in that: It also includes using the cross-entropy loss function to measure the difference between the predicted probability distribution and the true probability distribution. The cross-entropy loss is calculated by calculating the predicted cross-entropy loss of the questionnaire results and speech features, and performing weighted summation based on the weights assigned according to the feature importance. The calculation formula for the cross-entropy loss is: L = α ⋅ CEsurvey + β ⋅ CEspeech in, CEsurvey is the cross entropy loss of the questionnaire results, CEspeech is the cross entropy loss of speech features, α and β is the weight assigned according to the feature importance.

4. A Parkinson's disease detection model construction platform based on speech features, characterized by: The method for constructing a Parkinson's disease detection model based on speech features according to any one of claims 1 to 3 is applied, comprising a database module, a data acquisition module, a data preprocessing module, a model selection module, an output module, and a performance evaluation module; The data acquisition module is connected to the database module, the database module is connected to the data preprocessing module, the data preprocessing module is connected to the model selection module, and the output module and performance evaluation module are respectively connected to the model selection module.

Images