Parkinson's disease detection model construction method and platform based on voice features

Abstract

The invention discloses a Parkinson's disease detection model construction method and platform based on voice features, and the method comprises the steps: carrying out the normalization processing of an encoded questionnaire result and extracted voice features, and obtaining a normalized questionnaire result and voice features; respectively converting the normalized questionnaire result and the voice features into high-dimensional representation vectors through an embedded layer; obtaining an attention score between each embedding vector and questionnaire embedding, and normalizing the score through a softmax function; according to the attention score, performing weighted fusion on the embedded vector of the questionnaire result and the embedded vector of the voice feature to obtain a fused feature vector; and inputting the obtained fusion feature vector into a risk assessment model to obtain the risk prediction of the Parkinson's disease. Through the technical scheme provided by the invention, a convenient and accurate prediction service can be provided for the Parkinson's disease patient.

Description

Technical Field

[0001] The present invention relates to the field of speech processing, and in particular to a method and platform for constructing a Parkinson's disease detection model based on speech features. Background Art

[0002] In recent years, with the rapid development of artificial intelligence technology, speech detection has gradually attracted attention as a non-invasive and cost-effective method. Humans produce airflow through the lungs, causing the vocal cords to vibrate and make sounds. This process involves complex vocal tract structure and nervous system regulation. The degeneration of the nervous system in patients with Parkinson's disease can lead to the decline of the central nervous system and autonomic nervous system nerves related to voice production, as well as muscle stiffness, thus affecting the control of the vocal muscles. Therefore, by analyzing the speech characteristics of patients with Parkinson's disease, it is possible to provide a new way to predict Parkinson's disease.

[0003] Although early studies such as Little et al. have revealed some correlations between PD and speech parameters, existing speech detection models still have the problem of insufficient generalization. These models often rely on specific datasets for training and verification, resulting in poor performance when facing different groups of people or different speech environments. In addition, speech detection in real-world scenarios also faces challenges such as irregular user pronunciation and inconsistent content, so the model is required to have high fault tolerance and be independent of the content of the speech.

[0004] Internationally, research on the prediction of Parkinson's disease using voice data combined with artificial intelligence technology has made some progress, but most of the research is still in the scientific research stage and has not yet been widely transformed into practical applications. In China, although related research started late, it has developed rapidly, especially in the application of deep learning technology. However, current research results are still mainly limited to laboratories or medical professionals, and lack public service applications for the general public. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method for constructing a Parkinson's disease detection model based on speech features, comprising the following steps:

[0006] Step 1: extracting user voice features and obtaining questionnaire results, encoding the questionnaire results, and normalizing the encoded questionnaire results and the extracted voice features to obtain normalized questionnaire results and voice features;

[0007] Step 2: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors through the embedding layer. The speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer and concatenated into a single questionnaire embedding vector.

[0008] Step 3: Get the attention score between each embedding vector and the questionnaire embedding, and normalize the score through the softmax function;

[0009] Step 4: According to the attention score, the embedding vector of the questionnaire result and the embedding vector of the speech feature are weighted fused to obtain a fused feature vector;

[0010] Step 5: Input the obtained fused feature vector into the risk assessment model to obtain the risk prediction of Parkinson's disease.

[0011] Furthermore, the normalized questionnaire results and speech features are converted into high-dimensional representation vectors respectively through the embedding layer, wherein the speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question of the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and spliced ​​into a single questionnaire embedding vector, including:

[0012] Each speech feature is mapped to an embedding vector of fixed dimension through a fully connected layer. For each question in the questionnaire, it is converted to the same embedding dimension as the speech feature after mapping through another fully connected layer, and the embedding vectors of all questions in the questionnaire are concatenated into a single questionnaire embedding vector.

[0013] Furthermore, the step of obtaining the attention score between each embedding vector and the questionnaire embedding and normalizing the score by a softmax function includes:

[0014] Calculate the attention score of each embedding vector, and calculate the correlation between the speech feature embedding vector and the questionnaire embedding vector through the dot product or cosine similarity function; for each speech feature embedding vector, obtain the attention score between it and the questionnaire embedding vector, and normalize the attention score through the softmax function so that the sum of the normalized attention scores is 1.

[0015] Furthermore, the embedding vector of the questionnaire result and the embedding vector of the speech feature are weightedly fused according to the attention score to obtain a fused feature vector, including:

[0016] The speech feature embedding vectors are weighted and summed using the weights output by the attention layer to obtain a weighted speech feature embedding vector; the weighted speech feature embedding vector is concatenated with the questionnaire embedding vector to obtain a fused feature vector.

[0017] Furthermore, the cross entropy loss function is used to measure the difference between the predicted probability distribution and the true probability distribution. The predicted cross entropy loss of the questionnaire results and the speech features is calculated, and weighted summation is performed according to the weights assigned according to the feature importance. The calculation formula of the cross entropy loss is:

[0018] L=α·CEsurvey+β·CEspeech

[0019] Among them, CEsurvey is the cross entropy loss of the survey results, CEspeech is the cross entropy loss of the speech features, and α and β are weights assigned according to the importance of the features.

[0020] A Parkinson's disease detection model construction platform based on speech features, which applies the Parkinson's disease detection model construction method based on speech features, including a database module, a data acquisition module, a data preprocessing module, a model selection module, an output module and a performance evaluation module;

[0021] The data acquisition module is connected to the database module, the database module is connected to the data preprocessing module, the data preprocessing module is connected to the model selection module, and the output module and the performance evaluation module are respectively connected to the model selection module.

[0022] The beneficial effects of the present invention are: by comprehensively utilizing the A.Little dataset, the MDVR-KCL dataset, and the Saarbruecken Voice dataset, this product expands the age range of voice samples and improves the generalization ability of the model. In terms of voice feature selection, after medical analysis and optimization of the random forest algorithm, 15 key features are selected from 22 acoustic parameters to enhance the accuracy of the model. By developing a lightweight and convenient application platform WeChat applet, we have innovated the dataset training and applet collection mechanism, provided a user-friendly and highly interactive application platform, allowing users to easily access and use this diagnostic tool, and maximize the use of user-input voice information to enhance the system's voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of the method for building a Parkinson's disease detection model based on speech features;

[0024] Figure 2 Flowchart for MLP model training;

[0025] Figure 3 Use flowcharts for users;

[0026] Figure 4 Rank the 15 characteristics for their association with Parkinson's disease;

[0027] Figure 5 Schematic diagram of MLP model small-scale data cross-dataset training and detection;

[0028] Figure 6 Schematic diagram of small-scale cross-dataset training and detection of DNN models;

[0029] Figure 7 Schematic diagram of small-scale data cross-dataset training and detection for regression models;

[0030] Figure 8 Schematic diagram of small-scale data cross-dataset training and detection for random forest models;

[0031] Fig. 9 Schematic diagram of ROC curve for small-scale training of MLP model;

[0032] Fig.10 This is a schematic diagram of the discrimination between the patient group and the control group samples after integrating the 15 features in the three data sets;

[0033] Fig.11 This is a schematic diagram of the MFCC images of the 70-year-old and 80-year-old healthy control group and Parkinson's disease patient group synthesized by Pydub based on the MDVR-KCL dataset;

[0034] Fig.12 This is a diagram of the confusion matrix obtained after the model was trained on the Saarbruecken Voice Database and MDVR-KCL datasets and then tested on the Little dataset;

[0035] Fig.13 Schematic diagram of the confusion matrix obtained by randomly selecting 50 PD samples and 39 HC samples from 3 datasets as the test set and using all the remaining samples as the training set;

[0036] Fig.14 A diagram of the confusion matrix of the model of MIR, Anik and Ahmed team, trained with MDVR-KCL dataset and tested with Little dataset;

[0037] Fig.15 Schematic diagram of the MLP neural network used to detect Parkinson's disease. DETAILED DESCRIPTION

[0038] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings, but the protection scope of the present invention is not limited to the following.

[0039] The features and performance of the present invention are further described in detail below in conjunction with the embodiments.

[0040] like Figure 1 As shown, the method for constructing a Parkinson's disease detection model based on speech features includes the following steps:

[0041] Step 1: extracting user voice features and obtaining questionnaire results, encoding the questionnaire results, and normalizing the encoded questionnaire results and the extracted voice features to obtain normalized questionnaire results and voice features;

[0042] Step 2: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors through the embedding layer. The speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer and concatenated into a single questionnaire embedding vector.

[0043] Step 3: Get the attention score between each embedding vector and the questionnaire embedding, and normalize the score through the softmax function;

[0044] Step 4: According to the attention score, the embedding vector of the questionnaire result and the embedding vector of the speech feature are weighted fused to obtain a fused feature vector;

[0045] Step 5: Input the obtained fused feature vector into the risk assessment model to obtain the risk prediction of Parkinson's disease.

[0046] The normalized questionnaire results and speech features are converted into high-dimensional representation vectors respectively through the embedding layer, wherein the speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question of the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and spliced ​​into a single questionnaire embedding vector, including:

[0047] Each speech feature is mapped to an embedding vector of fixed dimension through a fully connected layer. For each question in the questionnaire, it is converted to the same embedding dimension as the speech feature after mapping through another fully connected layer, and the embedding vectors of all questions in the questionnaire are concatenated into a single questionnaire embedding vector.

[0048] The method of obtaining the attention score between each embedding vector and the questionnaire embedding and normalizing the score through the softmax function includes:

[0049] Calculate the attention score of each embedding vector, and calculate the correlation between the speech feature embedding vector and the questionnaire embedding vector through the dot product or cosine similarity function; for each speech feature embedding vector, obtain the attention score between it and the questionnaire embedding vector, and normalize the attention score through the softmax function so that the sum of the normalized attention scores is 1.

[0050] The method of weightedly fusing the embedding vector of the questionnaire results and the embedding vector of the speech features according to the attention score to obtain a fused feature vector includes:

[0051] The speech feature embedding vectors are weighted and summed using the weights output by the attention layer to obtain a weighted speech feature embedding vector; the weighted speech feature embedding vector is concatenated with the questionnaire embedding vector to obtain a fused feature vector.

[0052] It also includes using the cross entropy loss function to measure the difference between the predicted probability distribution and the true probability distribution, by calculating the predicted cross entropy loss of the questionnaire results and speech features, and performing weighted summation based on the weights assigned according to the feature importance, where the calculation formula for the cross entropy loss is:

[0053] L=α·CEsurvey+β·CEspeech

[0054] Among them, CEsurvey is the cross entropy loss of the survey results, CEspeech is the cross entropy loss of the speech features, and α and β are weights assigned according to the importance of the features.

[0055] A Parkinson's detection model construction platform based on speech features, applying any of the Parkinson's detection model construction methods based on speech features, including a database module, a data acquisition module, a data preprocessing module, a model selection module, an output module and a performance evaluation module;

[0056] The data acquisition module is connected to the database module, the database module is connected to the data preprocessing module, the data preprocessing module is connected to the model selection module, and the output module and the performance evaluation module are respectively connected to the model selection module.

[0057] Specifically, this system uses the MLP model to predict Parkinson's disease. Its feature is that it uses a joint data set, including voice samples of 70-year-olds and 80-year-olds, to train the model to improve generalization. At the same time, the mini program collects user voice information, extracts 15 parameter features, and labels them in combination with the user's feedback on the physical condition, and adds them to the training set to achieve automatic learning of the system.

[0058] Dataset integration and age range processing:

[0059] The model used in this product mainly refers to three datasets to assist in training, namely the A.Little dataset, the MDVR-KCL dataset, and the Saarbruecken Voice dataset. In addition, by synthesizing the age range based on the MDVR-KCL dataset, the age diversity of the voice samples is expanded, and the same amount of voice samples of 70-year-old and 80-year-old MDVR-KCL datasets are synthesized to improve the model's generalization ability for age.

[0060] Sound feature selection and optimization:

[0061] During the research and development process, the team selected 22 acoustic parameters from a medical perspective for neural network model training. Through a random forest ensemble algorithm, a controlled experiment was conducted and 15 key features were finally selected. This feature compression is designed to enhance the average accuracy of the model.

[0062] Model selection and training:

[0063] In the process of selecting the model, our team tried a variety of machine learning algorithms and deep learning algorithms, such as support vector machine (SVM), deep neural network (DNN), multi-layer perceptron (MLP), etc. In the end, MLP was selected and trained by our team due to its ability to handle complex nonlinear features and good robustness, and the core model of the product was obtained. Figure 5 To the attached Figure 8 These are four common machine learning algorithms: MLP, DNN, regression algorithm, and random forest. For the detection confusion matrix of small data sets (the detection set is 12 control group samples and 10 Parkinson's group samples randomly selected from three data sets), it can be seen that the MLP model has outstanding performance in small-scale cross-dataset training and detection. Fig. 9 This is the ROC curve during the MLP model training process.

[0064] Strong generalization ability:

[0065] Based on this information and data, it is clear that our prediction system has strong generalization capabilities and maintains a stable detection accuracy across data sets. Fig.12 and attached Fig.13 In , we show the confusion matrices of two test methods for evaluating the performance of the models on different datasets.

[0066] In the attached Fig.12 In the detection experiment, our model was trained on the Saarbruecken Voice Database and MDVR-KCL datasets, and then tested on the Little dataset, with an accuracy of 87.64%, a recall of 84.00%, a precision of 93.33%, and an F1 score of 88.42%.

[0067] In the attached Fig.13 In the detection experiment, our model randomly selected 50 PD samples and 39 HC samples from the three datasets and retained them as the test set, and all the remaining samples were used as the training set. The obtained model achieved an accuracy of 94.38%, a recall rate of 92.00%, a precision of 97.87%, and an F1 score of 94.85%.

[0068] We reproduced three high-accuracy models from cutting-edge research. Although these models achieved more than 90% detection accuracy when trained with the MDVR-KCL dataset and tested on the same dataset, in cross-dataset detection, after testing on the Little dataset, the detection results of each team are as follows:

[0069] Anik team's test results:

[0070] Accuracy:0.7538461538461538

[0071] Precision:0.7538461538461538

[0072] Recall:1.0

[0073] F1 Score:0.8596491228070176

[0074] AUC-ROC: 0.4421060090702948

[0075] Ahmed team's test results:

[0076] Accuracy:0.7282051282051282

[0077] Precision:0.7582417582417582

[0078] Recall:0.9387755102040817

[0079] F1 Score:0.8389057750759877

[0080] AUC-ROC: 0.5110544217687075

[0081] The test results of the MIR team are:

[0082] Accuracy:0.40512820512820513

[0083] Precision:0.6597938144329897

[0084] Recall:0.43537414965986393

[0085] F1 Score:0.5245901639344263

[0086] AUC-ROC: 0.32029478458049887

[0087] It can be seen that the accuracy of other models drops significantly when detecting across datasets. Our model significantly exceeds the average accuracy of other models with an average accuracy of 90.075%, demonstrating the significant improvement of our model in processing speech data and content collected in different scenarios. Fig.12 , attached Fig.13 With attached Fig.14 The comparison confirms the robustness and stability of our model in early warning of individuals who may have PD, because we focus more on recall and accuracy of detection across datasets to predict the risk of disease under different sampling conditions as much as possible.

[0088] Application development:

[0089] Our team also developed a lightweight and convenient application platform with the help of WeChat Mini Program, which allows users to use applications without downloading and installing. Using this platform, our team developed a user-friendly and interactive application terminal, providing users with an easy-to-access and easy-to-use Parkinson's disease prediction tool. The backend stores 15 features of the user's voice and uses the physical condition form filled out by the user (attached) to generate a prediction result. Figure 3 For this flowchart), the data is labeled and automatically transferred from the database to the backend for training every month.

[0090] Table 1. True Positives (TP): correctly classified Parkinson’s samples, True Negatives (TN): correctly classified healthy samples, False Negatives (FN): incorrectly classified Parkinson’s samples as healthy, False Positives (FP): incorrectly classified healthy samples as Parkinson’s.

[0091] DETAILED DESCRIPTION

[0093] Feature extraction:

[0094] Since we want to find some features with strong universality to ensure the generalization performance of the model, we selected 22 common features from 10 IEEE papers and selected 15 of them with stable differences in cross-dataset tests through the control variable method.

[0095] The following are the feature metrics used in our experiments for training and prediction:

[0096] MDVP:Fo(Hz)-average fundamental frequency of sound

[0097] MDVP:Fhi(Hz)-maximum sound fundamental frequency

[0098] MDVP:Flo(Hz)-Minimum fundamental frequency

[0099] MDVP:Jitter(%)、MDVP:Jitter(Abs)、MDVP:RAP、MDVP:PPQ、Jitter:DDP - Several ways to measure fundamental frequency changes

[0100] MDVP:Shimmer(%)、MDVP:Shimmer(dB)、Shimmer:APQ3、Shimmer:APQ5、Shimmer:DDA - Several ways to measure amplitude changes

[0101] NHR, HNR: Two ways to measure the ratio of noise to tone in sound

[0102] Calculate MDVP:Fo(Hz), MDVP:Fhi(Hz) and MDVP:Flo(Hz)

[0103] Read audio:

[0104] Given an audio path audio_path, we get the sound signal s(t) of the audio.

[0105] s(t)=Sound(audio_path)

[0106] Calculate the fundamental frequency trajectory:

[0107] We extract the fundamental frequency trajectory P(t) from the sound signal s(t).

[0108] P(t)=Pitch(s(t))

[0109] Extract non-zero fundamental frequency values:

[0110] From the pitch trace P(t), we extract the non-zero pitch values ​​and store them as a sequence P′.

[0111] P′={p∈P(t)|p≠0}

[0112] Calculate MDVP_Fo_Hz (mean fundamental frequency):

[0113]

[0114] where N is the number of elements in P'.

[0115] Calculate MDVP_Fhi_Hz (highest fundamental frequency):

[0116] MDVP_Fhi_Hz=max(P′)

[0117] Calculate MDVP_Flo_Hz (minimum fundamental frequency):

[0118] MDVP_Flo_Hz=min(P′)

[0119] Calculation of NHR and HNR

[0120] Calculate the spectrum of a sound:

[0121] S(f)=FFT{s(t)}

[0122] Where S(f) is the frequency spectrum of the sound signal s(t).

[0123] Calculate the fundamental frequency F0:

[0124] F0=mean(Pitch(s(t)))

[0125] Here, Pitch(s(t)) is the fundamental frequency trajectory of the sound signal.

[0126] Define the frequencies of the first 4 harmonics:

[0127] f h =n×F0,n∈{1,2,3,4}

[0128] For each harmonic f h , calculate its energy:

[0129]

[0130] Among them, bandwidth = 50Hz.

[0131] Calculate the total energy of all harmonics:

[0132]

[0133] Calculate the total energy of the sound signal:

[0134]

[0135] Calculate the noise energy:

[0136] E noise =E total -E harmonics

[0137] Calculate NHR:

[0138]

[0139] Calculate HNR:

[0140]

[0141] Calculate MDVP:Jitter(%), MDVP:Jitter(Abs), MDVP:RAP, MDVP:PPQ, Jitter:DDP

[0142] Extract the fundamental frequency trajectory:

[0143] P(t)=Pitch(s(t))

[0144] Where P(t) is the fundamental frequency trajectory of the sound signal s(t).

[0145] Calculate the jitter percentage (Jitter Percent):

[0146]

[0147] Where N is the number of fundamental frequency values.

[0148] Calculate the absolute jitter (Jitter Absolute):

[0149] Jitter Absolute=mean(|ΔP|)

[0150] where ΔP is the difference between adjacent fundamental frequency values.

[0151] Calculate RAP (Relative Average Perturbation):

[0152]

[0153] Calculate PPQ (Pitch Perturbation Quotient):

[0154]

[0155] Calculate MDVP:Shimmer(%), MDVP:Shimmer(dB), Shimmer:APQ3, Shimmer:APQ5, Shimmer:DDA

[0156] Extract the amplitude trace:

[0157] A(t)=Amplitude(s(t))

[0158] Calculate the Shimmer Percent:

[0159]

[0160] Calculate Shimmer in dB:

[0161] Assuming we use a small offset for slight amplitude adjustments to avoid taking the logarithm of 0, then:

[0162] Shimmer dB=20×mean(log 10 (|A|+offset) i+1 -log 10 (|A|+offset) i )

[0163] Calculate APQ3:

[0164]

[0165] Calculate APQ5:

[0166]

[0167] Calculate DDA:

[0168] DDA=3×mean(|A(i)-mean(A(i-1),A(i),A(i+1))|)

[0169] 2. Dataset training: Combine the 70-year-old and 80-year-old voice samples with other datasets and use the MLP model for training to improve the generalization ability of the model. Fig.11 These are MFCC images of the 70- and 80-year-old healthy control group and Parkinson's disease patient group synthesized using Pydub based on the MDVR-KCL dataset.

[0170] 2. System Overview:

[0171] The present invention provides an automatic learning Parkinson's prediction system with high generalization ability based on speech processing, which aims to assess the risk of Parkinson's disease through five different speech samples input by users. The system is presented in the form of WeChat applet "SoundHealth", and the backend stores 15 features of user voice. The system mainly includes the following components:

[0172] In the above Parkinson's disease identification scheme, in order to use feature selection technology to reduce irrelevant or redundant features, we can use a variety of specific feature selection algorithms. The following is a brief description of several commonly used feature selection algorithms and how they are applied:

[0173] Filter:

[0174] Mutual Information: Calculate the mutual information value between each feature and the target variable, sort them according to the mutual information value, and select the features with higher mutual information values. Mutual information directly measures the correlation between the feature and the target variable.

[0175] Chi-squared Test: Applicable to classification problems. It evaluates the correlation between each feature and the target variable by calculating the chi-square statistic between them and selects the feature with a higher statistic.

[0176] Correlation coefficient: For continuous features, you can use the correlation coefficient (such as the Pearson correlation coefficient) to measure the linear relationship between the feature and the target variable, and select features with stronger correlation.

[0177] Wrapper:

[0178] Recursive Feature Elimination (RFE): Combined with machine learning models (such as support vector machines, decision trees, etc.) for feature selection. Features are selected by recursively considering smaller and smaller feature sets. First, the model is trained on the original feature set and the importance of each feature is evaluated through indicators such as coefficients and feature importance; then, the least important features are removed and the model is retrained on the remaining feature set; this process is iterated until the required number of features is reached.

[0179] Embedded:

[0180] Model-based feature selection: Some machine learning algorithms (such as random forests, gradient boosted trees) automatically evaluate the importance of each feature during training. We can use these algorithms to train the model and perform feature selection based on the feature importance output by the model. For example, in random forests, the importance of each feature can be measured by calculating the average impurity reduction in the tree.

[0181] When applying these feature selection algorithms, the general steps are as follows:

[0182] Preprocess the data, including data cleaning, feature extraction, etc.

[0183] Select an appropriate feature selection algorithm and set the corresponding parameters.

[0184] Run the feature selection algorithm on the training data to get the feature importance ranking or the selected feature subset. The appropriate selection algorithm can be determined based on the final model output structure.

[0185] Based on the feature selection results, irrelevant or redundant features are eliminated, and features that contribute more to model performance are retained.

[0186] Retrain the model using the selected subset of features and evaluate the model's performance.

[0187] Voice collection and input: Users input voice through the WeChat applet "Sound Health". The system receives five different voice samples, covering the user's daily pronunciation, to more comprehensively and accurately assess the risk of Parkinson's disease.

[0188] Feature extraction: The system extracts 15 features from the user's voice, including but not limited to MDVP:Fo (average fundamental frequency), MDVP:Jitter (vibration rate), MDVP:Shimmer (sound instability), etc. These features have been proven to have strong universality on multiple data sets.

[0189] Questionnaire evaluation: Users fill out relevant questionnaires, and the system uses this information to further assess the risk of Parkinson's disease. By integrating questionnaire results and voice features, the accuracy of the system is improved.

[0190] Automatic learning and model update: The system automatically obtains user voice samples and related questionnaire results from the database every month to update the training model. A joint training method is adopted to integrate the Saarbruecken Voice Database, Oxford Parkinson's Disease Detection Dataset, and Mobile Device VoiceRecordings at King's College London datasets to improve the generalization ability of the model.

[0191] Specifically:

[0192] The core idea of ​​the algorithm is to assign different weights to different features according to their contribution to Parkinson's disease risk assessment. Through the attention mechanism, the model can automatically learn and adjust these weights, thereby achieving effective fusion of questionnaire results and speech features.

[0193] The specific implementation steps are as follows:

[0194] 1. Data preprocessing:

[0195] oEncode and normalize the questionnaire results and convert them into numerical data.

[0196] oPerform the same normalization process on the extracted speech features to ensure that the data are at the same scale.

[0197] 2. Feature Embedding:

[0198] Use embedding layers (such as fully connected layers or convolutional layers) to convert the questionnaire results and speech features into high-dimensional representation vectors respectively: Speech feature embedding: Use a fully connected layer to convert each speech feature into an embedding vector of fixed dimension. For example, each feature can be mapped to a 128-dimensional vector. Questionnaire embedding: For each question in the questionnaire, use another fully connected layer to convert it to the same embedding dimension (for example, 128 dimensions). Then, the embedding vectors of all questions can be concatenated or average-pooled into a single questionnaire embedding vector.

[0199] 3. Attention Mechanism:

[0200] The attention mechanism is introduced to dynamically adjust the weights of different features. By calculating the attention score, the model can assign a weight to each feature to reflect its importance in Parkinson's disease risk assessment. The calculation of the attention score can be based on the correlation, importance or other metrics between features. Specifically: In this layer, we calculate the attention score for each embedding vector. The correlation between the speech feature embedding and the questionnaire embedding can be calculated using dot product, cosine similarity or more complex functions.

[0201] For each embedding vector, we compute the attention score between it and the questionnaire embedding and normalize these scores through the softmax function to ensure that their sum is 1.

[0202] 4. Feature Fusion:

[0203] According to the attention score, the embedding vector of the questionnaire results and the embedding vector of the speech features are weightedly fused.

[0204] 5. Risk Assessment:

[0205] o Input the fused feature vector into a risk assessment model (such as a classifier or regression model).

[0206] o The model predicts or estimates the risk of Parkinson's disease based on the fusion features

[0207] Suppose we have 15 speech features and a questionnaire result containing multiple questions. We can design a network structure that includes an embedding layer, an attention layer, and a fusion layer.

[0208] Embedding layer:

[0209] Speech feature embedding: Use a fully connected layer to convert each speech feature into an embedding vector of fixed dimension. For example, each feature can be mapped to a 128-dimensional vector.

[0210] Questionnaire embedding: For each question in the questionnaire, use another fully connected layer to convert it to the same embedding dimension (e.g. 128). Then, the embedding vectors of all questions can be concatenated or average-pooled into a single questionnaire embedding vector.

[0211] Attention Layer:

[0212] At this layer, we calculate the attention score for each embedding vector. The correlation between the speech feature embedding and the questionnaire embedding can be calculated using dot product, cosine similarity, or more complex functions.

[0213] For each embedding vector, we compute the attention score between it and the questionnaire embedding and normalize these scores through a softmax function to ensure that their sum is 1.

[0214] Fusion layer:

[0215] Using the weights output by the attention layer, the speech feature embeddings are weighted summed to obtain a weighted speech feature embedding vector.

[0216] The weighted speech feature embedding vector is concatenated or added to the questionnaire embedding vector to obtain the final fused feature vector.

[0217] In the above scheme, for the task of fusing questionnaire results and speech features, the cross-entropy loss function can be used. The cross-entropy loss function is commonly used in classification problems and can measure the difference between the predicted probability distribution and the true probability distribution. Specifically, the cross-entropy loss is calculated for the questionnaire results and the prediction of speech features respectively, and then weighted summed according to their importance:

[0218] [L=\alpha\cdot CE_{survey}+\beta\cdot CE_{speech}]

[0219] Among them, (CE_{survey}) is the cross entropy loss of the survey results, (CE_{speech}) is the cross entropy loss of the speech features, and (\alpha) and (\beta) are weights assigned according to the importance of the features.

[0220] In summary, the MLP neural network is suitable for detecting Parkinson's disease.

[0221] 3. System implementation details:

[0222] Database construction: The system uses the MDVR-KCL dataset, Saarbruecken Voice Database, and Little dataset for joint training. The MDVR-KCL dataset contains 16 Parkinson's disease patient samples and 21 healthy control samples, while the Saarbruecken Voice Database contains 1,532 healthy or other disease samples and 13 Parkinson's disease patient samples. The Little dataset consists of 23 Parkinson's disease patients and 8 healthy controls, with 195 recorded samples.

[0223] Data preprocessing: To address the data imbalance problem, Synthetic Minority Over-sampling Technique (SMOTE) was used for data enhancement. In addition, in order to simulate different age characteristics, the samples were synthesized by adjusting the playback speed of the original audio.

[0224] Model selection and training: Among many machine learning algorithms, a neural network model based on multi-layer perceptron (MLP) was selected to handle the speech prediction problem. The MLP model showed good generalization ability for limited labeled data in multiple experiments. Stochastic gradient descent (SGD) was used to optimize the model, and the ReLU activation function was applied to improve the learning effect.

[0225] Since there are few samples of Parkinson's disease patients in the dataset, SMOTE (Synthetic Minority Over-sampling Technique) can be used for oversampling to generate synthetic samples to balance the dataset.

[0226] Another approach is to employ resampling techniques such as undersampling the majority class or using class weights to give more attention to the minority class in the loss function.

[0227] Extract meaningful features from raw audio data, such as Mel-frequency cepstral coefficients (MFCCs), spectrogram features, etc.

[0228] Consider using feature selection techniques to reduce irrelevant or redundant features to improve the generalization ability of the model.

[0229] Performance evaluation: Through cross-dataset testing, the performance of the model in different environments and voice collection conditions is demonstrated. ROC curves, confusion matrices, accuracy, recall, precision and other indicators are used to comprehensively evaluate the performance of the system.

[0230] 4. Implementation process:

[0231] After listening to the sample demonstration in the WeChat mini-program "Sound Health", the user entered the recording interface, recorded 5 audios according to the prompts, and filled out the relevant questionnaire.

[0232] The system receives and stores 15 features extracted from the user's voice sample.

[0233] The questionnaire information filled out by users is used in combination with voice features to conduct a comprehensive assessment of Parkinson's disease risk.

[0234] The system automatically updates the model every month and uses newly acquired data for training to continuously improve the prediction accuracy for Asians.

[0235] Users can obtain risk assessment results and relevant medical advice at any time through the WeChat mini program.

[0236] This implementation not only considers the feature extraction of user voice input, but also combines multi-source data for joint training to improve the generalization ability of the model, making it more suitable for users of different ages, contexts and regions.

[0237] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for constructing a Parkinson's disease detection model based on speech features, characterized in that: The steps include: Step 1: extracting user voice features and obtaining questionnaire results, encoding the questionnaire results, and normalizing the encoded questionnaire results and the extracted voice features to obtain normalized questionnaire results and voice features; Step 2: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors through the embedding layer. The speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question in the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer and concatenated into a single questionnaire embedding vector. Step 3: Get the attention score between each embedding vector and the questionnaire embedding, and normalize the score through the softmax function; Step 4: According to the attention score, the embedding vector of the questionnaire result and the embedding vector of the speech feature are weighted fused to obtain a fused feature vector; Step 5: Input the obtained fused feature vector into the risk assessment model to obtain the risk prediction of Parkinson's disease.

2. The method for constructing a Parkinson's disease detection model based on speech features according to claim 1, characterized in that: The normalized questionnaire results and speech features are converted into high-dimensional representation vectors respectively through the embedding layer, wherein the speech features are mapped into an embedding vector of fixed dimension through the fully connected layer, and each question of the questionnaire is converted into an embedding vector of the same dimension through the fully connected layer, and spliced ​​into a single questionnaire embedding vector, including: Each speech feature is mapped to an embedding vector of fixed dimension through a fully connected layer. For each question in the questionnaire, it is converted to the same embedding dimension as the speech feature after mapping through another fully connected layer, and the embedding vectors of all questions in the questionnaire are concatenated into a single questionnaire embedding vector.

3. The method for constructing a Parkinson's disease detection model based on speech features according to claim 2, characterized in that: The method of obtaining the attention score between each embedding vector and the questionnaire embedding and normalizing the score through the softmax function includes: Calculate the attention score of each embedding vector, and calculate the correlation between the speech feature embedding vector and the questionnaire embedding vector through the dot product or cosine similarity function; for each speech feature embedding vector, obtain the attention score between it and the questionnaire embedding vector, and normalize the attention score through the softmax function so that the sum of the normalized attention scores is 1.

4. The method for constructing a Parkinson's disease detection model based on speech features according to claim 3, characterized in that: The method of weightedly fusing the embedding vector of the questionnaire results and the embedding vector of the speech features according to the attention score to obtain a fused feature vector includes: The speech feature embedding vectors are weighted and summed using the weights output by the attention layer to obtain a weighted speech feature embedding vector; the weighted speech feature embedding vector is concatenated with the questionnaire embedding vector to obtain a fused feature vector.

5. The method for constructing a Parkinson's disease detection model based on speech features according to claim 4, characterized in that: It also includes using the cross entropy loss function to measure the difference between the predicted probability distribution and the true probability distribution, by calculating the predicted cross entropy loss of the questionnaire results and speech features, and performing weighted summation based on the weights assigned according to the feature importance, where the calculation formula for the cross entropy loss is: L=α·CEsurvey+β·CEspeech Among them, CEsurvey is the cross entropy loss of the survey results, CEspeech is the cross entropy loss of the speech features, and α and β are weights assigned according to the importance of the features.

6. A Parkinson's disease detection model construction platform based on speech features, characterized in that: A method for constructing a Parkinson's disease detection model based on speech features according to any one of claims 1 to 5 is applied, comprising a database module, a data acquisition module, a data preprocessing module, a model selection module, an output module and a performance evaluation module; The data acquisition module is connected to the database module, the database module is connected to the data preprocessing module, the data preprocessing module is connected to the model selection module, and the output module and the performance evaluation module are respectively connected to the model selection module.

Images