A method for constructing a full-automatic obstructive sleep apnea syndrome identification model

Abstract

The application discloses a kind of full-automatic obstructive sleep apnea syndrome identification model construction method;The present application extracts features from three different signals of human body shape characteristics (including basic information, such as age, gender, etc.), voice, three-dimensional face, and fuses the three types of features into a deep learning network for OSA patient identification and OSA severity discrimination.The present application uses a feature selection algorithm to select the optimal features from the extracted features, and fuses the selected features, uses a data balancing algorithm to balance the sample types, and uses a deep learning network to extract deeper features to obtain more accurate results.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of diagnostic screening of obstructive sleep apnea, in particular, relates to a method for constructing a full-automatic obstructive sleep apnea identification model. BACKGROUND

[0002] Obstructive Sleep Apnea Hypoapnea Syndrome (OSA) is a prevalent sleep-related breathing disorder characterized by recurrent episodes of upper airway collapse during sleep, manifested as recurrent apneas and hypopneas during sleep, severe snoring at night, and daytime sleepiness. OSA is a risk factor for chronic diseases (such as cardiovascular disease and hypertension) and acute diseases (such as stroke, myocardial infarction, pulmonary heart disease, congestive heart failure, and even sudden death). If not treated in time, it can seriously threaten personal health, lead to low quality of life, impaired work performance, and even cause traffic accidents.

[0003] The diagnosis of OSA is based on the calculation of Apneahypopnea index (AHI), which measures the frequency of airflow reduction associated with upper airway collapse or narrowing. When AHI ≥ 5 events / hour, OSA is diagnosed. The gold standard for determining AHI is polysomnography (PSG), but this is a relatively expensive and time-consuming method, and the acquisition and processing of neuroelectric signals are extremely precise and complex.

[0004] In clinical practice, scales are commonly used for preliminary OSA screening, including the Berlin Questionnaire, the Epworth Sleepiness Scale, and the STOP Bang Questionnaire. These questionnaires predict the risk level of OSA by asking about the subject's medical history, height, age, weight, body mass index (BMI), three circumferences (neck circumference, waist circumference, and chest circumference), snoring, fatigue, blood pressure, etc. Although this is a low-cost and efficient method, it still has the defect of excessive subjectivity.

[0005] Since the obstruction of the upper airway in OSA patients is in the pharynx, the pharynx is one of the vocal resonators closest to the vibrator, making OSA patients likely to have abnormal voice quality. Literature has shown that the upper airway of OSA patients has a significant tendency to close, and the upper airway partially overlaps with the vocal tract, so this can also explain that OSA patients may have speech abnormalities compared to normal people.

[0006] Another important risk factor in the pathogenesis of OSA is craniofacial features. Many imaging studies have confirmed that the craniofacial skull and soft tissue structure of OSA patients is significantly different from that of normal people. In the skull structure, the length of the maxilla is shortened, the length of the mandibular body is shortened, the maxilla is moved backward, the maxilla is narrowed, the mandibular plane is steep, the hyoid bone is moved downward, the anterior cranial base is narrow, the height of the forehead is longer, and the position of the head is extended. In the soft tissue structure, the tongue is enlarged, the uvula is elongated, the soft palate is enlarged, the adenoid tonsil is hypertrophic, and the airway space is narrow. Advanced imaging techniques for evaluating skull and soft tissue structure have been developed in clinical practice: X-ray cephalometry, computed tomography (CT) and magnetic resonance imaging (MRI). Although these imaging techniques can provide high-resolution morphological display, they have the disadvantages of complicated and expensive imaging methods.

[0007] Therefore, a simpler and more economical non-invasive early diagnosis method is being developed in clinical practice to evaluate the PSG needs priority of patients, so as to speed up the diagnosis of severe cases. Meanwhile, it is worth exploring how to use a simpler and more economical three-dimensional face image to detect OSA patients by imaging. SUMMARY

[0008] In view of the deficiencies of the prior art, the purpose of the present application is to provide a method for constructing a full-automatic obstructive sleep apnea syndrome recognition model. The present application extracts features from three different signals: human morphological features and basic information, speech, and three-dimensional face, and fuses the three types of features into a deep learning network for OSA patient recognition and OSA severity discrimination.

[0009] Human morphological features and basic information include height, age, weight, body mass index (BMI), and three circumferences (neck circumference, waist circumference, and chest circumference); speech signal design includes speech paradigms that may exhibit acoustic abnormalities in OSA patients, covering vowels, words, and sentences; three-dimensional face phenotype includes nose-lip distance, nose-philtrum distance, lip-philtrum distance, eye corner-philtrum distance, eye corner-eye corner distance, and eye corner-gnathion distance;

[0010] The technical solution of the present application is specifically introduced as follows.

[0011] A method for constructing a full-automatic obstructive sleep apnea syndrome recognition model, the full-automatic obstructive sleep apnea syndrome recognition model is used for recognition and severity discrimination of obstructive sleep apnea syndrome; the specific steps are as follows:

[0012] 1) Collecting appearance morphological phenotype data, speech signals and three-dimensional face phenotype data of the subjects;

[0013] In the human body shape feature and basic information data collection, the height information, the girth information and the width information of each subject are collected.

[0014] In the human body daytime speech signal collection, the prescribed speech signals, including vowels, words and sentences, are collected.

[0015] In the human body three-dimensional face phenotype data collection, the head of each subject is scanned from multiple angles and multiple directions.

[0016] 2) The AHI index obtained after the subject undergoes overnight PSG is acquired, and the subject is classified according to the severity:

[0017] 3) The human body shape feature and basic information data, speech signals and three-dimensional face data are preprocessed, feature extracted, feature selected and feature fused, and the fused features are taken as the final features.

[0018] Firstly, the human body shape feature and basic information data, speech signals and three-dimensional face data are preprocessed, including data cleaning (such as missing value deletion, repeated data removal, character type data conversion to digital type data, etc.) on the human body shape feature and basic information and three-dimensional face data, and the method for preprocessing the speech signals includes silence elimination, framing, windowing and pre-emphasis.

[0019] Then, the preprocessed data are feature extracted, including the following three aspects:

[0020] ① The extracted human body shape feature and basic information features include height, width and girth, etc.

[0021] ② The extracted three-dimensional face phenotype features include nose-lip distance, nose-philtrum distance, lip-philtrum distance, eye corner-philtrum distance, eye corner-eye corner distance and eye corner-submental distance, etc.

[0022] ③ The extracted speech features include time domain features, frequency domain features and prosody features, etc.

[0023] Then, the univariate selection method is used to select the first 20 features with the highest contribution value to the AHI value and capable of representing most information among the human body shape feature and basic information features, speech features and three-dimensional face features.

[0024] Next, the 60 optimal features selected by the feature selection are spliced to become a one-dimensional vector containing 60 optimal features, so as to realize feature fusion.

[0025] 4) The data are balanced by the data balancing algorithm.

[0026] 5) The deep learning network model is constructed and trained.

[0027] The deep learning network model comprises, from input to output, a full connection layer, a convolution layer, a nonlinear activation layer, a pooling layer and a Softmax layer in sequence, and the deep learning network model is trained to obtain a full-automatic obstructive sleep apnea syndrome identification model for realizing high-precision OSA patient identification and OSA severity discrimination.

[0028] In the application, in step 2), the subjects are classified according to severity by using binary classification and four-class classification.

[0029] The binary classification includes: no OSA or OSA.

[0030] The four-class classification includes: no OSA, mild OSA, moderate OSA and severe OSA.

[0031] When there is no OSA, AHI is less than 5 times per hour, and when there is OSA, AHI is greater than or equal to 5 times per hour.

[0032] When there is mild OSA, 5 times per hour is less than AHI, which is less than 15 times per hour; when there is moderate OSA, 15 times per hour is less than AHI, which is less than 30 times per hour; and when there is severe OSA, AHI is greater than or equal to 30 times per hour.

[0033] In step 3) of the application, the height features in the extracted human body shape features and basic information features include: height, head height, neck height, waist height, hip height, knee height, abdominal circumference height, chest height and waistband back height.

[0034] The width features include: shoulder width, chest point width, left arm length and right arm length.

[0035] The girth features include: head girth, chest girth, waist girth, hip girth, neck girth and thigh girth.

[0036] In step 3) of the application, the extracted speech features include:

[0037] The time domain features include short-time energy, root mean square energy, short-time zero-crossing rate and short-time average amplitude.

[0038] The frequency domain features include spectral centroid, spectral flux, differential mean of each order mel cepstral coefficient, standard deviation of each order mel cepstral coefficient, and mean of each order differential of each order mel cepstral coefficient.

[0039] The prosody features include fundamental frequency, mean and median of the fundamental frequency, maximum and minimum of the fundamental frequency, mean of the fundamental frequency perturbation, minimum and standard deviation of the first, second and third formants.

[0040] In step 4) of the application, the data balancing algorithm uses the ADASYN algorithm.

[0041] In the present application, in step 5), in the deep neural network, the full connection layer is 5 layers, the convolution layer is 3 layers, the nonlinear activation layer is 3 layers, the pooling layer is 3 layers, and the Softmax layer is 1 layer.

[0042] In the present application, the indexes for evaluating the performance of the model include accuracy, precision, recall and F1 score.

[0043] The present application is aimed at the defects of high incidence and low diagnosis rate of OSA, high disease risk factors, and high cost of money and time of PSG, the gold standard for diagnosing OSA. The present application fuses three different types of signals of human body shape features and basic information, speech and simpler and more economical three-dimensional facial phenotype features, and sends the three types of fused features to the advanced deep learning network for further feature extraction and training, to realize a robust method for identifying OSA patients and OSA severity. This diagnostic method can be used to evaluate the priority of PSG demand of patients, so as to speed up the diagnosis of severe cases. Compared with the prior art, the present application has the following beneficial effects:

[0044] 1. More rich and diverse signals are used, including human body shape features and basic information, speech signals and three-dimensional facial phenotype features.

[0045] 2. Rich features are extracted for the three different types of signals: height (such as height, head height, neck height, waist height, hip height, etc.), girth (such as head girth, chest girth, waist girth, etc.), width (such as shoulder width, chest point width, etc.), time domain features (such as short-time energy, root mean square energy, short-time zero-crossing rate, etc.), frequency domain features (such as spectral centroid, spectral flux, differential mean of each order mel cepstrum coefficient, etc.), nose-lip distance (right ala nasi inferior point to upper lip point distance, right ala nasi point to left corner point distance, etc.), nose-philtrum distance (left ala nasi point to right philtrum point distance, right ala nasi inferior point to left philtrum point distance, etc.) are extracted for human body shape features and basic information.

[0046] 3. Feature selection algorithm is used to select the optimal features of the extracted features, and the selected features are fused, data balancing algorithm is used to balance the sample types, and deep learning network is used to mine deeper features and get more accurate results. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is a full-automatic OSA recognition and severity discrimination method process schematic diagram based on fusion of human body shape features and basic information, daytime speech and three-dimensional facial phenotype.

[0048] Figure 2It is a schematic diagram of a full-automatic OSA severity grading system based on fusion of human body morphological features and basic information, daytime speech and human three-dimensional facial phenotype.

[0049] Figure 3 It is a model training flowchart of a full-automatic OSA severity grading method based on fusion of human body morphological features and basic information, daytime speech and human three-dimensional facial phenotype of the embodiment. DETAILED DESCRIPTION

[0050] The technical solutions of the present application will be described in detail below in combination with the drawings and embodiments.

[0051] The present application proposes to model three different types of signals of human body morphological features and basic information, speech, and three-dimensional facial phenotype features, and fuse the three types of features into a deep learning network to automatically extract high-precision features, and then realize a robust OSA patient recognition and OSA severity discrimination based on the model.

[0052] The method comprises:

[0053] (1) collecting human body morphological feature and basic information data of a subject through a three-dimensional human body scanning acquisition system;

[0054] (2) collecting speech signals of the subject, including vowels and designed words and sentences;

[0055] (3) collecting three-dimensional facial data of the subject through a human facial phenotype three-dimensional acquisition system;

[0056] (4) obtaining an AHI index of the subject after a whole-night PSG, and classifying the severity of the subject according to the following standards:

[0057] Task 1: binary classification, i.e. no OSA (AHI < 5 times / hour) or OSA (AHI ≥ 5 times / hour);

[0058] Task 2: four classification, i.e. no OSA (AHI < 5 times / hour) or mild OSA (5 times / hour ≤ AHI < 15 times / hour) or moderate OSA (15 times / hour ≤ AHI < 30 times / hour) or severe OSA (AHI ≥ 30 times / hour);

[0059] (5) first, the human body morphological feature and basic information data, speech signal and three-dimensional facial phenotype data are preprocessed and feature extracted, secondly, the top 20 features with the highest contribution to AHI value are extracted from the human body morphological feature and basic information feature, speech feature, and human three-dimensional facial data feature using a feature selection method, and finally, they are spliced into a one-dimensional vector containing 60 features and used as a fusion feature;

[0060] (6) To solve the problem of class imbalance, the ADASYN (Adaptive Synthetic, ADASYN) algorithm is used to construct an equalized data set;

[0061] (7) The deep learning network adopted includes a full connection layer, a convolution layer, a nonlinear activation layer, a pooling layer and a Softmax layer.

[0062] (8) The 10-fold cross-validation method is used: the fusion feature data set is divided into 10 equal subsets, and each subset is used as a validation set while the remaining subsets are used as training sets for model training and evaluation.

[0063] The system comprises a data acquisition module, a preprocessing, feature extraction, feature selection and feature fusion module, a deep learning network module, a binary classification module and a four-class classification module.

[0064] The automatic obstructive sleep apnea syndrome recognition and severity determination method based on the fusion of human body morphological features, basic information, daytime speech and three-dimensional facial phenotype features comprises the following detailed steps: Figure 3

[0065] I. Collecting appearance morphological phenotype data, speech signals and three-dimensional facial phenotype data of the subjects;

[0066] In the human body morphological feature and basic information data collection, the height information (such as height, head height, neck height, waist height, hip height, etc.), girth information (such as head girth, chest girth, waist girth, etc.) and width information (such as shoulder width, chest point width, etc.) of each subject are collected, and the standing posture and scene of the subject are uniformly controlled during the collection to obtain accurate collection results.

[0067] In the human body daytime speech signal collection, a microphone is fixed at a position about 20 cm away from the subject's oral cavity, and digital audio signals are recorded at a sampling rate of 44.1 kHz, the microphone is at an angle of 15° with the horizontal line, and the specified speech signals, including vowels, words and sentences, are collected.

[0068] In the human body three-dimensional facial phenotype data collection, a human head and face phenotype three-dimensional collection system is used to scan and image each subject, and the front face, left side face (the device is at an angle of 45° with the subject) and right side face (the device is at an angle of 45° with the subject) of the subject are collected, the sitting posture, scene, posture and expression of the subject are uniformly controlled during the scanning and imaging, the eyes of the subject are at the same horizontal line with the camera, and at the same time, the subject also needs to wear a special headgear to prevent hair from covering the face and affecting the collection of facial phenotype features.​

[0069] II. Obtain the AHI index derived after the subject undergoes an overnight PSG, and classify the subject according to the following criteria:

[0070] Task 1: Binary classification, i.e. no OSA (AHI < 5 events / hour) or OSA (AHI ≥ 5 events / hour);

[0071] Task 2: Four-class classification, i.e. no OSA (AHI < 5 events / hour) or mild OSA (5 events / hour ≤ AHI < 15 events / hour) or moderate OSA (15 events / hour ≤ AHI < 30 events / hour) or severe OSA (AHI ≥ 30 events / hour);

[0072] III. Preprocess, extract features, select features and fuse features of the human morphological features and basic information data, speech signal and three-dimensional face data, and take the fused features as the final features;

[0073] The preprocessing includes data cleaning (such as missing value deletion, repeated data removal, character type data conversion to digital type data, etc.) on the human morphological features and basic information, three-dimensional face data, and mute elimination, framing, windowing and pre-emphasis on the speech signal;

[0074] The feature extraction includes the following 3 aspects:

[0075] ① Batch extract human morphological features and basic information features, mainly extract the following features:

[0076] Height: height, head height, neck height, waist height, hip height, knee height, abdominal circumference height, chest height, waistband back height;

[0077] Width: shoulder width, chest point width, left arm length, right arm length;

[0078] Girth: head girth, chest girth, waist girth, hip girth, neck girth, thigh girth, ankle girth;

[0079] ② Batch extract three-dimensional face phenotype features, mainly extract the following features:

[0080] Nose-lip distance: right ala nasi inferior point to upper lip point distance, right ala nasi point to left corner of mouth point distance, right corner of mouth point to left ala nasi point distance, left ala nasi inferior point to upper lip point distance, left corner of mouth point to upper lip point distance;

[0081] Nose-philtrum distance: left ala nasi point to right philtrum point distance, right ala nasi inferior point to left philtrum point distance, left ala nasi inferior point to left philtrum point distance;

[0082] Cheek-nose distance: distance from left corner of mouth to right nose midpoint, distance from left corner of mouth to left nose midpoint, distance from right corner of mouth to left nose midpoint, distance from right corner of mouth to right nose midpoint;

[0083] Eye corner-nose distance: distance from left inner eye corner to right nose midpoint, distance from right inner eye corner to right nose midpoint;

[0084] Eye corner-eye corner distance: distance from right outer eye corner to right inner eye corner, distance from left outer eye corner to left inner eye corner;

[0085] Eye corner-chin distance: distance from left inner eye corner to submental point, distance from right inner eye corner to submental point;

[0086] ③Batch extraction of speech features, mainly extracting the following features:

[0087] Time domain features: short-time energy, root mean square energy, short-time zero-crossing rate, short-time average amplitude;

[0088] Frequency domain features: spectral centroid, spectral flux, differential mean of each order mel cepstral coefficient, standard deviation of each order mel cepstral coefficient, mean of each order differential of each order mel cepstral coefficient;

[0089] Rhythm features: fundamental frequency, mean and median of fundamental frequency, maximum and minimum of fundamental frequency, mean of fundamental frequency perturbation, minimum and standard deviation of first, second and third formants;

[0090] The feature selection uses univariate selection method to select the top 20 features with the highest contribution value to AHI value and representing most information from human body shape features, basic information features, speech features and three-dimensional face features;

[0091] The feature fusion splices the 60 optimal features obtained by feature selection to form a one-dimensional vector containing 60 optimal features.

[0092] Four, to improve the problem of data imbalance, the present example adopts ADASYN algorithm, its main principle is as follows:

[0093] Unlike interpolation algorithm which interpolates between minority samples, it automatically determines how many synthetic class samples each minority class sample needs to generate through some mechanism. The specific process is as follows:

[0094] ①Calculate the total number of samples that need to be synthesized:

[0095]

[0096] G=(S max -S min )×β

[0097] Where S maxFor the majority class sample number, S min For the minority class sample number, β is in the range of 0-1, G is the number of the minority class samples to be synthesized, if β is 1, it represents that the number of each class is equal after synthesis.

[0098] 2. For each minority class sample x i , find its K nearest neighbors according to the Euclidean distance, and calculate:

[0099]

[0100]

[0101] Where, Δ i is the number of majority class samples in the K nearest neighbors, r i is the newly generated minority class sample, r i ′ is the normalization factor.

[0102] 3. For each minority class sample x i , calculate the number of synthesized samples g i :

[0103] ′

[0104] g i = r i × G

[0105] 4. Select one minority class sample from the K neighbors around each minority class sample to be synthesized, and synthesize according to the following equation:

[0106] s i = x i + (x zi - x i ) × γ

[0107] Where, x zi - x i is the difference vector in N-dimensional space, and γ is a random number in the range of 0-1.

[0108] Five, construct a deep learning network model

[0109] The deep learning network model includes 5 fully connected layers, 3 convolutional layers, 3 nonlinear activation layers, 3 pooling layers, and 1 Softmax layer. The average of the accuracy, precision, recall, and F1 score of the test set for 5 times is taken as the model evaluation index.

[0110] The evaluation index of the model performance includes accuracy, precision, recall, and F1 score.

[0111] In this embodiment, 10-fold cross-validation method is used to divide the data set into 10 sub-samples with equal data volume, and the 10 sub-samples are traversed in turn, each time taking the current sub-sample as the test set and the remaining 9 sub-samples as the training set, so that 10 model evaluation performance indicators can be obtained, and the average value of the 10 results is taken as the final evaluation indicator of the model performance.

[0112] The application compares the results of using human morphological features and basic information features, speech features, three-dimensional face phenotype features and fusion features to perform OSA two-classification and four-classification under the condition that other factors (including deep learning network and ADASYN algorithm) do not change.

[0113] Table 1 two-classification verification result

[0114]

[0115] Table 2 four-classification verification result

[0116]

[0117] The application is different from the OSA recognition method based on a single signal, and the application first proposes to extract and fuse human morphological features (including human basic information), daytime speech and three-dimensional face features, and uses an advanced deep learning network to distinguish OSA patients and OSA severity, so that the method can help doctors quickly and accurately recognize OSA patients and distinguish the severity of different OSA patients, so as to preferentially treat severe patients.

[0118] The application uses rich feature extraction technology, adds feature selection and ADASYN data balancing algorithm, compares the results of single signal features and fusion signal features for diagnosing and classifying OSA, and the results show that the fusion signal features have better effect than the single signal features.

Claims

1. A method for constructing a fully automatic obstructive sleep apnea recognition model, characterized in that: The fully automatic obstructive sleep apnea recognition model is used to identify obstructive sleep apnea and determine its severity. The specific steps are as follows: 1) Collect the subject's appearance phenotype data, voice signal and three-dimensional facial phenotype data; In the data collection of human morphological characteristics and basic information, the age, weight, height, chest and abdominal circumference information of each subject were collected; In the daytime human voice signal collection, specified voice signals are collected, including vowels, words and sentences; In the collection of human 3D facial phenotype data, each subject’s head is scanned from multiple angles and directions; 2) Obtain the AHI index obtained from the subjects after overnight PSG and grade the severity of the subjects; 3) Preprocessing, feature extraction, feature selection, and feature fusion are performed on the human body morphological features and basic information data, voice signals, and three-dimensional face data in sequence, and the fused features are used as the final features; First, the human body morphological features and basic information data and three-dimensional face data are pre-processed by data cleaning. This includes deletion of missing values, removal of duplicate data, and conversion of character data into digital data; pre-processing of speech signals by performing silence removal, framing, windowing, and pre-emphasis; Then, feature extraction is performed on the preprocessed data, including the following three aspects: The extracted human body morphological features and basic information features include height, width and girth features; The extracted three-dimensional facial phenotypic features include nose-lip distance, nose-philtrum distance, lip-philtrum distance, eye corner-philtrum distance distance, canthus-canthus distance, canthus-chin distance, etc. The extracted speech features include time domain features, frequency domain features and prosodic features; Then, the univariate selection method was used to select the top 20 features that have the highest contribution to the AHI value and can represent most of the information among human morphological features, basic information features, voice features, and three-dimensional facial features; Then, the 60 best features obtained by feature selection are concatenated to form a one-dimensional vector containing the 60 best features, thus realizing feature fusion. 4) Data balancing algorithm balances the data; 5) Build and train deep learning network models; The deep learning network model includes a fully connected layer, a convolutional layer, a nonlinear activation layer, a pooling layer and a softmax layer from input to output. The deep learning network model is trained to obtain a fully automatic obstructive sleep apnea sign recognition model that can achieve high-precision OSA patient identification and OSA severity judgment.

2. The construction method according to claim 1, characterized in that In step 2), the severity of the subjects was graded using two or four categories; The two categories included: no OSA or OSA; The four categories include: no OSA, mild OSA, moderate OSA, and severe OSA; in: When there is no OSA, AHI < 5 times / hour; when there is OSA, AHI ≥ 5 times / hour; In mild OSA, 5 times / hour ≤ AHI < 15 times / hour; in moderate OSA, 15 times / hour ≤ AHI < 30 times / hour; in severe OSA, AHI ≥ 30 times / hour.

3. The construction method according to claim 1, characterized in that In step 3), the human body morphological features and basic information features are extracted. Height characteristics include: height, head height, neck height, waist height, hip height, knee height, abdominal height, chest height, belt back height; Width features include: shoulder width, chest point width, left arm length, right arm length; The circumference features include: head circumference, chest circumference, waist circumference, hip circumference, neck circumference, thigh circumference, and ankle circumference.

4. The construction method according to claim 1, wherein In step 3), among the extracted speech features, Time domain characteristics include short-time energy, root mean square energy, short-time zero-crossing rate, and short-time average amplitude; Frequency domain features include spectrum centroid, spectrum flux, mean of each order Mel-frequency cepstral coefficient difference, standard deviation of each order Mel-frequency cepstral coefficient, and mean of each order difference of each order Mel-frequency cepstral coefficient; The rhythmic features include fundamental frequency, mean and median of fundamental frequency, maximum value of fundamental frequency, mean value of fundamental frequency disturbance, minimum value and standard deviation of the first formant, second formant and third formant.

5. The construction method according to claim 1, characterized in that In step 4), the data balancing algorithm adopts the ADASYN algorithm.

6. The construction method according to claim 1, characterized in that In step 5), in the deep neural network, there are 5 fully connected layers, 3 convolutional layers, 3 nonlinear activation layers, 3 pooling layers, and 1 softmax layer.

7. The construction method according to claim 1, characterized in that When training deep neural networks, indicators for evaluating model performance include accuracy, precision, recall, and F1 score.

Images