A chronic obstructive pulmonary disease identification system based on digitized lung sound, a construction method and application thereof

Abstract

The application belongs to the field of digital medicine, and particularly relates to a chronic obstructive pulmonary disease (COPD) differential system based on digital lung sound, a construction method and application thereof. The system comprises a data acquisition module, a preprocessing module, a conversion module, a data module, a data processing module, a data enhancement training module, a deep learning module, a differential module, a grading module and a display module. Digital lung sound records of candidates are collected, converted into mel spectrograms, and diagnosed by using a deep neural network. The candidates are COPD patients, healthy people, or people suffering from other lung diseases, and the disease grade of the COPD patients is given by a device carrying the differential system. The examination only needs to collect lung sounds in clinical or other places, reduces radiation damage and possible infection risk in the conventional diagnosis process, and provides a scheme for early diagnosis of COPD, and improves the diagnosis efficiency of the respiratory department, especially the COPD disease.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of digital medicine, and particularly relates to a chronic obstructive pulmonary disease differential system based on digitized lung sound, a construction method and application thereof. BACKGROUND

[0002] Chronic obstructive pulmonary disease (COPD) is a common disease that seriously endangers human health, and is a frequently-occurring disease that seriously affects the quality of life of patients and has a high mortality rate.

[0003] Progressive irreversible airflow limitation is an important feature of COPD, and patients often show symptoms such as chest tightness, shortness of breath, chronic cough and repeated lung infections, which seriously affect the quality of life and health of patients. Timely treatment can control the disease progression of patients and improve the quality of life of patients. Pulmonary function test is the gold standard for clinically diagnosing COPD. In clinical practice, for patients with persistent, progressive and laborious dyspnea, wheezing sound on auscultation, repeated acute bronchitis and acute exacerbation of chronic bronchitis, and exposure to risk factors such as smoking, environmental pollution and high PM2.5, pulmonary function test is used to diagnose whether the patient has COPD. The ratio of forced expiratory volume in the first second after maximum inspiration to forced vital capacity FEV1 / FVC reflects the degree of airflow limitation, which is a sensitive indicator of COPD. FEV1 / FVC<70% after the patient inhales a bronchodilator is generally considered to have irreversible airflow limitation. However, due to the difference in reversibility test at different occasions and the lack of repeatability, the application of the FEV1 / FVC<70% indicator may lead to some healthy elderly people being diagnosed with COPD, and may also lead to insufficient diagnosis of COPD in adults less than 45 years old. In clinical practice, asthma is similar to COPD in terms of performance, and some asthma patients may also cause irreversible airflow limitation due to the extension of the disease course. The differential diagnosis of asthma and COPD is a difficult point in clinical practice. The NICE 2010 guidelines point out that, in addition to the lung function test, all patients should also undergo chest X-ray and complete blood count in order to accurately diagnose COPD and distinguish COPD from other diseases. Therefore, in clinical practice, the results of a single pulmonary function test cannot be used as the basis for the differential diagnosis and diagnosis of COPD, and other test results of the patient, as well as the patient's medical history and long-term follow-up, need to be comprehensively evaluated. The experience level of the doctor is particularly important, and insufficient experience of the doctor may lead to misdiagnosis and missed diagnosis.

[0004] Auscultation is one of the common examination methods in respiratory medicine due to its non-invasive and portable characteristics. Doctors make a preliminary judgment of the pathological condition of the patient's lung according to the abnormal lung sounds such as crackles and wheezing sounds, and assist in the diagnosis of the disease. Patients with COPD will have wheezing (Wheeze) in the whole chest due to obstructive lesions in the lung, and will also have the phenomenon of decreased respiratory sound. However, the results of auscultation have certain subjective randomness, and the decrease of respiratory sound and wheezing may also be caused by other diseases, so auscultation can only be used as an auxiliary means for disease diagnosis, and cannot be used as the basis for the diagnosis of COPD. With the advancement of technology, computer-recorded lung sounds are used for analysis to reveal the relationship between lung sounds and lung diseases. Studies have shown that digital lung sounds can be used as a marker for the diagnosis of COPD.

[0005] In recent years, deep learning has been widely used in medicine, and deep neural network models can extract features related to specific diseases from medical images or signals for disease diagnosis. There have been many studies using lung sounds to diagnose respiratory diseases. However, most existing studies focus on abnormal lung sound recognition, i.e. classifying lung sounds into crackles (Crackle and Wheeze, etc.), and classifying acute and chronic lung diseases through lung sounds. There are few studies on the use of digital lung sounds for the diagnosis of specific lung diseases (such as COPD). Existing COPD diagnosis studies lack data diversity and richness, with single data source and small sample size collected from one hospital. The model trained on such single-center dataset has high accuracy, but weak generalization ability, and the accuracy is greatly reduced when encountering other dataset samples, which cannot be applied to clinical practice. In addition, the COPD / healthy classification model or COPD / non-COPD classification model proposed in these studies is not convenient for actual clinical application. SUMMARY

[0006] The present application provides a method and system for identifying chronic obstructive pulmonary disease. A method for identifying COPD and COPD classification from digital lung sound recordings obtained from candidates suspected of having chronic obstructive pulmonary disease, the method comprising collecting digital lung sound recordings from specified auscultation positions; pre-processing the lung sound signal using digital signal processing method and generating mel-spectrogram; public dataset is used to train COPD identification model and COPD classification model; the identification model divides the input lung sound spectrogram into three categories: COPD, healthy and other lung diseases; the classification model divides COPD patients into 0-4 levels as specified in the treatment guidelines.

[0007] The application provides a chronic obstructive pulmonary disease identification system based on digitized lung sound, comprising the following modules: a data acquisition module, a preprocessing module, a conversion module, a data module, a data processing module, a data enhancement training module, a deep learning module, an identification module, a grading module and a display module.

[0008] Further, the data acquisition module comprises a lung sound acquisition device composed of an electronic stethoscope and a recording device. The electronic stethoscope is placed at the six auscultation positions and the trachea of the candidate, such as a patient, to collect the lung sound generated by the friction of the airflow in the lungs during breathing. The recording device digitizes the lung sound for analysis, collects the lung sound generated by the friction of the airflow in the lungs during the candidate's breathing, digitizes and stores it for subsequent preprocessing and other modules. The entire process of preprocessing and classification of the lung sound is automatically completed by a software and hardware device system (including a computer and other mobile devices), and the classification result is displayed by the device carrying the identification system. Figure 1 Further, the data acquisition module comprises a lung sound acquisition device composed of an electronic stethoscope and a recording device. The electronic stethoscope is placed at the six auscultation positions and the trachea of the candidate, such as a patient, to collect the lung sound generated by the friction of the airflow in the lungs during breathing. The recording device digitizes the lung sound for analysis, collects the lung sound generated by the friction of the airflow in the lungs during the candidate's breathing, digitizes and stores it for subsequent preprocessing and other modules. The entire process of preprocessing and classification of the lung sound is automatically completed by a software and hardware device system (including a computer and other mobile devices), and the classification result is displayed by the device carrying the identification system.

[0009] Further, the preprocessing module is used to remove the mixed environmental noise, heart sound and power frequency interference in the collected lung sound record, and simultaneously performs normalization processing to obtain effective lung sound data.

[0010] Further, the conversion module is used to convert the preprocessed audio into a mel spectrogram through Fourier transform and a mel filter bank.

[0011] Further, the data module comprises three data sets, namely ICBHI Database, King Abdullah University Hospital database and RespiratoryDatabase@TR; wherein the ICBHI Database and the King Abdullah University Hospital database are combined, and are used to provide a lung sound data set for COPD identification, and the data set is divided into a training set and a test set; the RespiratoryDatabase@TR public data set is used to provide a lung sound data set for COPD grading, and the data set is divided into a training set and a test set.

[0012] Further, the data processing module acquires the spectrogram through the preprocessing module and the conversion module.

[0013] Further, the data enhancement training module is used to train the training set of the converted spectrogram in the public data set through a data augmentation network, including a VQ-VAE deep learning data augmentation model. In the training set, the VQ-VAE is trained, and a balanced training data set after enhancement is obtained. The balanced training set enters the subsequent deep learning module.

[0014] Further, the deep learning module includes a Resnet50-based diagnosis and grading model, which is initialized based on the pre-trained weights on ImageNet, and then retrained using the training set of the diagnosis data set / grading data set after the data enhancement module. Model training adopts a focal loss function and a warmup learning rate optimization to improve model performance, and the test set is used to evaluate, test and verify the Resnet50-based model.

[0015] Further, the identification module is a Resnet50-based identification model, which is trained for 50 epochs, and the training accuracy no longer improves and the loss no longer decreases. The model with the highest accuracy, sensitivity and specificity on the test set is used as the final identification module. The identification module is provided with an input end and an output end. The input end is a mel spectrogram of the lung sound collected from the candidate after the preprocessing module and the conversion module. The output end outputs the identification result of the candidate, i.e., whether the candidate has COPD or other lung diseases other than COPD, or is healthy.

[0016] Further, the grading module is a Resnet50-based grading model, which is trained for 50 epochs, and the training accuracy no longer improves and the loss no longer decreases. The model with the highest accuracy, sensitivity and specificity on the test set is used as the final grading module. The grading module divides the input COPD lung sound into COPD0, COPD1, COPD2, COPD3, or COPD4. The grading module is provided with an input end and an output end. The input end is a mel spectrogram of the digital lung sound collected from the candidate after the preprocessing module and the conversion module. The output end gives the COPD disease level of the candidate.

[0017] Further, the display module displays the output end determination result in the device equipped with the identification system, and the device equipped with the identification system includes a computer, a mobile phone or hardware.

[0018] Further, the method for removing mixed environmental noise, heart sound and power frequency interference in the collected lung sound record is five-order Butterworth filter and empirical mode decomposition denoising.

[0019] Further, the training method of the data augmentation network is: different from traditional oversampling methods (random oversampling, random cropping, adding noise, SMOTE and ADASYN methods), the method is based on vector quantization autoencoder (vq_VAE). In the training set of the public data set with uneven sample distribution, the sound spectrum of the class with less samples is input into the vq-VAE network for training, the network "learns" the probability distribution characteristics and autoregressive prior characteristics of the sound spectrum through coding and decoding and PixelSnail algorithm, and then serves as a generator to sample and decode from the feature space to generate new samples belonging to the class, so that the balanced training data set after enhancement is obtained, the method is based on deep learning, overcomes the distribution edge problem of traditional methods, and can improve the performance of the subsequent identification model. The application provides a construction method of a chronic obstructive pulmonary disease identification system based on digitized lung sound, comprising the following steps:

[0020] 1) Collect two public data sets ICBHI Database and King Abdullah University Hospital database for lung sound used for training of the identification model, and divide the data sets into a training set and a test set in a ratio of 8:2 after denoising processing; collect RespiratoryDatabase@TR public data set to provide lung sound used for training of the grading model, and divide the data set into a training set and a test set in a ratio of 8:2 after denoising processing;

[0021] 2) Training of the enhancement network in the training set, in the training set with uneven sample distribution, the sound spectrum of the class with less samples is input into the vq-VAE network for training, the network "learns" the probability distribution characteristics and autoregressive prior characteristics of the sound spectrum through coding and decoding and PixelSnail algorithm. Then as a generator, sample and decode from the feature space to generate new samples belonging to the minority class. In this way, after the vq-VAE process, the balanced training data set after enhancement is obtained;

[0022] 3) The balanced training data set after enhancement adopts a transfer learning method for deep learning, the model based on Resnet50 is initialized by using the pre-trained weights on ImageNet, and then the enhanced training set is retrained. The model training adopts a focal loss function and a warmup learning rate optimization to improve the model performance, and the model is evaluated, tested and verified through the test set;

[0023] 4) The model with the best performance obtained through deep learning serves as the final identification and classification module. The identification and classification module categorizes the input lung sound spectrograms into three classes: COPD, healthy individuals, and other non-COPD lung diseases. The non-COPD category includes diseases such as pneumonia, asthma, bronchial asthma, bronchiolitis, and upper / lower respiratory tract infections. The classification module categorizes the input COPD lung sounds into COPD level 0, COPD level 1, COPD level 2, COPD level 3, or COPD level 4. Both the identification and classification modules have input and output terminals; the input terminal is the Mel spectrogram of the candidate.

[0024] 5) The Mel spectrograms of the candidates are obtained from the lung sounds generated by the friction of airflow in the lungs during the candidates' breathing. After noise reduction and normalization, the effective lung sound data is obtained. After Fourier transform and Mel filter bank, the lung sounds are converted into Mel spectrograms.

[0025] 6) The identification module and the grading module provide the identification results and grading results of the candidates and display them on the device interface equipped with the identification system.

[0026] This invention provides the use of a chronic obstructive pulmonary disease (COPD) identification system based on digital lung sounds in the preparation of auxiliary diagnostic tools for COPD.

[0027] The beneficial effects of this invention are as follows: This invention analyzes the lung sound spectrum through a fully trained deep learning model, which can make a judgment on the health status of the subject (having COPD, having other lung diseases besides COPD, healthy, and COPD disease level), and can be used for initial diagnosis and triage in clinical applications.

[0028] The lung sounds required for diagnosis can be collected clinically using an electronic stethoscope or recorded in non-clinical conditions using a microphone. In addition to being used for diagnosis using the methods proposed in this patent, digital lung sounds can also be uploaded to the cloud for doctors to listen to remotely. This provides the possibility for patients to have an initial diagnosis of COPD at home, reducing the risk of disease transmission.

[0029] A diagnosis of COPD in a candidate can be made solely through auscultation, a procedure currently unavailable in clinical practice. The diagnosis requires only auscultation of lung sounds, eliminating the need for pulmonary function tests and radiological examinations. This simplifies the diagnostic process, avoids radiation exposure to the subject during examinations, and mitigates missed diagnoses and misdiagnoses due to poor patient cooperation during pulmonary function tests. Furthermore, compared to the potential droplet generation during the blowing process required for pulmonary function tests, auscultation reduces the risk of disease transmission to some extent.

[0030] The identification model is trained on a multi-center public lung sound dataset. A deep generative network is used to augment the public dataset to improve the performance of the identification model. The identification result is completed by a fully trained deep learning model, avoiding the subjective interpretation of the clinician in auscultation and other test results, which may cause misdiagnosis and missed diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a system flowchart.

[0032] Figure 2 is a lung sound data collection diagram.

[0033] Figure 3 is a ROC curve diagram of the test set divided from the public dataset.

[0034] Figure 4 is a confusion matrix diagram of the test set divided from the public dataset.

[0035] Figure 5 is a digitized lung sound and mel-spectrogram. DETAILED DESCRIPTION

[0036] Figure 1 is a whole system flowchart. The microphone is placed at the position as shown in Figure 2 , and the candidate takes a sitting position during the collection process. The lung sound of the candidate for 20s when breathing calmly is recorded for subsequent analysis. The collected lung sound record enters the pre-processing system of the lung sound, and the Butterworth high-pass filter and the empirical mode decomposition method are used to filter and reduce the environmental noise, heart sound and power frequency noise mixed in the record during the collection process, and then the signal is normalized. The pre-processed audio is subjected to Fourier transform and mel filter bank, and the lung sound is converted into a mel-spectrogram.

[0037] ICBHI Database and King Abdullah University Hospital database are collected as two public data sets for lung sound used for identification model training. After denoising the data set, it is divided into training set and test set according to the ratio of 8:2; Respiratory Database@TR public data set is collected to provide lung sound for grading model training, and after denoising the data set, it is divided into training set and test set according to the ratio of 8:2.

[0038] As shown in Figure 3 and Figure 4 , the applied identification model performs well on the test set divided before training, with sensitivity, specificity and F1 score of 0.94, 0.98 and 0.94, respectively. High specificity and sensitivity indicate that the model has a very low rate of missed diagnosis and misdiagnosis.

[0039] The training of the enhancement network is carried out in the training set, and in the training set with uneven sample distribution, the vq-VAE network is trained by inputting the sound spectrum of the class with less sample number, and the network "learns" the probability distribution characteristics and autoregressive prior characteristics of the sound spectrum through coding and decoding and the PixelSnail algorithm. Then as a generator, it decodes the samples from the feature space to generate new samples belonging to the minority class. In this way, after the vq-VAE process, the balanced training data set after enhancement is obtained;

[0040] The balanced training data set after enhancement adopts a transfer learning method for deep learning, and the model based on Resnet50 is initialized with the pre-trained weights on ImageNet, and then retrained with the enhanced training set. The model training adopts a focal loss function and a warmup learning rate optimization to improve the model performance, and the model is evaluated, tested and verified through the test set;

[0041] The performance-optimized model obtained through deep learning is used as the final discrimination module and grading module. The discrimination module divides the input lung sound spectrum into three categories of COPD, healthy and non-COPD other lung diseases, and the non-COPD category includes diseases such as pneumonia, asthma, bronchial asthma, bronchiolitis, upper / lower respiratory tract infection, etc.

[0042] The grading module divides the input COPD lung sound into COPD0, COPD1, COPD2, COPD3 or COPD4, and the discrimination and grading modules are provided with input and output ends. The input end is the candidate's mel-spectrogram as shown in Figure 5 The diagnosis module and the grading module give the diagnosis result of the candidate and display it on the device interface equipped with the discrimination system.

Claims

1. A chronic obstructive pulmonary disease (COPD) identification system based on digital lung sounds, characterized in that, It includes the following modules: data acquisition module, preprocessing module, conversion module, data module, data processing module, data augmentation training module, deep learning module, discrimination module, classification module, and display module; The data acquisition module is used to acquire lung sounds generated by airflow friction during the breathing of candidates, digitize and store them for subsequent preprocessing module; the preprocessing module is used to remove environmental noise, heart sounds and power frequency interference mixed in the acquired lung sound records, and perform normalization processing to obtain effective lung sound data; the conversion module is used to convert the preprocessed audio into a Mel spectrogram by Fourier transform and Mel filter bank. The data module includes three datasets: the ICBHI Database, the King Abdullah University Hospital database, and the RespiratoryDatabase@TR. The ICBHI Database and King Abdullah University Hospital database are two publicly available datasets combined to provide a lung sound dataset for COPD identification; this dataset is divided into training and testing sets. The RespiratoryDatabase@TR publicly available dataset provides a lung sound dataset for COPD grading; this dataset is also divided into training and testing sets. The data processing module uses a preprocessing module and a transformation module to obtain the sound spectrum from all the data in the datasets. The data augmentation training module is used to train the data augmentation network using the spectrum conversion training set from the public dataset, including the VQ-VAE deep learning data augmentation model. The training set is used for VQ-VAE training, resulting in an augmented and balanced training dataset. This balanced training set is then fed into the subsequent deep learning module. The deep learning module includes ResNet50-based discrimination and classification models. The ResNet50-based model is initialized with weights pre-trained on ImageNet, and then trained again in this module using the balanced discrimination / classification dataset training set obtained from the data augmentation module. Model training employs a focus loss function and warmup learning rate optimization to improve model performance. The test set is used to evaluate, test, and validate the ResNet50-based model. The discrimination module is based on a ResNet50 discrimination model. After 50 epochs of training, the training accuracy no longer improves and the loss no longer decreases. The model with the highest accuracy, sensitivity, and specificity on the test set is selected as the final discrimination module. The discrimination module classifies the input lung sounds into those with COPD, those with other lung diseases besides COPD, or healthy individuals. The discrimination module has an input end and an output end. The input end is the Mel-spectrum of lung sounds collected from candidates, processed by the preprocessing and conversion modules. The output end outputs the discrimination result of the candidate, i.e., whether the candidate has COPD or other lung diseases besides COPD, or is healthy. The grading module is based on a ResNet50 grading model. After 50 epochs of training, the training accuracy no longer improves and the loss no longer decreases. The model with the highest accuracy, sensitivity, and specificity on the test set is used as the final grading module. The grading module classifies the input COPD lung sounds into COPD level 0, COPD level 1, COPD level 2, COPD level 3, or COPD level 4. The grading module has an input end and an output end. The input end is the Mel spectrogram of digitized lung sounds collected from candidates after preprocessing and conversion modules. The output end gives the COPD severity level of the candidate.

2. The chronic obstructive pulmonary disease identification system based on digital lung sounds according to claim 1, characterized in that, The display module is used to display the output judgment result in a device equipped with an identification system.

3. The chronic obstructive pulmonary disease identification system based on digital lung sounds according to claim 1, characterized in that, The method for removing ambient noise, heart sounds, and power frequency interference from the acquired lung sound recordings is to use a fifth-order Butterworth filter and empirical mode decomposition for denoising.

4. The chronic obstructive pulmonary disease identification system based on digital lung sounds according to claim 1, characterized in that, The training method of the data augmentation network is as follows: In the training set of the public dataset with uneven sample distribution, the spectrum of the class with fewer samples is input into the vq-VAE network for training. The network "learns" the probability distribution characteristics and autoregressive prior features of the spectrum of the class through encoding and decoding and the PixelSnail algorithm. Then, it is used as a generator to sample and decode from the feature space to generate new samples belonging to the class, thus obtaining the augmented and balanced training dataset.

5. A method for constructing a chronic obstructive pulmonary disease (COPD) identification system based on digital lung sounds as described in any one of claims 1 to 4, characterized in that, Includes the following steps: 1) Collect lung sounds from two public datasets, the ICBHI Database and the King Abdullah University Hospital database, for use in training the identification model. After denoising the datasets, divide them into training and test sets in an 8:2 ratio. Collect lung sounds from the RespiratoryDatabase@TR public dataset for training the graded model. After denoising the datasets, divide them into training and test sets in an 8:2 ratio. 2) The augmentation network is trained in the training set. In the training set with an imbalanced sample distribution, the sound spectrum of the category with a small number of samples is input into the vq-VAE network for training. The network "learns" the probability distribution features and autoregressive prior features of the sound spectrum through encoding and decoding and the PixelSnail algorithm. Then, as a generator, it samples and decodes from the feature space to generate new samples belonging to the minority class; Thus, through the vq-VAE process, an augmented and balanced training dataset is obtained; 3) The augmented and balanced training dataset is used for deep learning using transfer learning. The ResNet50-based model is initialized with weights pre-trained on ImageNet and then retrained using the augmented training set. The model training employs a focus loss function and warmup learning rate optimization to improve model performance, and the model is evaluated, tested, and validated using a test set. 4) The model with the best performance obtained through deep learning is used as the final identification module and classification module. The identification module classifies the input lung sounds into those with COPD, those with other lung diseases besides COPD, or healthy. The classification module classifies the input COPD lung sounds into COPD level 0, COPD level 1, COPD level 2, COPD level 3, or COPD level 4. The identification and classification modules have input and output terminals. The input terminal is the Mel spectrogram of the candidate. 5) The Mel spectrograms of the candidates are obtained from the lung sounds generated by the friction of airflow in the lungs during the candidates' breathing. After noise reduction and normalization, the effective lung sound data is obtained. After Fourier transform and Mel filter bank, the lung sounds are converted into Mel spectrograms. 6) The identification module and the grading module provide the identification results or grading results of the candidates and display them on the device interface equipped with the identification system.

6. The use of the chronic obstructive pulmonary disease identification system based on digital lung sounds as described in any one of claims 1 to 4 in the preparation of an auxiliary diagnostic tool for chronic obstructive pulmonary disease.

Images