Lung ventilation function measuring method and device based on smartphone microphone

Abstract

The invention belongs to the field of artificial intelligence and non-contact measurement, and relates to a lung ventilation function measurement method and device based on a smartphone microphone. A built-in microphone of the smart phone is used for collecting breathing audio signals of a user in the measurement process, and a lung ventilation function true value is obtained through the lung function instrument to serve as reference data; dividing an inspiration phase and an expiration phase of the breathing audio signal based on a threshold method, and extracting an expiration phase audio clip; processing the expiration signal by using a self-adaptive convolution frequency domain enhancement network, dynamically adjusting a convolution kernel to adapt to different frequency bands by self-adaptive convolution, and enhancing low-frequency components by combining frequency domain enhancement with fast Fourier transform; the transformation type convolutional network models time sequence dependence through causal convolution and jump convolution, and the feature learning ability is enhanced through residual connection and dense connection; and finally predicting lung ventilation function parameters through full connection layer regression. According to the invention, the accuracy of lung ventilation function measurement based on the microphone of the smart phone is improved, and the user can conveniently carry out self-health monitoring.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of artificial intelligence and non-contact measurement, and relates to a lung ventilation function measurement method and device based on a smartphone microphone. BACKGROUND

[0002] With the rising incidence of chronic obstructive pulmonary disease, asthma, and lung cancer, among other respiratory diseases, they have become one of the major diseases that seriously threaten human health worldwide. According to the data of the World Health Organization, the incidence and mortality rate caused by respiratory diseases is showing a continuous growth trend, which is a heavy burden on the public health system. In this context, the assessment of lung ventilation function as a key link in the screening and diagnosis of respiratory diseases is of great significance for timely detection of ventilation abnormalities, judgment of airflow limitation type and development of intervention measures. However, traditional lung ventilation function measurement usually relies on dedicated lung function instruments and needs to be completed by professionals in qualified medical institutions, with a cumbersome and time-consuming operation process, and high requirements for hardware resources and operating environment. This mode not only causes high inspection costs, but also limits the widespread application in primary and home scenarios. Therefore, developing a low-cost, easy-to-operate lung ventilation function detection means suitable for daily environment has become an urgent need to improve disease early screening ability and resource utilization efficiency.

[0003] In recent years, with the rapid development of smart wearable technology, portable devices such as smartwatches are gradually playing an increasingly important role in personal health management. Relying on built-in multi-modal high-precision sensors such as accelerometers, gyroscopes, and photoelectric sensors, smartwatches can continuously obtain the body movement state and physiological information of users, and show significant advantages in dynamic monitoring of various health indicators. Compared with traditional lung ventilation function measurement equipment, such smart devices not only have high portability and user-friendliness, but also can carry out continuous and non-invasive respiratory-related data collection in non-medical environments, providing a new feasible path for lung ventilation function monitoring in home or mobile scenarios. Especially in collecting signals such as user's respiratory sound and chest movement closely related to ventilation, smartwatches have significant practical potential.

[0004] The wide application of artificial intelligence technology, especially deep learning and machine learning algorithms in the medical and health field, provides a technical breakthrough for the intelligent assessment of lung ventilation function. By modeling a large number of time series signals collected by intelligent wearable devices, the AI system can automatically identify key feature parameters related to ventilation function, enabling early detection and type judgment of abnormal ventilation patterns. This technological progress not only improves the accuracy and automation of lung ventilation function assessment, but also provides strong support for individualized health intervention, chronic disease management, and remote medical services. In the future, lung ventilation function assessment methods based on the integration of intelligent devices and artificial intelligence are expected to be widely used in preclinical screening, chronic disease follow-up, and family health management, etc. SUMMARY

[0005] The present application aims to overcome the problems of expensive equipment, complex process and uneven distribution of medical resources in traditional lung ventilation function detection. Existing detection methods mostly rely on special instruments and need to be operated by professionals in medical institutions, which not only raises the detection threshold, but also limits the popularity and convenience of lung ventilation function monitoring to some extent. In view of the above limitations, the present application proposes a non-invasive lung ventilation function assessment method based on the built-in sensors of a smart phone. This method is low in cost, easy to operate and supports remote monitoring anytime and anywhere. Threshold segmentation technology is used to identify the respiratory cycle and extract the effective exhalation audio segment. Adaptive convolution and frequency domain enhancement network (ACF-Net) is used to process the exhalation audio signal, the adaptive convolution module dynamically adjusts the convolution kernel to capture different frequency band features, the frequency domain enhancement module combines the fast Fourier transform to strengthen the low frequency components, the transform convolution network (TCN) uses causal convolution and residual connection to model the time series dependence, and the dense connection enhances the feature learning ability, and finally the peak expiratory flow (PEF), forced expiratory volume in one second (FEV1) and forced vital capacity (FVC) are predicted by the full connection layer regression.

[0006] The technical solution of the present application is:

[0007] A lung ventilation function measurement method based on a smart phone microphone, comprising the following steps:

[0008] (1) Data acquisition

[0009] The built-in microphone of the smart phone is used to collect the breathing audio signal of the user during the measurement process, and the lung function instrument is used to obtain the true value of the lung ventilation function as reference data for subsequent modeling analysis; the collected true value of the lung ventilation function includes peak expiratory flow (PEF), forced expiratory volume in one second (FEV1) and forced vital capacity (FVC), which are used as the labeling benchmark for subsequent feature extraction and model training;

[0010] (2) Data preprocessing

[0011] The respiratory audio signal collected by the smart phone is time-aligned with the true value of lung ventilation function obtained by the lung function instrument, so as to realize accurate correspondence of the two types of data in the key exhalation stage; the respiratory audio signal is processed by a high-pass filter and a low-pass filter to remove low-frequency noise and high-frequency interference in the signal, remove the interference of external environmental noise, and divide the preprocessed respiratory audio signal into a training set and a test set;

[0012] (3) Building a lung ventilation function measurement model based on a smart phone microphone

[0013] The lung ventilation function measurement model comprises a respiratory phase segmentation module, an adaptive convolution and frequency domain enhancement network, a transform convolution network and a full connection layer.

[0014] The respiratory phase segmentation module: a threshold-based method is used to perform respiratory phase segmentation in the preprocessed respiratory audio signal, the inspiratory phase and expiratory phase in the respiratory audio signal are distinguished by analyzing the amplitude and frequency characteristics of the respiratory audio signal, the method can accurately identify and extract the audio segment of the expiratory phase, this step ensures that the model only performs subsequent feature extraction and analysis in the expiratory phase, thereby avoiding the interference of the inspiratory phase;

[0015] The adaptive convolution and frequency domain enhancement network: an adaptive convolution module and a frequency domain enhancement module are introduced to process expiratory audio signals of different frequency ranges; the functions of the adaptive convolution and frequency domain enhancement network: after the respiratory audio signal is segmented, the adaptive convolution module adaptively selects the size, number and structure of the convolution kernel according to the frequency characteristics of the expiratory audio signal, so as to capture the slight changes in the frequency response of the expiratory audio signal, adapt to the characteristics of different frequency bands, and ensure that the information of different frequency bands can be effectively extracted; the frequency domain enhancement module combines the fast Fourier transform technology to enhance the low-frequency components in the signal, and processes the frequency domain features through the convolution layer to combine the frequency domain features with the time domain features;

[0016] The transform convolution network comprises a plurality of scalable convolution filters, the scalable convolution filters are introduced to process the time sequence dependence in long sequence data, each scalable convolution filter is composed of a causal convolution, a jump convolution, normalization and a ReLU activation function; the combination of the causal convolution and the jump convolution enhances the feature expression after the time-frequency fusion of the adaptive convolution and the frequency domain enhancement network, and the combination ensures that each layer of convolution only depends on historical data without using future information; the normalization scales the enhanced features to the same scale and reduces the difference between different features; the activation function is a ReLU function; the residual connection combines the output of the first layer of scalable convolution filters with the third layer, ensures effective transmission of information between layers, avoids information loss, the dense connection combines the output of the first layer of scalable convolution filters with the second layer, and combines the output of the second layer of scalable convolution filters with the third layer, so that the output of each layer is connected with all previous layers, forms a dense feature mapping, and enables the network to more comprehensively utilize the features of each layer;

[0017] The full connection layer is used as an output layer to obtain PEF, FEV1 and FVC;

[0018] (4) The lung ventilation function measurement model based on the smart phone microphone built in step (3) is trained, the hyperparameters are adjusted, iterative optimization is performed to obtain optimal performance, and the optimal lung ventilation function measurement model with the lowest loss is saved;

[0019] (5) The optimal lung ventilation function measurement model is used to evaluate individuals in a test set, and the effectiveness of the method is verified;

[0020] (6) The lung ventilation function measurement model verified by the test is used to measure the actual lung ventilation function;

[0021] A lung ventilation function measurement device based on a smart phone microphone comprises at least one processor and a memory in communication connection with the at least one processor;

[0022] The memory stores instructions executable by the processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the lung ventilation function measurement method based on the smart phone microphone.

[0023] The present application has the following beneficial effects:

[0024] (1) In the present application, the audio data is captured in the microphone by the smart phone without any external device, and remote and home evaluation can be realized.

[0025] (2) The application provides a lung ventilation function measurement model based on a smartphone microphone, solves the problems of low feature extraction efficiency and inaccurate measurement results, and realizes accurate measurement of lung ventilation function.

[0026] (3) The application realizes measurement of lung ventilation function indicators by mining and utilizing respiratory audio signals, can be performed in a remote or home environment, greatly improves the accessibility and convenience of lung ventilation function detection, and facilitates self-health monitoring of users. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1 is a lung ventilation function measurement model structure diagram for a smartphone microphone according to the application;

[0028] Figure 2 is a structure diagram of an exhalation audio feature extraction network according to the application, corresponding to the adaptive convolution and frequency domain enhancement network in Figure 1 .

[0029] Figure 3 is a feature expression enhancement network structure diagram according to the application, corresponding to the transform convolution network in Figure 1 . DETAILED DESCRIPTION

[0030] In order to make the above-mentioned purpose, advantage more easy to understand, the following will combine the drawings and specific implementation mode to further explain the application.

[0031] A lung function measurement method based on a smartphone, as shown in Figure 1 . The specific implementation process is as follows:

[0032] Step 1: Create a data set;

[0033] An Android Studio development data collection application is used to collect audio data; the specific collection process is as follows:

[0034] A medical lung function instrument is used to collect lung ventilation function true values, including PEF, FEV1 and FVC, and a smartphone microphone is used to collect respiratory audio signals; combined with the lung function measurement guidelines jointly formulated by the American Thoracic Society (ATS) and the European Respiratory Society (ERS) and the lung function examination guidelines written by the National Key Laboratory of Respiratory System Diseases, a data collection process is developed, and the specific collection process is as follows:

[0035] Pulmonary ventilation function true value data collection: 1) First, take a deep breath and keep still, then try to completely expel the gas in the lungs; 2) Hold the mouthpiece correctly in the mouth, lightly bite the upper and lower edges of the mouthpiece with the teeth, and naturally cover the lips to ensure airtightness to prevent air leakage; 3) Then take a deep breath quickly and immediately exhale forcefully, simulating the explosive action of blowing out a candle, and the gas exchange should only be completed through the oral cavity during the entire process to avoid nasal involvement, and the blowing process should be continuous and uninterrupted; 4) Finally, take a deep breath to restore the breathing state, and then rest for about 5 minutes to prepare for the next measurement;

[0036] Smartphone audio data collection: 1) Fix the phone on a bracket parallel to the face, then take a deep breath and keep still, try to completely expel the gas in the lungs; 2) After clicking the "Start Recording" button, quickly complete a deep breath, followed by an explosive blowing action (similar to blowing out a candle), and only exhale through the oral cavity throughout the process to avoid nasal involvement, and the exhalation process needs to be continuous and smooth without interruption; 3) After completing the blowing, click the "End Recording" button, then take a deep breath to restore the resting state, and rest for about 3 minutes; each tester repeats three times.

[0037] Step 2: Preprocess the raw data and divide it into training and test sets;

[0038] The respiratory audio signal collected by the smartphone and the pulmonary ventilation function true value obtained by the lung function instrument are time-aligned, so that the two types of data are accurately corresponding in the key exhalation phase.

[0039] The respiratory audio signal is processed by high-pass and low-pass filters to remove low-frequency noise and high-frequency interference in the signal, as well as external environmental noise interference, obtaining preprocessed respiratory audio data;

[0040] The nested five-fold cross-validation strategy is used, with 80% of the data in each external fold used for model training and 20% for model testing, and in each internal fold, the training data is further divided into 80% for training and 20% for validation, i.e. the training set of each internal fold accounts for 64%, and the validation set accounts for 16%, the external fold is only used to evaluate the generalization performance of the model, which can avoid the data from the same individual appearing in the training set and test set of the same fold, thereby causing the model performance to be overestimated.

[0041] Step 3: Build a smartphone-based lung function measurement method model, as shown in Figure 1 ;

[0042] The model mainly includes a respiratory phase segmentation module, an adaptive convolution and frequency domain enhancement network, a transform convolution network, and a fully connected layer;

[0043] After preprocessing the respiratory audio signal, a threshold-based respiratory phase segmentation method is used to clearly distinguish the inspiration phase and expiration phase in the respiratory audio signal. By analyzing the amplitude and frequency characteristics of the respiratory audio signal, the audio segments of the expiration phase can be accurately identified and extracted, effectively avoiding the interference of the inspiration phase. After expiration phase segmentation, an adaptive convolutional frequency domain enhancement network is applied to process the expiration audio signal in lung ventilation function measurement. The network uses an adaptive convolutional frequency domain enhancement module to process the features in different frequency ranges of the audio signal and extract effective data related to lung ventilation function. The adaptive convolutional frequency domain enhancement network is shown in Figure 2 The model automatically selects the size, number and structure of the convolution kernel according to the frequency characteristics of the expiration audio signal, so as to capture the subtle changes in the frequency response of the expiration audio signal. The low-frequency component is related to the strength of the expiration, while the high-frequency component is related to the subtle changes in the expiration, lung structure and other factors. Through this dynamic adjustment, ACF-Net can more flexibly cope with signals in various frequency ranges, ensuring that information in different frequency bands can be effectively extracted. The frequency domain enhancement module combines fast Fourier transform and is specifically used to enhance the low-frequency component in the expiration audio signal, ensuring that the model pays enough attention to the key features in the respiratory cycle. The module further processes the frequency domain features through a convolutional layer, combining frequency domain features with time domain features, so that the model can capture frequency changes and preserve temporal continuity.

[0044] The transform convolutional network is shown in Figure 3 It is used to process the time series dependence in long sequence data and model the time series features in the expiration audio signal. TCN can handle long-time span dependencies by introducing scalable convolutional filters, allowing the model to better understand the temporal dynamics of the expiration audio signal. The combination of causal convolution and jump convolution ensures that each convolution operation only depends on historical data, thus meeting the causal relationship requirements of time series analysis. The TCN model introduces residual connections to ensure effective training of the network, especially when dealing with complex lung ventilation function signals. Residual connections help preserve key information and prevent information loss during multi-layer transmission. Dense connections connect the output of each layer to all previous layers, forming a dense feature mapping that enhances the model's ability to learn complex lung function features, allowing the network to effectively extract key information in each layer.

[0045] The output of TCN is processed through a fully connected layer to obtain the final lung ventilation function measurement result.

[0046] Step 4: Set hyperparameters and train the model.

[0047] In the model training process, the hyperparameter setting of the model epoch is 100, the learning rate is set to 0.001, the Adam optimizer is used, the batch size is set to 8, in addition, the learning rate dynamic reduction strategy is used, when the loss of 5 consecutive epochs does not appear, the learning rate is reduced to half of the original.

[0048] Step 5: Test the model, use the trained optimal model to predict the individuals in the test set, and calculate the average absolute error percentage as the evaluation index.

Claims

1. A method for measuring pulmonary ventilation function based on a smartphone microphone, characterized in that, Includes the following steps: (1) Data collection; The user's breathing audio signals during the measurement process are collected using the microphone built into the smartphone, and the true value of lung ventilation function is obtained through the pulmonary function instrument as reference data for subsequent modeling and analysis. (2) Data preprocessing; (3) Build a lung ventilation function measurement model based on smartphone microphone; The lung ventilation function measurement model includes a respiratory phase segmentation module, an adaptive convolutional and frequency domain enhancement network, a transformative convolutional network, and fully connected layers; The respiratory phase segmentation module: performs respiratory phase segmentation using a threshold-based method on the preprocessed respiratory audio signal, and distinguishes the inspiratory and expiratory phases in the respiratory audio signal by analyzing the amplitude and frequency characteristics of the respiratory audio signal. The adaptive convolution and frequency domain enhancement network: introduces an adaptive convolution module and a frequency domain enhancement module to process expiratory audio signals in different frequency ranges; The functions of the adaptive convolution and frequency domain enhancement network are as follows: After the respiratory audio signal is segmented by the respiratory phase, the adaptive convolution module adaptively selects the size, number, and structure of the convolution kernel according to the frequency characteristics of the expiratory audio signal, thereby capturing the subtle changes in the frequency response of the expiratory audio signal to adapt to the characteristics of different frequency bands and ensure that information from different frequency bands can be effectively extracted; the frequency domain enhancement module combines the Fast Fourier Transform technique to enhance the low-frequency components in the signal and processes the frequency domain features through convolutional layers, combining the frequency domain features with the time domain features; The transformative convolutional network includes multiple scalable convolutional filters. These scalable convolutional filters are introduced to handle temporal dependencies in long sequence data. Each scalable convolutional filter consists of causal convolution, skip convolution, normalization, and ReLU activation functions. The combination of causal convolution and skip convolution enhances the feature representation after time-frequency fusion of adaptive convolution and frequency domain enhancement networks. This combination ensures that each convolutional layer relies only on historical data and does not use future information. Normalization scales enhanced features to the same scale, reducing the differences between different features; The activation function is the ReLU function; residual connections combine the output of the first layer scalable convolutional filter with the third layer to ensure effective information transfer between layers and avoid information loss; dense connections combine the output of the first layer scalable convolutional filter with the second layer, and at the same time combine the output of the second layer scalable convolutional filter with the third layer, so that the output of each layer is connected to all the previous layers, forming a dense feature map, which allows the network to make more comprehensive use of the features of each layer. The fully connected layer is used as the output layer to obtain PEF, FEV1, and FVC; (4) The lung ventilation function measurement model based on the smartphone microphone built in step (3) is trained, the hyperparameters are adjusted, and iterative optimization is performed to obtain the best performance. The optimal lung ventilation function measurement model with the lowest loss is saved. (5) The effectiveness of the method was verified by evaluating individuals in the test set using the optimal lung ventilation function measurement model; (6) Measure actual lung ventilation function using a well-tested and validated lung ventilation function measurement model.

2. The method for measuring lung ventilation function based on a smartphone microphone according to claim 1, characterized in that, In step (1), the collected true values ​​of lung ventilation function include peak expiratory flow (PEF), forced expiratory volume in one second (FEV1), and forced vital capacity (FVC), which are used as the annotation benchmarks for subsequent feature extraction and model training.

3. The method for measuring pulmonary ventilation function based on a smartphone microphone according to claim 1, characterized in that, In step (2), the data preprocessing specifically involves: aligning the respiratory audio signal collected by the smartphone with the true value of lung ventilation function obtained by the pulmonary function instrument in time, thereby achieving accurate correspondence between the two types of data in the key expiratory phase; the respiratory audio signal is processed by a high-pass filter and a low-pass filter to remove low-frequency noise and high-frequency interference in the signal, while also removing interference from external environmental noise, and the preprocessed respiratory audio signal is divided into a training set and a test set.

4. A lung ventilation function measurement device based on a smartphone microphone, characterized in that, include: At least one processor, and a memory communicatively connected to said at least one processor; The memory stores instructions executable by the processor, which are executed by the at least one processor to cause the at least one processor to perform a method for measuring lung ventilation function based on a smartphone microphone as described in any one of claims 1-3.

Images