Intelligent pneumonia prediction system based on cough sound and pulmonary respiration sound

Abstract

The invention relates to the technical field of cough sound and pulmonary respiration sound processing and prediction, and discloses an intelligent pneumonia prediction system based on cough sound and pulmonary respiration sound. The sound collection module comprises a cough sound collection unit and a pulmonary respiration sound collection unit, and the cough sound collection unit is used for collecting cough sound and sending the cough sound to the server detection module through the data transmission module for pneumonia prediction; the pulmonary respiration sound collection unit is used for collecting pulmonary respiration sound of a user and sending the pulmonary respiration sound to the server detection module through the data transmission module for pneumonia prediction; a trained Transform model is carried in the server detection module and is used for converting the audio data transmitted by the data transmission module into pictures for processing and prediction. According to the system, an open-source cough and pulmonary respiration sound database is concluded and arranged, and available cough sound and pulmonary respiration sound data are selected to be used for training a classifier through a Transform-based voice machine learning algorithm; and the accuracy of predicting pneumonia by cough and pulmonary respiration sound higher than 80% is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of cough sound and lung breath sound processing prediction, and in particular to an intelligent pneumonia prediction system based on cough sound and lung breath sound. Background Art

[0002] When a patient has pneumonia, due to lesions in the lungs, fluid will be retained in or between the alveoli in most cases, so the patient's cough and lung sounds will be abnormal. By collecting and analyzing these sounds, it is possible to predict whether the person being analyzed has pneumonia.

[0003] However, the traditional methods of diagnosing pneumonia are mainly CT imaging, biological testing, and stethoscope lung breath sounds, which are difficult to implement in a non-professional environment, rely on professional equipment, and require specialists to make a final judgment. For people who have difficulty in seeking medical treatment, a means of preliminarily predicting pneumonia is needed; Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In view of the deficiencies in the prior art, the present invention provides an intelligent pneumonia prediction system based on cough sounds and lung breath sounds, which has the advantages of making preliminary predictions of pneumonia based on the processing of cough sounds and lung breath sounds, and solves the above-mentioned technical problems.

[0006] (II) Technical solution

[0007] To achieve the above-mentioned object, the present invention provides the following technical solutions: an intelligent prediction system for pneumonia based on cough sounds and lung breath sounds, comprising a sound collection module, a data transmission module and a server detection module;

[0008] The sound collection module includes a cough sound collection unit and a lung breathing sound collection unit, wherein the cough sound collection unit is used to collect cough sounds and send them to the server detection module through the data transmission module for pneumonia prediction;

[0009] The lung breath sound collection unit is used to collect the user's lung breath sounds and send them to the server detection module through the data transmission module for pneumonia prediction;

[0010] The server detection module is equipped with a trained Transformer model for converting the audio data transmitted by the data transmission module into images for processing and prediction.

[0011] As a preferred technical solution of the present invention, the cough sound collection unit collects the sound through a device equipped with a microphone.

[0012] As a preferred technical solution of the present invention, the lung breath sound collection unit collects lung breath sounds through a lung breath sound collection device, and the lung breath sound collection device includes a piezoelectric sensor, an amplifier, an analog-to-digital converter, and a power supply.

[0013] Compared with the prior art, the present invention provides an intelligent pneumonia prediction system based on cough and lung breath sounds, which has the following beneficial effects:

[0014] 1. The present invention summarizes and organizes open source cough and lung breath sound databases, selects available cough and lung breath sound data for training classifiers based on Transformer-based speech machine learning algorithms, and obtains an accuracy rate of more than 80% for predicting pneumonia using cough and lung breath sounds.

[0015] 2. The present invention has completed a product for preliminary prediction of pneumonia based on cough and lung breath sounds; successfully designed an electronic stethoscope with a signal-to-noise ratio of up to 20:1, which can collect lung breath sounds and transmit them to the audio port or Type-C port of the mobile phone. It is suitable for users who may have pneumonia and cannot go to the hospital for diagnosis in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of cutting an audio signal according to the present invention;

[0017] Figure 2 A schematic diagram of converting the audio signal of the present invention into a time-frequency diagram;

[0018] Figure 3 It is a schematic diagram of the network structure diagram of the present invention;

[0019] Figure 4 This is a schematic diagram of the HST block structure of the present invention;

[0020] Figure 5 A schematic diagram of processing an audio signal according to the present invention;

[0021] Figure 6 This is a schematic diagram of the transformer of the present invention processing an image;

[0022] Figure 7 It is a schematic diagram of the working principle of the piezoelectric sensor and amplifier of the present invention;

[0023] Figure 8 This is a schematic diagram of the initial test of sound collection of the present invention;

[0024] Fig. 9 This is a schematic diagram of the improved sound collection test of the present invention;

[0025] Fig.10 This is a schematic diagram of further improvement of the sound collection of the present invention;

[0026] Fig.11 This is a schematic diagram of the recording operation interface of the present invention;

[0027] Fig.12 This is a schematic diagram of the recording operation interface of the present invention;

[0028] Fig.13 This is a schematic diagram of the change of the accuracy of the verification set data with the number of rounds of the present invention;

[0029] Fig.14 Schematic diagram of the timing of cough and lung breath sounds of the present invention;

[0030] Fig.15 For the present invention Fig.14 Schematic diagram of the time-frequency correspondence between cough and lung breath sounds;

[0031] Fig.16 Schematic diagram of the system framework of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] See also Figure 1-Figure 16 , an intelligent prediction system for pneumonia based on cough sounds and lung breath sounds, including a sound collection module, a data transmission module and a server detection module;

[0034] The sound collection module includes a cough sound collection unit and a lung breathing sound collection unit. The cough sound collection unit is used to collect cough sounds and send them to the server detection module through the data transmission module for pneumonia prediction. The cough sound collection unit collects the sounds through a device equipped with a microphone.

[0035] The lung breath sound collection unit is used to collect the user's lung breath sounds and send them to the server detection module through the data transmission module for pneumonia prediction. The lung breath sound collection unit collects the lung breath sounds through the lung breath sound collection device. The lung breath sound collection device includes a piezoelectric sensor, an amplifier, an analog-to-digital converter, and a power supply. The specific parameters are as shown in Table 1:

[0036]

[0037]

[0038] Table 1

[0039] Piezoelectric sensors and amplifiers:

[0040] The CM-01B contact microphone is composed of a piezoelectric film with good sensitivity and stable performance combined with a low-noise preamplifier circuit. It can provide unique sound and vibration signal pickup in the form of buffered output. When designing the product structure, the external noise interference is minimized as much as possible, so that when the rubber pad in the middle of the product receives the vibration signal, it can output a highly sensitive signal. CM-01B is an ideal sensor for monitoring body sound.

[0041] The amplifier is a CM-01 dedicated amplifier that is a public amplifier module for voltage AC signal amplification; the amplified signal can also be used for power output drive. Note that it only has audio output. If analog-to-digital conversion is required, type-C output is used, and the following analog-to-digital converter is required;

[0042] Analog-to-digital converter:

[0043] 1. AD7606 digital-to-analog conversion

[0044] As shown in Table 1, USB2AD7606 is a USB-controlled sampling board. The USB works in high-speed mode, and the data sampled by AD706 is transmitted to the computer. As a sampling board, AD7606 has 8 synchronous sampling channels, which can support signal changes within 10V. In addition, the host computer software and the sampling board are directly connected via a USB cable, and there is no need to connect the sampling board to an additional power supply.

[0045] 2. 10-channel USB interface AD ​​acquisition module

[0046] The STM32 10-channel ADC module uses the internal 10-channel ADC resources of the ST company's STM32F103 series microcontroller as the sampling core, and transfers the acquired sampling data to the memory through the internal DMA of the STM32, which reduces the CPU burden and enhances the stability and accuracy of AD sampling. At the same time, the onboard USB to serial port chip can transmit the data to the host computer;

[0047] When using, you only need to load and install the driver of the USB to serial port chip CH340, and then you can use the serial port debugging assistant to receive data files normally. CH340 is a USB bus adapter chip, which realizes USB to serial port, USB to IrDA infrared or USB to printer port. In serial port mode, CH340 provides common MODEM communication signals, which are used to expand asynchronous serial ports for computers, or directly upgrade ordinary serial port devices to USB bus.

[0048] After the data is sampled and transmitted to the mobile phone, the distortion rate of the audio signal will be very small, and it can have a higher clarity when analyzed at the receiving end; when the present invention is implemented, considering the trouble of audio analysis and re-synthesis, the method of transmitting audio data in wave format is selected for uploading to the cloud for analysis;

[0049] Battery Power:

[0050] The lithium battery is a 18650 battery pack. The power switch is responsible for control after the battery is inserted. When the switch is connected, the rear indicator light will be always on. Click twice to turn off the switch to save power. This battery pack has three groups of interfaces on the left and right sides, of which the left side is a 3.3V 1A interface and the right side is a 5v 2A interface.

[0051] The server detection module is equipped with a trained Transformer model to convert the audio data transmitted by the data transmission module into images for processing and prediction. The specific construction process is as follows:

[0052] S1. Collect open source lung breath sound datasets and open source cough databases. The open source lung breath sound datasets include:

[0053] ICBHIDatabase: The 2017 International Conference on Biomedical and Health Informatics (ICBHI) database was collected independently over several years by two research teams in two different countries (The University de Aveiro, Portugal and The Aristotle University of Thessaloniki, Greece); the database consists of a total of 5.5 hours of recordings from 126 subjects with an age range of 4 months to 91 years; the sounds were collected sequentially from 6 chest locations, as shown in the figure; it contains 6898 respiratory cycles, of which 1864 contain crackle sounds, 886 contain wheezes, and 506 contain both crackle and wheezes;

[0054] King Abdullah University Hospital (KAUH) Database: This dataset includes lung sounds from 112 subjects aged 12 to 90 years (35 healthy subjects and 77 unhealthy subjects); the database has respiratory sound annotations, including 35 normal lung sounds, 23 crepitus, 4 wheezes, 8 crackles, 1 bronchial sound, 2 wheezes with crackles, and 2 bronchial sounds with crackles;

[0055] There are three types of filters included in the data:

[0056] Bell mode: amplifies the sound in the frequency range [20-1000] Hz and emphasizes the low-frequency sound in the frequency range [20-200] Hz;

[0057] Diaphragm mode: amplifies the sound in the frequency range [20-2000] Hz and emphasizes the sound in the frequency range [100-500] Hz;

[0058] Extend mode: amplifies the sound in the frequency range [20-1000] Hz and emphasizes the sound in the frequency range [50-500] Hz;

[0059] The open source cough database includes:

[0060] Coswara Database: collected by the Coswara project of the Indian Institute of Science (IISc), Bangalore, it collects 1398 audio files of breathing-deep, breathing-shallow, cough-heavy, cough-shallow, counting fast, counting normal, vowel-a, vowel-e, vowel-o of subjects aged 1 to 81 years;

[0061] COUGHVID Database: Deployed in Switzerland The dataset was collected using a web application on a private server at the Fédérale de Lausanne (EPFL) facility in Lausanne. The dataset provides 25,000 cough recordings with the participants’ age, gender, geographic location, and COVID-19 status recorded. Four experienced physicians labeled more than 2,800 recordings.

[0062] Fortis Hospital, Vasant Kunj Databse: This dataset contains a total of 532 audio samples, including 207 crepitus audio samples, 60 normal audio samples, 106 dry rales audio samples, and 159 wheezing audio samples. Due to ethical reasons, the identity information of the participants is not disclosed in the dataset.

[0063] S2, screen the data collected in S1 for pneumonia samples, and establish a lung sound dataset and a cough database;

[0064] Lung sounds dataset:

[0065] Since ICBHIDatabase and KAUH Database contain a variety of respiratory diseases such as pneumonia, asthma, bronchitis and acute lower respiratory tract infection, it is necessary to screen out the pneumonia samples. 34 healthy samples and 37 pneumonia samples in ICBHIDatabase are selected, each sample is 20s long, the healthy samples are 4 months to 16 years old, including 13 females and 13 males, the pneumonia samples are 4 to 81 years old, including 2 females and 4 males, 81 healthy samples and 15 pneumonia samples are selected from KAUH Database, among which the BDE patterns of healthy samples are 32, 26, and 23 respectively, and the BDE patterns of pneumonia samples are 5, and the length of each sample segment is 15s-30s. The healthy samples are 18 to 81 years old, including 11 females and 24 males, and the pneumonia samples are 36 to 70 years old, including 2 females and 3 males, which constitute the total data set, namely the healthy sample lung sound data set.

[0066] Cough Database:

[0067] Similar to the lung sound dataset, when building a cough database, we must select the audio files corresponding to the pneumonia samples and the healthy samples. Therefore, we selected 210 healthy samples and 112 pneumonia samples from the Coswara Database ( Figure 2 .9-2.11); only cough-heavy and cough-shallow audio files were collected; each sample was 5 seconds long; the collected pneumonia samples were aged 1 to 62 years old, and the healthy samples were aged 8 to 79 years old; the specific statistical information is shown in the figure; 150 pneumonia samples were selected from the Fortis Hospital, VasantKunj dataset. Since the identity information of the subjects was not disclosed in the dataset, the age and gender could not be counted;

[0068] S3 selects a machine learning algorithm and uses a machine learning method based on sound, video and Transformer methods to complete classifier training:

[0069] S3.1 Preprocessing: Perform operations such as cutting and conversion on the audio signal. There are multiple coughs in an audio signal of cough. In order to solve the problem of small number of samples, the input audio signal is first segmented, and each cough segment is independently converted into a single sample audio signal (such as Figure 1 As shown in the figure, since lung sounds are relatively continuous, no segmentation is performed. The principle of segmentation is: the part with loudness less than -35dB is considered as a silent segment, and the silent segment greater than 500ms is considered as the interval between two coughs. In this way, a cough signal is divided into multiple samples and then converted into a time-frequency graph and input into the network; for each sample audio signal, linear trend removal, windowing and other processing are performed;

[0070] S3.2, audio signal to image signal: audio signal can be converted into two-dimensional forms such as Mel spectrogram, short-time Fourier transform spectrum, MFCC feature map and time-frequency map, so the audio is converted into a time-frequency map at a commonly used sampling rate of 22050Hz for processing. Figure 2 What is shown are waveform diagrams and corresponding time-frequency diagrams of typical cough and lung breath sounds of pneumonia and non-pneumonia in the data set of the present invention;

[0071] S3.3. Build the Transformer machine learning algorithm:

[0072] Convolutional neural networks (CNN) and recurrent neural networks (RNN) are commonly used models in the field of deep learning. However, in many cases, Transformers that fully use attention mechanisms instead of convolution and loops are replacing CNNs and RNNs. For Transformers, CNN operators have the problem of limited local perception field of view. In order to obtain global information, multiple layers need to be stacked, but as the number of layers increases, the amount of information will be exhausted. Therefore, the extracted features are focused on certain areas. Transformers have a self-attention mechanism that can effectively obtain global information, and multi-head attention can map it to multiple spaces, making the model more expressive. Transformers have good modal fusion capabilities. For images, the initial Embeddings obtained by convolution or direct pixel operations on the image can be fed into the Transformer without always maintaining the H×W×C Feature Map structure, similar to Position. Embedding can easily use any information that can be encoded. For transformers, RNN uses machine learning by processing sequential data. When processing data, it will contact the data of the previous moment. Therefore, it is easy to cause gradient explosion or gradient disappearance during back propagation. This is a long-term dependency problem of RNN. Although LSTM can optimize this problem, it will still be prominent when the amount of data is particularly large. Transformer is based on the Attention mechanism and can process multiple different tasks in parallel. Therefore, the Transformer model can be used to process it. Therefore, in terms of model selection, converting audio into pictures and using transformers can reduce system complexity. Therefore, a Transformer-based model is selected to identify pneumonia from audio signals. The improved transformer-based algorithm is used to classify audio. The audio is first mapped to a two-dimensional time-frequency diagram and sent to the network in the form of 1 channel and 224×224. Then, the time-frequency diagram is processed by an attention mechanism based on layered patches to capture the local to global context across network layers. A total of 5 stages are passed, and 1 HST is used in the 1st, 2nd and 4th stages respectively. block, 9 HST blocks are used in the third stage, and finally in the fifth stage, it is determined whether it is in a healthy state. The network structure diagram is as follows Figure 3 As shown;

[0073] The multi-head self-attention module in HST utilizes local window attention, which is inspired by the success of restricted attention models in computer vision tasks. The structure of the HST block is as follows: Figure 4As shown in the figure, it consists of layer normalization (Layer Norm), local windowed multi-head self-attention (LWMSA), multi-layer perceptron (Multi-Layer Perceptron) and residual connection. Layer normalization is to normalize all features of each sample. Local windowed multi-head self-attention is used to extract different features in the sample. Multi-layer perceptron uses hidden layers and activation functions to obtain a nonlinear model. Residual connection is used to solve the problems of network failure to converge to the desired results and network overfitting when the network is deepened.

[0074] When using Transformer to process an audio signal, it is necessary to process the audio signal into multiple fixed-length vectors (such as Figure 5 As shown in the figure, a vector has overlapping information with other vectors) entering the network. When the audio is too long, the network complexity will be increased and the training difficulty will be greatly increased. When the lengths of the audios vary greatly, the parameters need to be set based on the longest audio. If other shorter audios are encountered, resources will be wasted. Finally, the voice is converted into images and processed in the Transformer way to reduce the complexity of the system. The specific flow diagram is shown in the figure. Figure 6 As shown;

[0075] The following is the code and the introduction of the running process:

[0076] The program is run in pytorch 1.13 and python 3.9. The optimizer is SGD (stochastic gradient descent). Compared with non-stochastic algorithms, SGD can use information more effectively, especially when the information is redundant, and SGD has excellent results in the early iterations. The loss function is Cross Entropy, which is a commonly used loss function in classification problems. It is mainly used to measure the difference between two probability distributions. Cross entropy is defined as:

[0077]

[0078] p represents the correct answer, q represents the predicted value, the smaller the cross entropy, the closer the distribution of the two probabilities is; the cross entropy loss is expressed as:

[0079]

[0080] Where is the true label of the j-th sample, is the predicted probability of the j-th sample, and J is the number of samples;

[0081] The HST block uses an improved Transformer block with a local windowed attention mechanism to process the input graph to achieve linear complexity, which is used to efficiently capture contextual features in the spectrogram at multiple scales. The specific code is as follows:

[0082]

[0083]

[0084]

[0085]

[0086]

[0087] #Residual connections

[0088] x=residual2+self.drop_path2(x)

[0089] x=x+self.drop_path2(self.mlp2(self.norm4(x)))

[0090] return x;

[0091] HST model structure: HSTModel is a complete network architecture composed of multiple HST_Layer, which is used for the whole process from original image to final classification;

[0092] 1.HST_Block

[0093] 1.1.HST_Block is the basic building block of HSTModel, responsible for implementing the local window multi-head self-attention mechanism (LWMSA).

[0094] 1.2. Each HST_Block contains two LWMSA modules. The first LWMSA module performs self-attention calculation of features in the local window without any displacement operation. The second LWMSA module applies a cyclic displacement operation on the feature map and then performs self-attention calculation to enhance the feature interaction capability across windows.

[0095] 1.3.HST_Block retains input information through residual connections, thereby enhancing the flow and transmission of information. This structure can effectively extract local features while maintaining stability during model training.

[0096] 2.HST_Layer

[0097] 2.1.HST_Layer is a modular structure composed of multiple HST_Blocks. An HST_Layer contains several HST_Blocks, which are used to process feature extraction tasks at a specific stage.

[0098] 2.2.HST_Layer is not only responsible for local feature extraction, but also ensures the learning and fusion of features of different scales through feature hierarchy transmission between each stage.

[0099] 3.HSTModel

[0100] 3.1.HSTModel is a complete model architecture, which is composed of multiple HST_Layer cascades. The model first divides the input image into multiple non-overlapping image blocks, and each block is processed by multiple HST_Layer to gradually extract multi-scale and multi-semantic features.

[0101] 3.2. As the model is processed layer by layer, the resolution of the feature map continues to decrease, while the semantic information is gradually enriched. Through this hierarchical feature extraction mechanism, HSTModel can effectively capture information of different resolutions and different semantic levels.

[0102] 3.3. After all HST_Layers complete feature extraction, the model uses the global average pooling layer to reduce the dimension of the feature map, and finally outputs the classification result through the classification head;

[0103] The following describes the main assembly and experimental process, and the initial test results are as follows Figure 8 As shown in the figure, although the breathing sound signal can be roughly seen, the noise is too large and the signal-to-noise ratio is low, so it is impossible to clearly distinguish useful information. Such a graph is difficult for the classifier to distinguish signals and cannot be used. Through trial experiments, it is found that the main cause is the noise caused by hand shaking. This noise is large and cannot be avoided, and can only improve the problem of slight hand shaking;

[0104] The vibrations are mitigated by binding a large piece of plastic and securing it with insulating tape. Fig. 9 As shown, it can be found that after being stabilized by plastic, the signal is much higher than the original signal-to-noise ratio, and the breathing sound can also be heard when using the speaker peripheral, but the noise is still very large and the signal-to-noise ratio is not high;

[0105] The improvement was further improved to use the cavity of the mechanical stethoscope and paper to wrap the sensor as a stable bracket. The final test showed that the improvement effect was obvious, the signal-to-noise ratio was significantly increased, and the background noise was greatly reduced ( Fig.10 ). The signal-to-noise ratio can reach more than 10, and the lung sounds can be heard stably and clearly, and the image is clear with one breathing package per second;

[0106] After various tests, we finally found that another major cause of noise was the poor quality of the Vcc line connector connecting the sensor to the amplifier. Later, cutting off a section eliminated a large part of the noise. The final version of the waveform shows that the signal-to-noise ratio reached 22:1, which is comparable to the quality of lung sounds in the database;

[0107] The present invention also provides a process of building an APP software based on the above system, a complete mobile phone software APP developed and designed based on Android Studio, and this APP software is developed using the Android Studio platform to achieve a friendly human-computer interaction interface. Android Studio is a platform developed by Google for writing Android software. The Android system is an open source operating system developed based on the Linux kernel, mainly suitable for mobile devices such as mobile phones and tablets;

[0108] The data collection part can be divided into two parts: cough and lung breath sound auscultation. The process is as follows: first, select the diagnosis method, and then the user fills in the basic information of height and weight; then in the recording step, there are icons on the preparation interface to tell the user how to record. If you are not sure how to operate, you can click Help to view a detailed description. Cough sounds can be collected directly through the mobile phone mic, and lung breath sound auscultation is collected through a self-designed electronic stethoscope; the background then uses socket communication to upload the recorded audio to the cloud for judgment and analysis; finally, after the processing is completed, the server will return the results (whether there is pneumonia) to the mobile phone and display the test results;

[0109] Cough recording interface, lung breath sound recording interface, such as Fig.11 and 12 As shown, users can easily know the position of the phone during recording. If they don't understand, they can click Help, which has a detailed description. The simple loading interface will also be retained during recording to prevent users from thinking that the phone is stuck.

[0110] The data in step S2 are sorted and cut, and 711 cough data (358 pneumonia cough data and 353 non-pneumonia cough data) are sorted and cut: 569 are used as training sets and 142 are used as validation sets. The final program can identify whether pneumonia is present from the cough sound, with an accuracy rate of 82.64%. 97 lung breathing sound data (48 pneumonia breathing sound data and 49 non-pneumonia breathing sound data) are sorted: 78 are used as training sets and 19 are used as validation sets;

[0111] In the following Table 2, TP means the sample is classified as a positive sample and is correct, TN means the sample is classified as a negative sample and is correct, FP means the sample is classified as a positive sample but is incorrect, and FN means the sample is classified as a negative sample but is incorrect. Accuracy is defined as: (TP+TN) / (TP+FN+FP+TN), which is the most commonly used parameter and characterizes the overall accuracy of positive and negative samples; precision is defined as: P=TP / (TP+FP), which mainly characterizes the verification of positive samples. It can be seen that pneumonia infection can be finally identified from cough and lung breath sounds, and the accuracy is good, about 83.33% and 82.64% respectively. Subsequent improvement of accuracy mainly depends on collecting a large number of high-quality data sets. With smaller samples, the result is still acceptable.

[0112] Accuracy Precision TP TN FP FN Lung sounds to detect pneumonia 83.33％ 83.33％ 9 6 1 3 Cough Detection 82.64％ 77.31％ 61 56 14 11

[0113] Table 3 Validation set data Accuracy, Precision, TP, TN, FP, FN

[0114] from Fig.13 The effect of the number of rounds on accuracy can also be seen. The cough sounds tend to be consistent after about 30 rounds, and the lung breath sounds tend to be consistent after about 10 rounds, both of which show that the results of the trainer are reliable;

[0115] Since the algorithm used needs to convert the time series diagram into a time-frequency diagram, the APP program automatically completes this function. Fig.14 for Fig.15 Corresponding time-frequency diagram. It can be seen that there are signals with larger bandwidth (from a few to 10KHz) in coughing sounds, while there are few high-frequency signals in lung breath sounds, which is also consistent with the characteristics of lung breath sounds, indicating that the sampling process of the present invention is basically correct;

[0116] The above results show that a good classifier can be obtained based on the open source database sound data and Transformer machine learning algorithm; the simple high signal-to-noise ratio lung breath sound collector built can also meet the requirements; the overall product has also been tested by many people and completed the full process test of "mobile phone data collection" - "cloud data analysis" - "mobile phone display results". At the same time, the result feedback is accurate and reasonable.

[0117] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent pneumonia prediction system based on cough and lung breath sounds, characterized by: It includes a sound collection module, a data transmission module and a server detection module; The sound collection module includes a cough sound collection unit and a lung breathing sound collection unit, wherein the cough sound collection unit is used to collect cough sounds and send them to the server detection module through the data transmission module for pneumonia prediction; The lung breath sound collection unit is used to collect the user's lung breath sounds and send them to the server detection module through the data transmission module for pneumonia prediction; The server detection module is equipped with a trained Transformer model for converting the audio data transmitted by the data transmission module into images for processing and prediction.

2. The pneumonia intelligent prediction system based on cough and lung breath sounds according to claim 1, characterized in that: The cough sound collection unit collects the sound through a device equipped with a microphone.

3. The pneumonia intelligent prediction system based on cough and lung breath sounds according to claim 1, characterized in that: The lung breath sound collection unit collects the lung breath sounds through a lung breath sound collection device, and the lung breath sound collection device includes a piezoelectric sensor, an amplifier, an analog-to-digital converter, and a power supply.

Images