Respiratory disease diagnosis device and method

Abstract

The invention particularly relates to a respiratory disease diagnosis device and method, and the device comprises a respiratory sound signal collection unit which is used for obtaining a current respiratory sound signal; the breathing sound signal preprocessing unit is used for performing segmentation processing and denoising processing on the current breathing sound signal to obtain a preprocessed breathing sound signal; the feature extraction unit is used for extracting multi-modal features of the preprocessed breath sound signals; and the detection unit is used for inputting the multi-modal features into a preset deep learning network model to obtain a respiratory system disease detection result. Therefore, through multi-modal feature extraction, dynamic fusion and an efficient classification strategy, the problems of relatively low feature identification degree, relatively low disease classification accuracy and the like in the process of researching disease diagnosis based on the breath sound signals in related technologies are solved, and the detection performance of chronic respiratory system diseases based on the breath sound signals is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of biomedical signal processing technology, and in particular to a diagnostic device and method for respiratory diseases. Background Technology

[0002] Breath sounds are an important physiological signal that directly reflects changes in airflow in the respiratory tract and the state of lung tissue. Analysis of breath sounds can aid in the diagnosis of various respiratory diseases.

[0003] In related technologies, models such as Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and self-attention mechanisms (such as Transformer) are used to model respiratory sounds. Typically, the respiratory sound signal is converted into a spectrogram (such as Mel spectrogram, log Mel graph, or continuous wavelet transform graph) as model input to extract effective time-frequency features.

[0004] However, respiratory sound signals exhibit different pattern characteristics at different time scales. The methods in related technologies have limited ability to express features at a single scale, and cannot fully capture disease-related local details and global patterns, thus limiting the model's discriminative ability. Modeling based solely on spectrograms or raw audio as a single modality cannot fully exploit the complementarity between different modal information, affecting diagnostic accuracy and generalization performance. Although deep neural network models in related technologies can extract a large number of high-dimensional features, they suffer from problems such as information redundancy and limited feature expression ability, which can easily lead to overfitting or insufficient feature expression, and these issues urgently need to be addressed. Summary of the Invention

[0005] This application provides a respiratory disease diagnostic device and method to solve the problems of low feature identifiability and low disease classification accuracy in the process of disease diagnosis based on respiratory sound signals in related technologies, and significantly improves the detection performance of chronic respiratory diseases based on respiratory sound signals.

[0006] The first aspect of this application provides a respiratory disease diagnostic device, including: a respiratory sound signal acquisition unit, a respiratory sound signal preprocessing unit, a feature extraction unit, and a detection unit;

[0007] The respiratory sound signal acquisition unit is used to acquire the current respiratory sound signal;

[0008] The respiratory sound signal preprocessing unit is used to perform segmentation and noise reduction processing on the current respiratory sound signal to obtain a preprocessed respiratory sound signal.

[0009] The feature extraction unit is used to extract the multimodal features of the preprocessed respiratory sound signal;

[0010] The detection unit is used to input the multimodal features into a preset deep learning network model to obtain respiratory disease detection results.

[0011] Optionally, the feature extraction unit includes:

[0012] The time series feature calculation module is used to extract feature representations from the time series signal of breath sounds to obtain time series features;

[0013] The Mel spectrum calculation module is used to obtain the Mel spectrum diagram based on the preprocessed breath sound signal using short-time Fourier transform and Mel filter bank.

[0014] The wavelet transform module is used to calculate the wavelet transform time-spectrum diagram of the logarithmic frequency channel of the preprocessed breath sound signal;

[0015] The MFCC extraction module is used to extract MFCC coefficients from the preprocessed breath sound signal to characterize the vocal tract resonance characteristics.

[0016] The feature fusion module is used to extract feature representations from the Mel spectrogram, the wavelet transform time-frequency diagram, and the MFCC coefficients, and fuse them with the time series features to obtain the multimodal features.

[0017] Optionally, it also includes: a deep learning model unit, wherein,

[0018] The deep learning model unit is specifically used for:

[0019] Acquire multimodal features corresponding to multiple target breath sound signals;

[0020] Based on a preset loss function, a preset deep learning network is trained using the multimodal features corresponding to the multiple target respiratory sound signals to obtain the preset deep learning model.

[0021] Optionally, the deep learning model unit further includes: a feature enhancement module and an information bottleneck compression module, wherein,

[0022] The feature enhancement module is used to apply a random masking operation to the multimodal features corresponding to the multiple target breath sound signals to perform data enhancement, thereby obtaining enhanced multimodal features;

[0023] The information bottleneck compression module is used to construct a compressed feature space for the enhanced multimodal features based on the principle of minimizing mutual information, so as to obtain the compressed feature representation.

[0024] Optionally, the detection unit includes: a hierarchical classification module, used to classify the compressed feature representation based on a preset classification model, determine whether the compressed feature representation is in an abnormal state, and identify the disease type of the compressed feature representation when the compressed feature representation is in an abnormal state.

[0025] Optionally, the breath sound signal preprocessing unit is specifically used for:

[0026] The breath sound signal is divided into data segments of preset duration to obtain the segmented breath sound signal;

[0027] The segmented breath sound signal is decomposed using an empirical mode decomposition algorithm to obtain multiple intrinsic mode functions (IMFs). IMF components of a preset frequency band are selected from the multiple IMFs for reconstruction to obtain the reconstructed signal. The reconstructed signal is then converted into a Mel spectrogram.

[0028] Optionally, the formula for the feature fusion module is:

[0029] F i =[TS i Mel i WT i MFCC i ];

[0030] Among them, TS i Mel represents the time series features of the i-th sample. i WT is the Mel spectrum feature representation of the i-th sample. i For the wavelet transform time-frequency plot feature representation of the i-th sample, MFCC i The MFCC coefficient feature representation for the i-th sample.

[0031] A second aspect of this application provides a method for diagnosing respiratory diseases, employing the respiratory disease diagnostic device described in any one of the above-mentioned methods, including the following steps:

[0032] Acquire breath sound signals;

[0033] The respiratory sound signal is segmented and denoised to obtain a preprocessed respiratory sound signal;

[0034] Multimodal features of the preprocessed respiratory sound signal are extracted, and the multimodal features are input into a preset deep learning network model to obtain respiratory disease detection results.

[0035] Optionally, the extraction of multimodal features from the preprocessed breath sound signal includes:

[0036] Feature representations are extracted from the time-series signal of breath sounds to obtain time-series features;

[0037] Based on the short-time Fourier transform and the Mel filter bank, the Mel spectrum is obtained from the preprocessed breath sound signal.

[0038] Calculate the wavelet transform spectrum of the logarithmic frequency channel of the preprocessed respiratory sound signal;

[0039] MFCC coefficients are extracted from the preprocessed breath sound signal to characterize the vocal tract resonance characteristics;

[0040] Feature representations are extracted from the Mel spectrogram, the wavelet transform time-frequency diagram, and the MFCC coefficients, and fused with the time series features to obtain the multimodal features.

[0041] Optionally, the step of inputting the multimodal features into a preset deep learning network model to obtain respiratory disease detection results includes:

[0042] Acquire multimodal features corresponding to multiple target breath sound signals;

[0043] Based on a preset loss function, a preset deep learning network is trained using the multimodal features corresponding to the multiple target respiratory sound signals to obtain the preset deep learning model.

[0044] Optionally, the step of inputting the multimodal features into a preset deep learning network model to obtain respiratory disease detection results includes:

[0045] A random masking operation is applied to the multimodal features corresponding to the multiple target breath sound signals to perform data augmentation, resulting in augmented multimodal features;

[0046] Based on the principle of minimizing mutual information, a compressed feature space is constructed for the enhanced multimodal features to obtain the compressed feature representation.

[0047] Optionally, the step of inputting the multimodal features into a preset deep learning network model to obtain respiratory disease detection results includes:

[0048] The compressed feature representation is classified based on a preset classification model to determine whether the compressed feature representation is in an abnormal state. When the compressed feature representation is in an abnormal state, the disease type of the compressed feature representation is identified.

[0049] Optionally, the step of segmenting and denoising the breath sound signal to obtain a preprocessed breath sound signal includes: segmenting the breath sound signal into data segments of a preset duration to obtain segmented breath sound signals;

[0050] The segmented breath sound signal is decomposed using an empirical mode decomposition algorithm to obtain multiple intrinsic mode functions (IMFs). IMF components of a preset frequency band are selected from the multiple IMFs for reconstruction to obtain the reconstructed signal. The reconstructed signal is then converted into a Mel spectrogram.

[0051] Optionally, feature representations are extracted from the Mel spectrum, the wavelet transform time-frequency plot, and the MFCC coefficients, and fused with the time series features to obtain multimodal features. The calculation formula is as follows:

[0052] F i =[TS i Mel i WT i MFCC i ];

[0053] Among them, TS i Mel represents the time series features of the i-th sample. i WT is the Mel spectrum feature representation of the i-th sample. i For the wavelet transform time-frequency plot feature representation of the i-th sample, MFCC i The MFCC coefficient feature representation for the i-th sample.

[0054] Therefore, the respiratory sound signal acquisition unit acquires the current respiratory sound signal; the respiratory sound signal preprocessing unit performs segmentation and noise reduction on the current respiratory sound signal to obtain the preprocessed respiratory sound signal; the feature extraction unit extracts the multimodal features of the preprocessed respiratory sound signal; and the detection unit inputs the multimodal features into a preset deep learning network model to obtain the respiratory disease detection result. Thus, through multimodal feature extraction, dynamic fusion, and efficient classification strategies, the problems of low feature identifiability and low disease classification accuracy in disease diagnosis based on respiratory sound signals in related technologies are solved, significantly improving the detection performance of chronic respiratory diseases based on respiratory sound signals.

[0055] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0056] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0057] Figure 1 This is a schematic diagram of a respiratory disease diagnostic device provided according to an embodiment of this application;

[0058] Figure 2This is a schematic diagram of the original respiratory sound signal and multi-view transformation spectrum of a respiratory disease diagnosis method according to an embodiment of this application;

[0059] Figure 3 This is a system flowchart of a respiratory disease diagnosis method according to an embodiment of this application;

[0060] Figure 4 This is a schematic diagram of the network structure of the time series feature extraction module of a deep learning model for a respiratory disease diagnosis method according to an embodiment of this application;

[0061] Figure 5 This is a schematic diagram of the network structure of a multi-scale Mel-spectrum feature extraction module of a deep learning model for a respiratory disease diagnosis method according to an embodiment of this application;

[0062] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0063] Figure 7 This is a flowchart of a respiratory disease diagnosis method provided according to an embodiment of this application. Detailed Implementation

[0064] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0065] Before introducing the respiratory disease diagnostic device of the embodiments of this application, let me briefly introduce the respiratory disease diagnostic device in the related art.

[0066] The methods in related technologies have many shortcomings in terms of respiratory sound signal processing, feature extraction, model design and classification performance, including insufficient noise processing ability, single feature extraction, insufficient model generalization ability and insufficient disease detection specificity.

[0067] Specifically, methods in related technologies typically employ simple filtering or noise reduction techniques (such as wavelet transform and low-pass filtering), which are insufficient to effectively remove complex noise from breath sound signals, resulting in poor signal quality and affecting subsequent feature extraction and classification. These methods often employ single feature extraction approaches, failing to comprehensively capture the temporal and frequency domain characteristics of breath sound signals, leading to limited feature representation capabilities and low classification accuracy. Existing technologies lack effective feature fusion mechanisms, typically employing simple concatenation or weighted averaging methods to integrate multimodal features, failing to adaptively adjust the contribution of each modality, thus hindering the model's ability to fully exploit the time-frequency information of breath sound signals. Models in related technologies exhibit poor adaptability to different individuals and environments, struggling to cope with the diversity and variability of breath sound signals, resulting in unstable performance in practical applications. Models lack hierarchical design, making it difficult to simultaneously meet the recognition needs of multiple disease categories. Many models in related technologies focus on general breath sound classification, lacking optimized designs for specific chronic respiratory diseases (such as chronic obstructive pulmonary disease, pneumonia, and asthma), leading to low accuracy and practicality in specific disease detection.

[0068] As a result, the aforementioned shortcomings make it difficult for the methods in related technologies to meet the requirements of practical applications in terms of the accuracy and robustness of the breath sound detection model.

[0069] This application addresses the aforementioned problems by proposing a respiratory disease diagnostic device. The device comprises a respiratory sound signal acquisition unit that acquires the current respiratory sound signal; a respiratory sound signal preprocessing unit that segments and denoises the current respiratory sound signal to obtain a preprocessed respiratory sound signal; a feature extraction unit that extracts multimodal features from the preprocessed respiratory sound signal; and a detection unit that inputs the multimodal features into a pre-defined deep learning network model to obtain respiratory disease detection results. Thus, through multimodal feature extraction, dynamic fusion, and efficient classification strategies, this application solves the problems of low feature identifiability and low disease classification accuracy in disease diagnosis based on respiratory sound signals in related technologies, significantly improving the detection performance of chronic respiratory diseases based on respiratory sound signals.

[0070] Specifically, Figure 1 This is a flowchart illustrating a respiratory disease diagnosis method provided in an embodiment of this application.

[0071] like Figure 1 As shown, the respiratory disease diagnostic device 10 includes: a respiratory sound signal acquisition unit 100, a respiratory sound signal preprocessing unit 200, a feature extraction unit 300, and a detection unit 400.

[0072] Among them, the breath sound signal acquisition unit 100 is used to acquire the current breath sound signal;

[0073] The respiratory sound signal preprocessing unit 200 is used to perform segmentation and noise reduction processing on the current respiratory sound signal to obtain a preprocessed respiratory sound signal.

[0074] The feature extraction unit 300 is used to extract multimodal features of the preprocessed respiratory sound signal;

[0075] The detection unit 400 is used to input multimodal features into a preset deep learning network model to obtain respiratory disease detection results.

[0076] Understandably, the respiratory sound signal acquisition unit 100 acquires respiratory sound signals through a signal acquisition device, stores the acquired signals in a local storage module, and then transmits them to a computer system for processing; or directly transmits the signals to a data processing system for analysis in real time via wired or wireless transmission. The respiratory sound signal preprocessing unit 200 segments the acquired respiratory sound signals, performs denoising on the original respiratory sound signals, improves the signal-to-noise ratio, and retains key pathological features. The feature extraction unit 300 extracts features at different scales and from different perspectives to obtain data for model training. The multi-scale and multi-view deep neural network unit (i.e., the detection unit 400) extracts semantic information from the features at different scales and from different perspectives obtained by the feature extraction unit. Through different deep learning network model designs, it achieves efficient fusion between local and global features, time-domain and frequency-domain features, and features at different levels. Combining multi-scale and multi-view features and using a designed loss function, it improves the model's ability to discriminate complex respiratory system pathological states and obtains respiratory system disease detection results.

[0077] Optionally, the feature extraction unit 300 includes: a time series feature calculation module for extracting feature representations of the breath sound time series signal to obtain time series features; a Mel spectrum calculation module for obtaining a Mel spectrum diagram based on the preprocessed breath sound signal using short-time Fourier transform and Mel filter bank; a wavelet transform module for calculating the wavelet transform time spectrum diagram of the logarithmic frequency channel of the preprocessed breath sound signal; an MFCC extraction module for extracting MFCC coefficients from the preprocessed breath sound signal to characterize the vocal tract resonance characteristics; and a feature fusion module for extracting feature representations from the Mel spectrum diagram, wavelet transform time-frequency diagram, and MFCC coefficients, and fusing them with the time series features to obtain multimodal features.

[0078] Optionally, the formula for the feature fusion module is:

[0079] F i =[TS i Mel i WT i MFCC i ];

[0080] Among them, TS i Mel represents the time series features of the i-th sample. i Let WT be the Mel spectrum feature representation of the i-th sample. i For the wavelet transform time-frequency plot feature representation of the i-th sample, MFCC i Let represent the MFCC coefficient feature of the i-th sample.

[0081] It is understood that the embodiments of this application decompose each preprocessed breath sound signal into three complementary representations using a multimodal decomposition algorithm: Mel spectrogram, wavelet transform time-frequency graph, and MFCC coefficients. These, along with the original time series signal, capture the characteristics of the breath sound signal from different angles. This multi-view representation method is significantly superior to traditional single-view analysis and can comprehensively capture different pathological manifestations of respiratory diseases. Specifically, the wavelet transform time-frequency graph calculates the time spectrum of 36 logarithmic frequency channels, focusing on capturing the pathological frequency band of 200-2000Hz. The MFCC coefficients extract the first 20 cepstral coefficients to characterize the vocal tract resonance characteristics. A single breath sound signal sample is represented as follows:

[0082]

[0083] Among them, X i Let C be the number of channels for the breath sound, and N be the number of acquisition points for each breath sound signal.

[0084] Specifically, the Mel spectrogram, by simulating the human ear's perception of sound frequencies, converts the linear frequency scale to the Mel scale, which is more consistent with human auditory perception. Its calculation formula is as follows:

[0085] m = 2595 × log 10 (1+f / 700);

[0086] Where f is the linear frequency and m is the Mel frequency. In this embodiment, a short-time Fourier transform (STFT) and a Mel filter bank are used to calculate the Mel spectrum for the original breath sound signal X. i Perform a short-time Fourier transform to obtain its frequency domain representation STFT(x). i ′), and calculate its power spectrum |STFT(x′). i ′)| 2 Then, the power spectrum |STFT(Xx′) is processed through the Mel filter bank to obtain the power spectrum. i )| 2The linear frequency f is mapped to the Mel frequency m using the Mel filter bank (MelFilterbank()). A set of triangular filters is designed to cover a preset frequency range (e.g., from 0Hz to half the sampling rate). The power spectrum is convolved with each filter to obtain the output energy. Finally, the logarithm of the output energy is taken to obtain the Mel spectrum M. i ,Right now:

[0087] M i =MelFilterbank(|STFT(Xx′) i )| 2 );

[0088] Among them, M i This represents the Mel spectrum amplitude of the i-th frame signal.

[0089] Furthermore, wavelet transform time-frequency mapping is a multi-resolution time-frequency analysis method that provides better frequency resolution in the low-frequency region and better time resolution in the high-frequency region, making it particularly suitable for analyzing pathological features in breath sound signals. The discrete wavelet transform can be expressed as:

[0090]

[0091] Where ψ is the wavelet basis function, j is the scaling parameter, and k is the translation parameter. In this embodiment, wavelet transform time-frequency plots for 36 logarithmic frequency channels are calculated, with particular focus on the pathological frequency band of 200-2000Hz.

[0092] Furthermore, the MFCC coefficients are used to extract the vocal tract resonance characteristics, which can effectively characterize the structural changes of the airway. The calculation steps include pre-emphasis and windowing of the signal, calculation of the power spectrum, passing the power spectrum through a Mel filter bank, taking the logarithm and performing a discrete cosine transform (DCT), and extracting the first 20 MFCC coefficients to characterize the vocal tract resonance characteristics. The calculation formula is as follows:

[0093] MFCC i =DCT(log(MelFilterbank(|STFT(x i ′)| 2 )));

[0094] Furthermore, in this embodiment, based on the obtained Mel spectrum, wavelet transform time-frequency diagram, and MFCC coefficients, a multi-view feature representation is constructed, and the three representations are fused in a specific manner to form a multi-view feature representation F. i .

[0095] Optionally, the respiratory disease diagnostic device 10 further includes a deep learning model unit, wherein the deep learning model unit is specifically used to: acquire multimodal features corresponding to multiple target respiratory sound signals; and train a preset deep learning network using the multimodal features corresponding to multiple target respiratory sound signals based on a preset loss function to obtain a preset deep learning model.

[0096] Optionally, the deep learning model unit further includes: a feature enhancement module and an information bottleneck compression module. The feature enhancement module is used to perform data enhancement by applying random masking operations to the multimodal features corresponding to multiple target respiratory sound signals to obtain enhanced multimodal features. The information bottleneck compression module is used to construct a compressed feature space for the enhanced multimodal features based on the principle of minimizing mutual information to obtain a compressed feature representation.

[0097] Optionally, the detection unit includes: a hierarchical classification module, used to classify the compressed feature representation based on a preset classification model, determine whether the compressed feature representation is in an abnormal state, and identify the disease type of the compressed feature representation when the compressed feature representation is in an abnormal state.

[0098] It is understandable that a random masking mechanism is applied to F. i Data augmentation is represented as:

[0099] F i ′=F i ⊙M;

[0100] Where ⊙ represents element-wise multiplication, and M is a randomly generated binary mask matrix that randomly masks specific time-frequency units to simulate noise interference in the real environment and improve the robustness of the model.

[0101] The generation of the mask matrix M follows these rules:

[0102] M j,k ~Bernoulli(p);

[0103] Where p is the retention probability, usually set to 0.7-0.9. To expose the model to a sufficiently diverse range of masking patterns during training, a new mask matrix is ​​generated for each training batch. This random masking mechanism has three key advantages: (1) it simulates noise interference in the real world, enhancing the model's robustness to noise; (2) by randomly masking some features, it prevents the model from over-relying on a specific feature, reducing overfitting; (3) it introduces random perturbations, increasing the diversity of training samples, which is equivalent to a data augmentation method. In addition, the information bottleneck theory is applied to constrain the extraction of key features, including: constructing a feature compression loss function based on the information bottleneck theory; determining the optimal feature representation based on the feature compression loss function, so that it minimizes redundant information while retaining information related to disease classification; and constructing a compressed feature space based on the optimal feature representation. Traditional methods in related technologies often contain a large amount of redundant information during feature fusion, while the information bottleneck theory can effectively solve this problem.

[0104] Furthermore, mathematically, this can be expressed as minimizing the following objective function:

[0105] L IB =I(X;Z)-β·I(Z;y);

[0106] Where x represents the original breath sound signal features, z represents the compressed feature representation, Y represents the disease category label, I(X;Z) represents the mutual information between X and Z, I(Z;Y) represents the mutual information between Z and Y, and β is a trade-off parameter that controls the balance between information compression and classification performance.

[0107] The definition of mutual information is:

[0108]

[0109] According to the information bottleneck theory, an ideal feature representation Z should minimize the mutual information I(X;Z) with the original feature X while maximizing the mutual information I(Z;Y) with the target label Y. This mainly includes: minimizing I(X;Z): compressing the original feature X to discard redundant information (such as noise or irrelevant features) unrelated to the target label Y; and maximizing I(Z;Y): ensuring that the compressed feature representation Z retains sufficient information to accurately predict the target label Y. Through this constraint, the feature representation Z can retain key information related to Y while removing redundant information as much as possible, thereby achieving efficient and robust disease diagnosis. By minimizing L... IB The optimal feature representation Z can be obtained. * In practical implementation, to calculate the mutual information term, we use a variational approximation method to construct a parameterized function f representing the encoder, mapping X to Z:

[0110] Z = f(X,θ);

[0111] Where θ represents the encoder parameters. By minimizing L... IB The variational upper bound yields the optimal parameter θ. * This leads to the compressed feature representation Z. * .

[0112] The loss function for the Variational Information Bottleneck (VIB) can be expressed as:

[0113]

[0114] Where q(z|x) is the encoder distribution, p(y|z) is the predictor distribution, r(z) is the prior distribution (usually a standard normal distribution), and KL represents the KL divergence. Used to calculate the expected performance of the model under different distributions.

[0115] In this embodiment of the application, the encoder q(z|x) is defined as a Gaussian distribution:

[0116]

[0117] Where μ(x) and σ 2 (x) represents the mean and variance, respectively, which are calculated by a neural network.

[0118] This feature compression method based on the information bottleneck theory has significant advantages over traditional feature selection or dimensionality reduction methods, including: (1) clear objectives, directly optimizing the retention of information related to the classification task; (2) solid theoretical foundation, supported by information theory; and (3) the ability to adaptively learn the optimal feature representation without relying on manually designed rules.

[0119] Furthermore, a hierarchical classification model is used to diagnose diseases based on the compressed feature representation, obtaining the disease category corresponding to the breath sound signal. In this embodiment, the hierarchical classification model adopts a two-level classification strategy, first distinguishing between healthy and abnormal states, and then further identifying specific disease types (such as chronic obstructive pulmonary disease, healthy, etc.) under abnormal states; the classification model can be represented as:

[0120] P(Y|Z) = softmax(g(Z,φ));

[0121] Where Y is the disease category label, Z is the compressed feature representation, g is the parameterized classification function, φ is the parameter of the classification function, and P(Y|Z) represents the posterior probability of disease category Y given feature Z.

[0122] The hierarchical classification model is specifically implemented as a first-level classification (healthy / abnormal):

[0123] P(Y1|Z)=sigmoid(g1(Z,φ1));

[0124] Where Y1∈{0,1} represents healthy (0) or abnormal (1), and g1 is the first-level classifier.

[0125] Second-level classification (specific disease type):

[0126] P(Y2|Z,Y1=1)=softmax(g2(Z,φ2));

[0127] Here, Y2∈{1,2,...,K} represents K different disease types, and g2 is the second-level classifier, which is only activated when Y1=1 (judged as abnormal). This hierarchical classification strategy, compared to a single classifier, conforms to the thought process of clinical diagnosis, first determining whether an abnormality exists, and then determining the specific disease type; it decomposes complex classification problems into relatively simple sub-problems, improving classification accuracy; and it can flexibly respond to diagnostic needs of different granularities.

[0128] It is worth noting that respiratory sound analysis in related technologies mainly focuses on feature extraction from a single perspective, such as using only spectral features or time-domain statistical features, which cannot comprehensively capture the multidimensional pathological manifestations of respiratory diseases. Although a few studies have attempted multi-perspective analysis, they often employ simple feature concatenation or weighted averaging, failing to effectively integrate complementary information from different perspectives. Furthermore, these methods tend to learn shallow, dataset-specific features, exhibiting poor generalization performance and instability when faced with different acquisition environments and patient groups. Addressing the limitations of single-modal feature representation, simplistic feature fusion methods, severe noise interference, and high feature redundancy in related technologies, this invention uses a multimodal decomposition algorithm to represent respiratory sound signals from three perspectives: spectral, time-frequency, and cepstrum. A masking mechanism is applied to improve feature robustness, and key features are extracted using information bottleneck theory. Finally, a hierarchical classification model is used to achieve accurate disease diagnosis. This solves the problems of low feature identifiability and low disease classification accuracy in disease diagnosis research based on respiratory sound signals in related technologies.

[0129] Therefore, as Figure 2 As shown, Figure 2This diagram illustrates the original respiratory sound signal and multi-view transformation spectrum of a respiratory disease diagnostic method according to an embodiment of this application. It shows the original respiratory sound signal and the multi-view signal used in this embodiment. The preprocessing stage of this invention employs a multi-level noise filtering strategy to effectively suppress irrelevant noise while preserving key pathological features. Signal decomposition and multi-view feature extraction methods (such as Mel spectrograms, wavelet transform, MFCC, empirical mode decomposition, etc.) are combined with random masking and dynamic noise suppression techniques to comprehensively capture the time-domain, frequency-domain, and resonance characteristics of the respiratory sound signal. This enhances the ability to express pathological features while effectively improving the model's robustness to clinical environmental noise, ensuring stability and reliability under different acquisition conditions. By integrating multiple hybrid deep learning models instead of using a single model, features at different scales and from different perspectives can be obtained, thereby improving classification accuracy. Through the synergistic effect of dynamic fusion, information optimization, and hierarchical representation, the problems of low feature utilization and poor generalization in traditional respiratory sound analysis models are solved, providing reliable technical support for the accurate diagnosis of respiratory diseases. Furthermore, regarding the loss function during model training, this invention replaces the traditional cross-entropy loss function with a novel one. This allows the model to focus more on minority class samples during training, thereby addressing the data imbalance problem and improving the model's detection accuracy. Moreover, this invention employs a hierarchical progressive classification strategy. By simulating the diagnostic thought process of a clinician, it decomposes the complex multi-class respiratory disease classification problem into multiple hierarchical sub-tasks, significantly improving the model's classification accuracy and clinical applicability. For example, it first performs a binary classification of COPD (Chronic Obstructive Pulmonary Disease) and healthy individuals, then a ternary classification of COPD, healthy individuals, and other lung diseases, and finally a multi-class classification of COPD, healthy individuals, and various lung diseases such as pneumonia and bronchitis, simulating the clinical diagnostic process and improving accuracy in complex multi-class scenarios.

[0130] Optionally, the breath sound signal preprocessing unit 200 is specifically used for: dividing the breath sound signal into data segments of a preset duration to obtain segmented breath sound signals; decomposing the segmented breath sound signals using an empirical mode decomposition algorithm to obtain multiple intrinsic mode functions; selecting IMF components of a preset frequency band from the multiple intrinsic mode functions for reconstruction to obtain a reconstructed signal; and converting the reconstructed signal into a Mel spectrogram.

[0131] The preset duration can be a threshold set by the user, a threshold obtained through a limited number of experiments, or a threshold obtained through a limited number of computer simulations; no specific limitation is made here.

[0132] Understandably, in the signal preprocessing stage, after segmenting the acquired breath sound signal, the original breath sound signal undergoes denoising to improve the signal-to-noise ratio and preserve key pathological features. Denoising methods include, but are not limited to, Empirical Mode Decomposition (EMD) and reconstruction, wavelet decomposition and reconstruction, or Butterworth filtering. The acquired breath sound signal is segmented into data segments of length Len = Ts (T * fs samples), where T is the number of seconds to be segmented and fs is the sampling rate. A bandpass filter is used to remove environmental noise and artifacts from the breath sound signal. The preset duration can be set according to specific circumstances, for example, from 2 to 10 seconds.

[0133] To facilitate those skilled in the art to further understand the respiratory disease diagnostic device of this application, the respiratory disease diagnostic method of this application is illustrated below through a specific embodiment, including the following steps:

[0134] Breath sound signal acquisition: Multi-channel breath sound signals were acquired using an electronic stethoscope, covering the lungs and trachea of ​​the subject (such as the suprasternal notch and the second intercostal space along the midclavicular line in the anterior chest, and the interscapular region and subscapular region in the posterior back). According to the different disease states of the subjects (healthy, COPD, asthma, pneumonia, etc.), corresponding breath sound signals were collected and labeled with disease information. The signals were then transmitted to the computer for storage using the provided Bluetooth adapter as .wav files, with a sampling frequency of 4KHz.

[0135] Preprocessing: The acquired breath sound signals were segmented into 5-second segments to extract stable feature fragments. Simultaneously, a 100-2000Hz bandpass filter was used for signal preprocessing to effectively remove environmental noise and artifacts, ensuring signal quality and providing a reliable data foundation for subsequent model training and disease diagnosis.

[0136] Feature extraction: Time series feature representation is calculated. The respiratory sound signal of each channel is decomposed into Mel spectrogram, wavelet transform time-frequency plot and MFCC coefficients through a multimodal decomposition algorithm and feature representation is extracted. Mel spectrogram of 40 channels is calculated and wavelet transform time-frequency plot of 36 logarithmic frequency channels is calculated. Special attention is paid to the pathological frequency band of 200-2000Hz. The first 20 MFCC coefficients are extracted to characterize the vocal tract resonance characteristics.

[0137] Deep learning model: Design a feature enhancement algorithm based on masking mechanism to fuse four representations into multimodal features, apply random masks to enhance the data of features and improve the robustness of the model; construct a feature compression loss function based on information bottleneck theory to obtain the optimal feature representation, and minimize redundant features while retaining classification-related information.

[0138] Detection results: During the model training phase, a hierarchical classification model is used to train the compressed feature representation, and disease label information is combined to learn to distinguish between healthy and abnormal states, and further identify specific disease types in abnormal states; During the disease diagnosis phase, new respiratory sound signals are input into the trained model, the compressed feature representation is extracted, and the diagnostic results are output through the hierarchical classification model. The model training requires disease label information as a supervision signal, while the disease diagnosis does not require label information and directly uses the trained model to complete real-time diagnosis.

[0139] Therefore, the multi-view feature analysis of this invention can capture the pathological features of respiratory diseases more comprehensively compared to a single view; the random masking mechanism significantly improves the generalization ability of the model and solves the problem that existing multi-view methods are prone to learning shallow features; the information bottleneck theory constraint effectively solves the feature redundancy problem and improves classification efficiency; the hierarchical classification strategy conforms to clinical diagnostic thinking and can meet the diagnostic needs of different granularities. This invention has achieved significant performance improvement in distinguishing between COPD and healthy individuals. In a study using publicly available ICBHI data, the embodiments of this application achieved an average accuracy of 95.5%, a sensitivity of 94.7%, and a specificity of 96.3% in 5-fold cross-validation, which is significantly better than existing single-view methods (average accuracy of 85.2%) and simple feature fusion multi-view methods (average accuracy of 91.3%). The respiratory disease diagnosis method proposed in this application analyzes respiratory sound signals from different perspectives using a multimodal decomposition algorithm to form complementary feature representations; it enhances feature robustness using a masking mechanism to effectively cope with noise interference in the clinical environment; it achieves feature compression based on the information bottleneck theory, retaining diagnostic-related information while minimizing redundant features; it adopts a hierarchical classification strategy to adapt to the needs of disease classification at different granularities; the algorithm is novel, efficient, and clinically reliable, significantly improving the accuracy and robustness of respiratory disease diagnosis.

[0140] Furthermore, the following is combined with Figure 3 The embodiments shown will be described in detail.

[0141] like Figure 3 As shown, Figure 3 The present invention provides a system flowchart of a respiratory disease diagnosis method according to an embodiment of the present application, which mainly includes a respiratory sound signal acquisition module, a preprocessing module, a feature extraction module, a deep learning model, and a detection result output module.

[0142] Breath sound signal acquisition: Multi-channel breath sound signals were acquired using an electronic stethoscope, covering the lungs and trachea of ​​the subject (such as the suprasternal notch and the second intercostal space along the midclavicular line in the anterior chest, and the interscapular region and subscapular region in the posterior back). According to the different disease states of the subjects (healthy, COPD, asthma, pneumonia, etc.), corresponding breath sound signals were collected and labeled with disease information. The signals were then transmitted to the computer for storage using the provided Bluetooth adapter as .wav files, with a sampling frequency of 4KHz.

[0143] Breath sound signal preprocessing: The acquired breath sound signal is divided into data segments of length Len = Ts (T * fs samples), where T is the number of seconds to be segmented and fs is the sampling rate. A single breath sound signal sample is represented as:

[0144]

[0145] Among them, X i Let C be the number of channels for the breath sound, and N be the number of acquisition points for each breath sound signal.

[0146] Feature extraction: The breath sound signal is decomposed using the Empirical Mode Decomposition (EMD) algorithm to obtain multiple Intrinsic Mode Functions (IMFs). Appropriate IMF components are selected to reconstruct the signal, noise interference is removed, and then the reconstructed signal is converted into a Mel spectrogram. The specific steps are as follows:

[0147] The original breath sound signal x(t) is divided into data segments of length T = 4s (4s in this example):

[0148] x i [k]=x[k+iN], k∈[0,T), i=0, 1, 2...;

[0149] Where, x i [k] represents the data segment of the i-th original breath sound signal, N = fs × T, where fs is the sampling frequency.

[0150] For each data segment x i [k] is subjected to EMD decomposition, resulting in a series of intrinsic mode functions (IMFs):

[0151]

[0152] Among them, c m [k] represents the m-th intrinsic mode function (IMF), r i [k] represents the residual component, and M is the total number of IMFs.

[0153] Based on the characteristics of breath sounds, select L appropriate IMF components for reconstruction:

[0154]

[0155] Where, x′ i [k] represents the reconstructed breath sound signal.

[0156] Specifically, the frequency characteristics of the IMF components are determined using Hilbert transform or fast Fourier transform, and then L IMF components that conform to the breathing frequency range of [100-2000] Hz are selected for reconstruction.

[0157] The reconstructed breath sound signal was converted into a Mel spectrogram:

[0158] S i =MelSpectrogram(x′) i [k]);

[0159] Among them, S i This is the Mel spectrogram converted from the reconstructed breath sound signal. After scaling, the resulting Mel spectrogram is (128, 128) in size, which is convenient for subsequent input into the model.

[0160] Feature extraction is mainly divided into a time-series feature extraction module and a multi-scale Mel-spectrum feature extraction module, extracting features at different scales. This is achieved through a combination of time-domain and frequency-domain multi-scale analysis. The time-series feature extraction module focuses on the dynamic changes of breath sounds, capturing short-term fluctuations and long-term trends in the signal; the multi-scale Mel-spectrum feature extraction module analyzes the spectral characteristics of breath sounds from a frequency domain perspective, revealing the distribution patterns of the signal at different scales and frequency ranges.

[0161] Deep learning models are mainly divided into two parts: a time series extraction module and a multi-scale Mel spectrogram feature extraction module, which are used to extract features from different perspectives and at different scales.

[0162] The first part, the time series feature extraction module, mainly includes two networks: a one-dimensional CNN and a bidirectional LSTM. The specific network structures are as follows: Figure 4 As shown, Figure 4 This is a schematic diagram of the time-series feature extraction module network structure of a deep learning model for a respiratory disease diagnosis method according to an embodiment of this application. The one-dimensional CNN network consists of 6 convolutional layers, each followed by a max-pooling layer. The convolutional kernel size is 3, the stride is 1, and the activation function is ReLU. The bidirectional LSTM also includes two layers, each with 244 neurons. The features output from the one-dimensional CNN and the bidirectional LSTM are scaled by a fully connected layer. Where N is the number of samples. The multi-scale Mel-ray spectral image feature extraction module mainly consists of a two-dimensional CNN and a VisionTransformer, with the specific network structure as follows: Figure 5 As shown, Figure 5 This is a schematic diagram of the network structure of a multi-scale Mel spectrogram feature extraction module in a deep learning model of a respiratory disease diagnosis method according to an embodiment of this application. The two-dimensional CNN specifically includes four convolutional layers, each followed by a batch normalization layer and a max-pooling layer, and finally outputs through a fully connected layer. The number of convolutional kernels is 3, the stride is 1, and the activation function is ReLU to extract local features of the Mel spectrogram. The Vision Transformer is mainly divided into two parts. The first part extracts multi-scale Mel spectrogram features from the two-dimensional CNN network. Specifically, it extracts features after passing through convolutional layers, batch normalization layers, and max-pooling layers, mainly extracting the first three layers. Then, the extracted features are input into the Vision Transformer to obtain multi-scale self-attention mechanism features. The Transformer encoder in the Vision Transformer has one layer, including a multi-head self-attention layer, a first summation & normalization layer, a forward propagation layer, and a second summation & normalization layer. The multi-head self-attention mechanism has two heads, and the calculation formula for the self-attention mechanism is:

[0163]

[0164] Where Q, K, and V represent Query, Key, and Value, respectively. This represents the scaling factor.

[0165] The second part directly extracts the global self-attention mechanism features from the Mel spectrogram. The Vision Transformer has 4 Transformer encoder layers and 2 heads for the multi-head self-attention mechanism. Finally, the features output by the CNN and Vision Transformer are scaled to a size of [size missing] after passing through a fully connected layer. The multi-scale Mel spectrum features of the i-th signal sample.

[0166] It should be noted that the number of layers, dimensions, and number of attention heads in the above network structure can be configured according to actual application requirements; the specific connection method and parameter settings of each module can be adjusted based on the characteristics of the input data; the network configuration given in the embodiments is only an illustrative example and does not constitute a limitation on the scope of protection.

[0167] Furthermore, the dynamic feature fusion and detection module primarily fuses time-series features and multi-scale Mel-spectrum features. The dynamic feature fusion method adaptively adjusts the weights of each modality feature and generates a fused feature representation. The specific calculation formula is as follows:

[0168] F=(σ(w1f1+b1)⊙f1)+((1-(σ(w2f2+b2)⊙f2)));

[0169] Where w1 and w2 are learnable weight matrices; f1 and f2 are two input feature vectors; b1 and b2 are learnable bias vectors; σ is the sigmoid activation function; and F is the fused feature vector.

[0170] During training, data classes are often imbalanced. To address this issue, a focal loss function is used to train the model, resulting in a well-trained multi-scale hybrid deep learning model. This loss function allows the model to focus more on hard-to-classify samples, thus mitigating the negative impact of class imbalance. The focal loss function introduces an adjustable focusing parameter to reduce the loss contribution of easily classified samples while increasing the loss weight of hard-to-classify samples. The specific formula is as follows:

[0171] FL(p t )=-α t (1-p t ) γ log(p t );

[0172] Where, p t It is the model's predicted probability of the true class, α. t γ is the class weight parameter, used to balance the loss contribution of positive and negative samples, and γ is the focusing parameter, used to adjust the loss weight of easy and difficult samples.

[0173] Detection results: The hierarchical classification model is trained on the fused feature representation, learns to distinguish between healthy and abnormal states by combining disease label information, and further identifies specific disease types in abnormal states; in the disease diagnosis stage, new respiratory sound signals are input into the trained model, the compressed feature representation is extracted, and the diagnosis result is output through the hierarchical classification model. The model training requires disease label information as a supervision signal, while the disease diagnosis does not require label information and directly uses the trained model to complete the real-time diagnosis.

[0174] Therefore, this application proposes a multi-scale hybrid deep learning model, which achieves a breakthrough performance improvement in the field of respiratory disease diagnosis through unique architecture design and algorithm optimization. This model effectively solves key problems in breath sound classification, such as incomplete feature representation, large noise interference, and poor model generalization, through an innovative architecture of multimodal fusion, hierarchical feature extraction, and dynamic gating. It provides an efficient and reliable technical solution for intelligent assisted diagnosis of respiratory diseases. This application's embodiments have achieved significant performance improvements in distinguishing between COPD and healthy individuals. In a study using the publicly available ICBHI dataset, this invention achieved an average accuracy of 96.64% for three-class classification and 95.96% for six-class classification in 5-fold cross-validation, significantly outperforming existing single-network model methods (CNN: 91.32% average accuracy for three-class classification and 91.10% for six-class classification; ViT: 90.25% average accuracy for three-class classification and 91.38% for six-class classification).

[0175] Figure 6 This is a schematic diagram illustrating the structure of an electronic device according to an embodiment of the present application. The electronic device may include:

[0176] The memory 601, the processor 602, and the computer program stored on the memory 601 and capable of running on the processor 602.

[0177] When the processor 602 executes the program, it implements the respiratory disease diagnosis method provided in the above embodiments.

[0178] Furthermore, electronic devices also include:

[0179] Communication interface 603 is used for communication between memory 601 and processor 602.

[0180] The memory 601 is used to store computer programs that can run on the processor 602.

[0181] The memory 601 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0182] If the memory 601, processor 602, and communication interface 603 are implemented independently, then the communication interface 603, memory 601, and processor 602 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0183] In a specific implementation, if the memory 601, processor 602, and communication interface 603 are integrated on a single chip, then the memory 601, processor 602, and communication interface 603 can communicate with each other through their internal interfaces.

[0184] The processor 602 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0185] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the respiratory disease diagnostic device described above. This device is primarily used to distinguish between COPD and healthy individuals, but can also be applied to the diagnosis and classification of various respiratory diseases.

[0186] According to the respiratory disease diagnostic device proposed in this application, a respiratory sound signal acquisition unit acquires the current respiratory sound signal; a respiratory sound signal preprocessing unit performs segmentation and noise reduction processing on the current respiratory sound signal to obtain a preprocessed respiratory sound signal; a feature extraction unit extracts multimodal features from the preprocessed respiratory sound signal; and a detection unit inputs the multimodal features into a preset deep learning network model to obtain respiratory disease detection results. Thus, through multimodal feature extraction, dynamic fusion, and efficient classification strategies, the device solves the problems of low feature identifiability and low disease classification accuracy in related technologies based on respiratory sound signals during disease diagnosis, significantly improving the detection performance of chronic respiratory diseases based on respiratory sound signals.

[0187] Next, refer to the attached diagram (i.e.) Figure 7This application describes a respiratory disease diagnostic device proposed according to an embodiment.

[0188] Figure 7 This is a flowchart of a respiratory disease diagnosis method according to an embodiment of this application.

[0189] like Figure 7 As shown, this respiratory disease diagnostic method includes the following steps:

[0190] In step S701, a breath sound signal is acquired.

[0191] In step S702, the breath sound signal is segmented and denoised to obtain a preprocessed breath sound signal.

[0192] Optionally, in some embodiments, the breath sound signal is segmented and denoised to obtain a preprocessed breath sound signal, including: segmenting the breath sound signal into data segments of a preset duration to obtain segmented breath sound signals; decomposing the segmented breath sound signal using an empirical mode decomposition algorithm to obtain multiple intrinsic mode functions; selecting IMF components of a preset frequency band from the multiple intrinsic mode functions for reconstruction to obtain a reconstructed signal; and converting the reconstructed signal into a Mel spectrogram.

[0193] In step S703, the multimodal features of the preprocessed respiratory sound signal are extracted, and the multimodal features are input into a preset deep learning network model to obtain the respiratory disease detection results.

[0194] Optionally, in some embodiments, extracting multimodal features from the preprocessed respiratory sound signal includes: extracting feature representations from the respiratory sound time series signal to obtain time series features; obtaining the Mel spectrum based on the preprocessed respiratory sound signal using short-time Fourier transform and Mel filter bank; calculating the wavelet transform time-frequency spectrum of the logarithmic frequency channel of the preprocessed respiratory sound signal; extracting MFCC coefficients from the preprocessed respiratory sound signal to characterize the vocal tract resonance characteristics; and extracting feature representations from the Mel spectrum, wavelet transform time-frequency spectrum, and MFCC coefficients, and fusing them with the time series features to obtain multimodal features.

[0195] Optionally, in some embodiments, feature representations are extracted from the Mel spectrogram, wavelet transform time-frequency plot, and MFCC coefficients, and fused with time series features to obtain multimodal features. The calculation formula is as follows:

[0196] F i =[TS i Mel i WT i MFCC i ];

[0197] Among them, TSi Mel represents the time series features of the i-th sample. i Let WT be the Mel spectrum feature representation of the i-th sample. i For the wavelet transform time-frequency plot feature representation of the i-th sample, MFCC i Let represent the MFCC coefficient feature of the i-th sample.

[0198] Optionally, in some embodiments, the multimodal features are input into a preset deep learning network model to obtain respiratory disease detection results, including: acquiring multimodal features corresponding to multiple target breath sound signals; and training a preset deep learning network using the multimodal features corresponding to multiple target breath sound signals based on a preset loss function to obtain a preset deep learning model.

[0199] Optionally, in some embodiments, the multimodal features are input into a preset deep learning network model to obtain respiratory disease detection results, including: applying random masking operation to the multimodal features corresponding to multiple target respiratory sound signals for data augmentation to obtain augmented multimodal features; and constructing a compressed feature space for the augmented multimodal features based on the principle of minimizing mutual information to obtain compressed feature representations.

[0200] Optionally, in some embodiments, the multimodal features are input into a preset deep learning network model to obtain respiratory disease detection results, including: classifying the compressed feature representation based on a preset classification model, determining whether the compressed feature representation is in an abnormal state, and identifying the disease type of the compressed feature representation when the compressed feature representation is in an abnormal state.

[0201] It should be noted that the foregoing explanation of the respiratory disease diagnostic device embodiment also applies to the respiratory disease diagnostic method of this embodiment, and will not be repeated here.

[0202] According to the respiratory disease diagnosis method proposed in this application, breath sound signals are acquired, segmented, and denoised to obtain preprocessed breath sound signals. Multimodal features of the preprocessed breath sound signals are extracted, and these features are input into a pre-defined deep learning network model to obtain respiratory disease detection results. Therefore, by employing multimodal feature extraction, dynamic fusion, and efficient classification strategies, this method solves the problems of low feature identifiability and low disease classification accuracy in related technologies based on breath sound signals for disease diagnosis, significantly improving the detection performance of chronic respiratory diseases based on breath sound signals.

[0203] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0204] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0205] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0206] It should be understood that the various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0207] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

Claims

1. A respiratory disease diagnostic device, characterized by, include: The unit includes a breath sound signal acquisition unit, a breath sound signal preprocessing unit, a feature extraction unit, and a detection unit. The respiratory sound signal acquisition unit is used to acquire the current respiratory sound signal; The respiratory sound signal preprocessing unit is used to perform segmentation and noise reduction processing on the current respiratory sound signal to obtain a preprocessed respiratory sound signal. The feature extraction unit is used to extract the multimodal features of the preprocessed respiratory sound signal; The detection unit is used to input the multimodal features into a preset deep learning network model to obtain respiratory disease detection results.

2. The apparatus of claim 1, wherein, The feature extraction unit includes: The time series feature calculation module is used to extract feature representations from the time series signal of breath sounds to obtain time series features; The Mel spectrum calculation module includes a short-time Fourier transform and a Mel filter bank, which obtains the Mel spectrum diagram based on the preprocessed breath sound signal. The wavelet transform module is used to calculate the wavelet transform time-spectrum diagram of the logarithmic frequency channel of the preprocessed breath sound signal; The MFCC extraction module is used to extract MFCC coefficients from the preprocessed breath sound signal to characterize the vocal tract resonance characteristics. The fusion module is used to extract feature representations from the Mel spectrogram, the wavelet transform time-frequency diagram, and the MFCC coefficients, and fuse them with the time series features to obtain the multimodal features.

3. The apparatus of claim 2, wherein, Also includes: Deep learning model unit, where The deep learning model unit is specifically used for: Acquire multimodal features corresponding to multiple target breath sound signals; Based on a preset loss function, a preset deep learning network is trained using the multimodal features corresponding to the multiple target respiratory sound signals to obtain the preset deep learning model.

4. The apparatus of claim 3, wherein, The deep learning model unit further includes: a feature enhancement module and an information bottleneck compression module, wherein, The feature enhancement module is used to apply a random masking operation to the multimodal features corresponding to the multiple target breath sound signals to perform data enhancement, thereby obtaining enhanced multimodal features; The information bottleneck compression module is used to construct a compressed feature space for the enhanced multimodal features based on the principle of minimizing mutual information, so as to obtain the compressed feature representation.

5. The apparatus of claim 1, wherein, The detection unit includes: The hierarchical classification module is used to classify the compressed feature representation based on a preset classification model, determine whether the compressed feature representation is in an abnormal state, and identify the disease type of the compressed feature representation when the compressed feature representation is in an abnormal state.

6. The apparatus of claim 1, wherein, The breath sound signal preprocessing unit is specifically used for: The breath sound signal is divided into data segments of preset duration to obtain the segmented breath sound signal; The segmented breath sound signal is decomposed using an empirical mode decomposition algorithm to obtain multiple intrinsic mode functions (IMFs). IMF components of a preset frequency band are selected from the multiple IMFs for reconstruction to obtain the reconstructed signal.

7. The apparatus of claim 2, wherein, The formula for the feature fusion module is: F i = [TS i ; Mel i ; WT i ; MFCC i ] ; wherein TS i is the time series feature representation of the i-th sample, Mel i is the Mel-spectrogram feature representation of the i-th sample, WT i is the wavelet transform time-frequency map feature representation of the i-th sample, MFCC i is the MFCC coefficient feature representation of the i-th sample.

8. A method for diagnosing respiratory disease, characterized by, Using the respiratory disease diagnostic device as described in any one of claims 1-5, wherein the method comprises the following steps: Acquire breath sound signals; The respiratory sound signal is subjected to segmentation processing and denoising processing to obtain a preprocessed respiratory sound signal; Multi-modal features of the preprocessed respiratory sound signal are extracted, and the multi-modal features are input into a preset deep learning network model to obtain a respiratory system disease detection result.

9. The method of claim 8, wherein, The multi-modal features of the preprocessed respiratory sound signal are extracted, including: a time series feature calculation module configured to extract feature representation from a respiratory sound time series signal to obtain time series features; a Mel spectrogram is obtained from the preprocessed respiratory sound signal based on short-time Fourier transform and a Mel filter bank; a wavelet transform time-frequency spectrogram of a log frequency channel of the preprocessed respiratory sound signal is calculated; MFCC coefficient representing vocal tract resonance characteristics is extracted from the preprocessed respiratory sound signal; feature representation is extracted from the Mel spectrogram, the wavelet transform time-frequency spectrogram and the MFCC coefficient, and is fused with the time series features to obtain the multi-modal features.

10. The method of claim 8, wherein, The multi-modal features are input into a preset deep learning network model to obtain a respiratory system disease detection result, including: multi-modal features corresponding to a plurality of target respiratory sound signals are obtained; the preset deep learning network is trained based on the multi-modal features corresponding to the plurality of target respiratory sound signals to obtain the preset deep learning model.

Images