Respiratory sound accessory sound automatic recognition method based on double-branch attention pooling

Abstract

The application provides a respiratory sound accessory sound automatic identification method based on a double-branch attention pooling, comprising the following steps: converting an acquired one-dimensional audio signal into a two-dimensional logarithmic mel time-frequency spectrum; inputting a serialized time-frequency block vector into a pre-trained audio encoder to extract a global acoustic feature matrix; constructing two parallel physical heuristic attention pooling branches to process the global acoustic feature matrix respectively; inputting a continuous accessory sound feature vector into a first classifier to output a first prediction result of wheezing sound and stridor sound; and inputting a non-continuous accessory sound feature vector into a second classifier to output a second prediction result of fine crackles and coarse crackles. The method can accurately and robustly identify a plurality of different physical characteristic accessory sounds that may coexist, thereby achieving the purpose of providing more comprehensive and accurate digital auscultation assistance for clinical use.

Description

[0001] Cross-reference to related applications This application is a divisional application of application number 202610483060.6 filed on April 13, 2026, entitled "Automatic Recognition Method for Respiratory Sound Supplementary Sounds Based on Multi-Label Deep Learning", which is incorporated herein by reference in its entirety. Technical Field

[0002] This invention belongs to the field of speech recognition technology, and in particular relates to an automatic recognition method for respiratory sounds and additional sounds based on dual-branch attention pooling. Background Technology

[0003] In the initial screening and long-term monitoring of clinical respiratory diseases, auscultation of breath sounds plays an irreplaceable role. Healthy breathing typically produces only steady breath sounds, but when airway inflammation, secretions, spasms, or solid lesions occur, abnormal "additional sounds" are produced. Currently, the most diagnostically valuable additional sounds in clinical practice mainly include wheezing, stridor, and crackles.

[0004] With the advancement of digital audio processing and deep learning technologies, automatic recognition algorithms based on convolutional neural networks (CNN), recurrent neural networks (RNN), or audio transformers have become mainstream. Existing deep learning recognition schemes typically treat breath sound recognition as a "single-label multi-classification" task, that is, presuppose that an audio segment belongs to only one state at a time (e.g., either normal, wheezing, or popping).

[0005] In actual clinical cases, patients' lung lesions are often complex. For example, a patient with chronic obstructive pulmonary disease (COPD) may experience both wheezing from airway spasm and crackles from secretion buildup within the same respiratory cycle. Existing "single-label" modeling methods force the model to choose between multiple coexisting features, which not only leads to interference from conflicting labels during training but also results in serious missed diagnoses during inference, failing to comprehensively reflect the patient's true respiratory condition. Crackles are strictly classified clinically into coarse crackles and fine crackles based on their duration and frequency characteristics. The former is commonly seen in chronic bronchitis, while the latter is a typical sign of interstitial lung diseases such as pulmonary fibrosis. Existing identification methods often categorize them broadly as "crackles," limiting the reference value of the identification results for accurate clinical diagnosis. In other words, existing methods suffer from inaccurate classification of breath sounds. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this invention provides an automatic method for identifying respiratory sound supplementary sounds based on dual-branch attention pooling.

[0007] This disclosure provides an embodiment of an automatic method for identifying respiratory sound embellishments based on dual-branch attention pooling, including: The acquired one-dimensional audio signal is converted into a two-dimensional logarithmic Mel-time spectrum. The two-dimensional log-mel-time spectrogram is divided into blocks, linearly projected, and positionally encoded to obtain a serialized time-frequency block vector. The serialized time-frequency block vector is then input into a pre-trained audio encoder to extract the global acoustic feature matrix. Two parallel physical heuristic attention pooling branches are constructed to process the global acoustic feature matrix respectively. The attention pooling branches include continuous tone branches and non-continuous tone branches. The continuous tone branch uses the Tanh activation function to generate a first attention weight matrix. The first attention weight matrix is ​​used to perform weighted aggregation on the global acoustic feature matrix to obtain the continuous additional tone feature vector. The discontinuous tone branch uses the LeakyReLU activation function to generate a second attention weight matrix. The second attention weight matrix is ​​used to weight and aggregate the global acoustic feature matrix to obtain the discontinuous additional tone feature vector. The continuous additional sound feature vector is input into the first classifier, which outputs the first prediction results for wheezing and panting; the non-continuous additional sound feature vector is input into the second classifier, which outputs the second prediction results for fine crackling and coarse crackling. The first and second classifiers are two parallel multilayer perceptron classifiers; the classifier architecture includes two linear layers. A Dropout regularization layer and a GELU activation function, as detailed below: , Representing the Feature vectors of deep semantics for each time-frequency segment.

[0008] Optionally, the step of dividing the two-dimensional logarithmic Mel-time spectrum into blocks includes: sampling the logarithmic Mel-time spectrum in blocks using a sliding window with overlapping elements, wherein adjacent time-frequency blocks have 6 overlapping pixels on the time axis and frequency axis.

[0009] Optionally, the pre-trained audio encoder is a pre-trained audio Transformer encoder, which is initialized based on weights pre-trained on the AudioSet audio dataset.

[0010] Optionally, the continuous tone branch uses the Tanh activation function to generate a first attention weight matrix, including: Will represent the Feature vectors of deep semantics for each time-frequency segment First, the signal undergoes a linear transformation, then is processed by the Tanh function to smooth the signal, enhancing the continuity pattern and suppressing abrupt noise. Next, another linear transformation is performed. Finally, the processing results of all time-frequency segments are used to calculate the attention weight that segment should receive in representing the continuous additional sounds using the Softmax function. .

[0011] Optionally, the continuous tone branch uses the Tanh activation function, and the calculation process for generating the first attention weight matrix is ​​as follows: .

[0012] Optionally, the discontinuous tone branch uses the LeakyReLU activation function to generate a second attention weight matrix, including: Will represent the Feature vectors of deep semantics for each time-frequency segment First, a linear transformation is performed, followed by processing using the LeakyReLU function to address positive transitions in the signal, enhancing the response to sudden and impulsive characteristics. Then, another linear transformation is performed. Finally, the processing results of all time-frequency segments are used to calculate the attention weight that segment should receive in representing discontinuous additional sounds using the Softmax function. .

[0013] Optionally, the discontinuous tone branch uses the LeakyReLU activation function, and the calculation process for generating the second attention weight matrix is ​​as follows: .

[0014] Optionally, both the first classifier and the second classifier are multilayer perceptrons.

[0015] Optionally, the loss function of the pre-trained audio encoder is: , in, This is the loss value. As a class balance factor, To focus parameters, For the model to the first The confidence probability value of each additional sound.

[0016] Optional, =2.

[0017] The invention provides an automatic identification method for respiratory sound auscultation sounds based on bi-branch attention pooling. This method allows for parallel processing of the same audio signal through bi-branch attention pooling. The continuous tone branch extracts long-range, stable harmonic features such as wheezing and stridor; the discontinuous tone branch focuses on capturing instantaneous, sudden pulse features such as fine / coarse crackles. This effectively overcomes the feature confusion and missed diagnosis problems caused by the forced "either / or" selection in existing single-label models when multiple sounds coexist. Furthermore, it can accurately and robustly identify multiple auscultation sounds with different physical characteristics that may coexist, thus providing more comprehensive and accurate digital auscultation assistance for clinical practice. Attached Figure Description

[0018] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0019] Figure 1 This is a diagram of the DualAttentionAST model architecture in the automatic recognition method for respiratory sounds and additional sounds based on dual-branch attention pooling provided in the embodiments of this disclosure. Figure 2 ROC curve of the DualAttentionAST model provided in the embodiments of this disclosure. Detailed Implementation

[0020] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0021] It should be understood that the following specific examples illustrate the implementation of this disclosure, and those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0022] It should be noted that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice method. Furthermore, this device and / or practice method can be implemented using other structures and / or functionalities besides one or more of the aspects set forth herein.

[0023] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The illustrations only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0024] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0025] This embodiment discloses an automatic method for identifying respiratory sound embellishments based on dual-branch attention pooling, including: The acquired one-dimensional audio signal is converted into a two-dimensional logarithmic Mel-time spectrum. The two-dimensional log-mel-time spectrogram is divided into blocks, linearly projected, and positionally encoded to obtain a serialized time-frequency block vector. The serialized time-frequency block vector is then input into a pre-trained audio encoder to extract the global acoustic feature matrix. Two parallel physical heuristic attention pooling branches are constructed to process the global acoustic feature matrix respectively. The attention pooling branches include continuous tone branches and non-continuous tone branches. The continuous tone branch uses the Tanh activation function to generate a first attention weight matrix. The first attention weight matrix is ​​used to perform weighted aggregation on the global acoustic feature matrix to obtain the continuous additional tone feature vector. The discontinuous tone branch uses the LeakyReLU activation function to generate a second attention weight matrix. The second attention weight matrix is ​​used to weight and aggregate the global acoustic feature matrix to obtain the discontinuous additional tone feature vector. The continuous additional sound feature vector is input into the first classifier, which outputs the first prediction results of wheezing and panting sounds; the non-continuous additional sound feature vector is input into the second classifier, which outputs the second prediction results of fine crackling sounds and coarse crackling sounds.

[0026] In specific application scenarios, the details are as follows: 1. Clinical data collection and standardized distribution: Data was collected from Beijing Jishuitan Hospital. High-sensitivity electronic stethoscopes were used for data acquisition, covering real cases from the respiratory and emergency departments. A total of 1233 valid respiratory cycle samples were ultimately included, involving 123 independent patients and encompassing various lung diseases such as pneumonia, chronic obstructive pulmonary disease, and asthma. In this embodiment, a strict patient-individual-based randomization strategy was implemented for each individual dataset during model construction. To eliminate the model's physical "memory" of specific patients' thoracic anatomy, subcutaneous fat thickness, and skin viscosity, and to ensure that the model learns universal respiratory acoustic pathological features, this embodiment ensured complete patient isolation between the training set (746 audio samples) and the test set (487 audio samples). That is, any patient and their breath sound sample appearing in the test set did not participate in the model's training process. This partitioning mechanism effectively evaluates the model's generalization performance and diagnostic robustness when facing entirely new clinical individuals. When processing the labels, each sample corresponds to a four-dimensional Boolean vector Y=[y1, y2, y3, y4], which represents whether it contains wheezing, panting, fine crackling, and coarse crackling, respectively.

[0027] 2. Audio signal processing: In terms of audio preprocessing, this embodiment converts the one-dimensional audio signal into a two-dimensional Mel-spectrum and performs feature extraction. The specific operation steps are as follows: (1) Signal resampling and normalization: Resample the original signal to the target frequency. To avoid distortion, a multiphase filtering resampling algorithm is used, which is mathematically expressed as: , in For resampling operators, The original input discrete audio signal. This is the resampled signal. The signal amplitude is then normalized to... Eliminate sample amplitude differences: , (2) Frame division and windowing: Convert the frame length and frame shift in the time dimension to the sampling point dimension: , , Calculated , ,in The frame length is the dimension of the sampling points. For the sampling point dimension frame shift, For the time dimension, frame length Frame shift in the time dimension; The formula for extracting the k-th frame signal is: ,in The time-domain signal vector of the k-th frame Hamming window is used to suppress spectral leakage; the window function is defined as follows: The windowing operation is as follows: , , (3) Short-time Fourier transform: The windowed frame is converted to the frequency domain using FFT, and the complex spectrum and power spectrum are calculated as follows: , , in, This represents the complex spectrum value at the m-th frequency point in the k-th frame. This represents the power spectral density value at the m-th frequency point in the k-th frame; The formula for converting frequency index to physical frequency is: Only the first one is retained. A single frequency point can fully characterize the frequency domain energy.

[0028] (4) Mel frequency conversion: Using the Mel frequency scale to match the nonlinear auditory characteristics of the human ear, the conversion relationship is as follows: , Construct a trigonometric Mel filter bank, the response of the p-th filter. for: , Energy value of the p-th Mel channel in frame k The result is obtained by summing the power spectrum and the filter response: .

[0029] (5) Logarithmic compression: Logarithmic compression of the Mel energy reduces the dynamic range while ensuring numerical stability. .

[0030] (6) Global normalization: Global normalization of log-Mel features improves the model's generalization ability. , in It is the global mean of the Mel logarithmic energy. It is the global standard deviation of the Mel logarithmic energy; Stacking normalized features of all frames The final two-dimensional time-frequency feature matrix is ​​obtained. : .

[0031] 3. DualAttention AST model architecture: This embodiment combines a pre-trained audio Transformer backbone with a physics-inspired dual-branch attention mechanism. The entire process is as follows: Figure 1 As shown, the specific steps are as follows: (1) Time-frequency serialization block segmentation: The model first discretizes the input two-dimensional log-Mel spectrum into blocks. Specifically, a 16×16 sliding window is used as the sampling unit, and its sliding step size on both the time and frequency axes is uniformly set to 10.

[0032] This setup ensures a physical overlap of 6 pixels between adjacent time-frequency patches. Compared to regular non-overlapping patches, this overlapping sampling mechanism effectively captures transient acoustic features (such as bursts of popping sounds) that cross patch edges, guaranteeing the continuity and integrity of time-frequency information.

[0033] (2) Linear projection and position coding: Each 16×16 time-frequency block is mapped to a 768-dimensional sequence vector via a linear projection layer. Subsequently, to compensate for the model architecture's insufficient ability to perceive the spatial location of the sequence, learnable two-dimensional positional codes are added to these vectors, enabling the model to identify the relative positions of different acoustic events in time and frequency.

[0034] (3) Feature evolution based on pre-trained weights: The aforementioned sequence vectors containing location information are input into a multi-layer self-attention architecture. For model initialization, weight parameters are pre-trained on the large-scale general-purpose audio dataset AudioSet. AudioSet is a massive acoustic ontology library containing over 2 million audio segments and covering more than 500 audio categories. Its data distribution is extremely broad, encompassing a large number of complex acoustic patterns in nature and human activities (such as harmonics, non-stationary impulses, and background environmental noise).

[0035] The rationale for choosing this pre-trained weight for downstream fine-tuning in this embodiment lies in the fact that, in terms of underlying physical features, the additional sounds of breath sounds have a high degree of signal similarity to "continuous rhythms" and "sudden impulses" in the general acoustic domain. By introducing an encoder trained on a large-scale corpus, the model already possesses powerful general feature extraction capabilities. Based on this, fine-tuning for breath sound-specific tasks enables the model to quickly migrate from the general acoustic space to the medical clinical auscultation space. Even when clinical labeled data is limited, it can still achieve high-precision capture of complex pathological acoustic patterns through knowledge transfer.

[0036] After undergoing nonlinear transformation by a deep Transformer encoder, the model ultimately outputs a global feature matrix. (where N is the total sequence length of the time-frequency block generated after overlapping cutting), providing a high-dimensional semantic representation for the subsequent dual-branch attention mechanism.

[0037] (4) Physics-inspired dual-branch attention pooling: Branch A: Continuous sound attention: For wheezing and panting sounds with smooth spectral lines and long-range stability, the Tanh activation function is used to map the hidden layer features to [-1, 1]. Its saturation properties act like a "low-pass filter," suppressing unstable impulses and locking in long-range signals. The attention weights for continuous additional sounds are obtained using the following formula. : , The continuous additional sound representation vector is obtained by weight aggregation. .

[0038] Branch B: Attention to discontinuous sounds: This implementation targets fine / coarse popping sounds with wideband pulse and transient burst characteristics. The LeakyReLU function is employed, whose nonlinear characteristics are extremely sensitive to positive gradients (signal jumps), enabling it to capture energy spikes within very short timeframes. The attention weights for discontinuous additional sounds are obtained using the following formula. : , The non-continuous additional sound representation vector is obtained by weight aggregation. .

[0039] (5) Hierarchical decoupling classification head: Two parallel multilayer perceptron (MLP) classifiers were designed at the end of the model: Classifier 1: Input It outputs a two-dimensional vector specifically for distinguishing between wheezing and whimpering sounds.

[0040] Classifier 2: Input It outputs a two-dimensional vector specifically for distinguishing between fine and coarse popping sounds.

[0041] The two classifiers have the same architecture, and the formula is as follows: , , in, and These represent the prediction results for continuous and non-continuous additional sounds, respectively.

[0042] This design, through "inter-task association learning," enables the model to improve the recognition performance of non-continuous additional sounds when it learns to easily recognize continuous additional sounds.

[0043] (6) Optimization algorithm and loss function design: To address the issue of "more normal samples and fewer abnormal samples" in medical data, this embodiment employs an improved FocalLoss loss function: , Setting the focus parameter γ=2.0 allows the model to assign greater weight to difficult samples with prediction errors during training, forcing the model to continuously refine the boundary during iteration.

[0044] The optimizer uses AdamW and the learning rate is set to [value missing]. By using decoupled weight decay, we can prevent the AST model with a large number of parameters from overfitting severely on small-scale medical datasets.

[0045] 4. Verification results: The DualAttentionAST model proposed in this embodiment was validated on a real breath sound dataset from Beijing Jishuitan Hospital. To comprehensively evaluate the effectiveness of the multi-label classification system, the following metrics were used for performance measurement: Micro-AUC: Measures the model's ability to distinguish across all label categories as a whole.

[0046] Hamming Accuracy: Measures the average degree of consistency between multi-label predictions and the true labels.

[0047] Specificity measures the accuracy of a model in excluding negative samples (i.e., identifying "normal" or "absence of the murmur"), and is a core indicator for reducing false alarm rates in clinical auxiliary diagnosis.

[0048] Accuracy: The accuracy of classification for each specific additional sound category.

[0049] In the overall performance evaluation of the multi-label classification task, the system demonstrated excellent recognition accuracy and robustness, with an overall Micro-AUC of 0.9009. This high level of performance fully demonstrates the model's strong feature extraction and class separation capabilities when dealing with complex time-frequency spaces where multiple pathological murmurs coexist. Meanwhile, the system's Hamming Accuracy reached 0.9045, indicating that the model's judgments at the vast majority of predicted sites are highly consistent with the gold standard of experienced clinical experts.

[0050] In terms of specific category performance, the model maintained an extremely high level of recognition for each sub-category of additional sounds. Specifically, the accuracy for wheezing was 84.60%, with a high discrimination index (AUC) of 0.8737. For clinically significant wheezing and crackles, the model demonstrated extremely high sensitivity, achieving recognition accuracies of 98.77% and 98.15% respectively, with both maintaining AUCs consistently above 0.84. The accuracy for fine crackles also reached 80.29%.

[0051] The data above demonstrates that this embodiment not only exhibits extremely high specificity, effectively suppressing false alarms caused by environmental noise, but also demonstrates excellent generalization performance when faced with novel clinical individuals. This embodiment's physical-based decoupled modeling scheme serves as a robust digital auscultation tool, assisting medical personnel in accurately locating high-risk pathological features from massive amounts of respiratory signals, providing solid data support for early clinical warning and triage optimization.

[0052] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0053] In this disclosure, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The block diagrams of devices, apparatuses, devices, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as "comprising," "including," "having," etc., are open-ended terms meaning "including but not limited to," and are used interchangeably with them. The terms "or" and "and" as used herein refer to the terms "and / or," and are used interchangeably with them unless the context clearly indicates otherwise. The term "such as" as used herein refers to the phrase "such as but not limited to," and is used interchangeably with it.

[0054] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0055] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0056] Various changes, substitutions, and modifications can be made to the techniques described herein without departing from the teachings defined in this embodiment. Furthermore, the scope of this embodiment is not limited to the specific aspects of the processes, machines, manufacturing processes, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing processes, events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein can be utilized. Therefore, this embodiment includes such processes, machines, manufacturing processes, events, means, methods, or actions within its scope.

[0057] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0058] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for automatic recognition of respiratory sound embellishments based on bi-branch attention pooling, characterized in that, include: The acquired one-dimensional audio signal is converted into a two-dimensional logarithmic Mel-time spectrum. The two-dimensional log-mel-time spectrogram is divided into blocks, linearly projected, and positionally encoded to obtain a serialized time-frequency block vector. The serialized time-frequency block vector is then input into a pre-trained audio encoder to extract the global acoustic feature matrix. Two parallel physical heuristic attention pooling branches are constructed to process the global acoustic feature matrix respectively. The attention pooling branches include continuous tone branches and non-continuous tone branches. The continuous tone branch uses the Tanh activation function to generate a first attention weight matrix. The first attention weight matrix is ​​used to perform weighted aggregation on the global acoustic feature matrix to obtain the continuous additional tone feature vector. The discontinuous tone branch uses the LeakyReLU activation function to generate a second attention weight matrix. The second attention weight matrix is ​​used to weight and aggregate the global acoustic feature matrix to obtain the discontinuous additional tone feature vector. The continuous additional sound feature vector is input into the first classifier, and the first prediction result of wheezing and panting is output; the non-continuous additional sound feature vector is input into the second classifier, and the second prediction result of fine crackling and coarse crackling is output. The first and second classifiers are two parallel multilayer perceptron classifiers; the classifier architecture includes two linear layers. A Dropout regularization layer and a GELU activation function, as detailed below: , Representing the Feature vectors of deep semantics for each time-frequency segment.

2. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 1, characterized in that, The step of dividing the two-dimensional logarithmic Mel-time spectrum into blocks includes: sampling the logarithmic Mel-time spectrum in blocks using a sliding window with overlaps, wherein the overlap of adjacent time-frequency blocks on the time axis and frequency axis is 6 pixels.

3. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 1, characterized in that, The pre-trained audio encoder is a pre-trained audio Transformer encoder, which is initialized based on weights pre-trained on the AudioSet audio dataset.

4. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 1, characterized in that, The continuous tone branch uses the Tanh activation function to generate a first attention weight matrix, including: Will represent the Feature vectors of deep semantics for each time-frequency segment First, the signal undergoes a linear transformation, then is processed by the Tanh function to smooth the signal, enhancing the continuity pattern and suppressing abrupt noise. Next, another linear transformation is performed. Finally, the processing results of all time-frequency segments are used to calculate the attention weight that segment should receive in representing the continuous additional sounds using the Softmax function. .

5. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 4, characterized in that, The continuous tone branch uses the Tanh activation function, and the calculation process for generating the first attention weight matrix is ​​as follows: 。 6. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 1, characterized in that, The discontinuous tone branch uses the LeakyReLU activation function to generate a second attention weight matrix, including: Will represent the Feature vectors of deep semantics for each time-frequency segment First, a linear transformation is performed, followed by processing using the LeakyReLU function to address positive transitions in the signal, enhancing the response to sudden and impulsive characteristics. Then, another linear transformation is performed. Finally, the processing results of all time-frequency segments are used to calculate the attention weight that segment should receive in representing discontinuous additional sounds using the Softmax function. .

7. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 6, characterized in that, The non-continuous tone branch uses the LeakyReLU activation function, and the calculation process for generating the second attention weight matrix is ​​as follows: 。 8. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 1, characterized in that, Both the first classifier and the second classifier are multilayer perceptrons.

9. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 1, characterized in that, The loss function of the pre-trained audio encoder is: ， in, The loss value. As a category balance factor, To focus parameters, For the model to the first The confidence probability value of each additional sound.

10. The method for automatic recognition of respiratory sound supplementary sounds based on dual-branch attention pooling according to claim 9, characterized in that, =2。

Images