Multi-feature fusion enhanced feature extraction method and system based on breath sound signal

Abstract

The invention discloses a breath sound signal-based multi-feature fusion enhanced feature extraction method and system, and relates to the field of machine learning. The problems that according to a breathing sound signal modeling method in the prior art, due to the fact that average processing and dimension reduction splicing are conducted on different features in the time dimension, time sequence information loss is likely to be caused, the complementary advantages of the features cannot be effectively exerted due to the fact that the multiple features are fused in a simple splicing mode, and the recognition capacity of a model for breathing sound dynamic changes is limited are solved. The method comprises the following steps: acquiring original breath sound signal data, and preprocessing; extracting time-frequency characteristics of the preprocessed original breath sound signal data; a dual-channel synchronous random masking mechanism is designed, and random masking of time and frequency dimensions is carried out on the static features and the dynamic features at the same time; and constructing an adaptive feature fusion and enhancement module, and inputting the fused and enhanced features into a backbone network of the pre-trained AST to complete feature extraction of multi-feature fusion and enhancement based on the breath sound signals.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, specifically to a feature extraction method and system based on multi-feature fusion enhancement of respiratory sound signals. Background Technology

[0002] Faced with the multi-layered challenges posed by respiratory diseases, computed tomography (CT) and chest X-rays are commonly used methods for diagnosing respiratory illnesses. While these techniques provide detailed anatomical information, their application is often limited by equipment availability and the potential health risks of radiation. Therefore, auscultation, as a more economical, portable, and non-invasive diagnostic method, is gaining increasing attention. For breath sound data acquired with a stethoscope, effective preprocessing and feature extraction methods are needed to extract key time-frequency information, providing a reliable input foundation for subsequent disease classification models.

[0003] Although existing research has attempted to model respiratory sound signals using multiple features, certain limitations remain. For example, some methods average different features over time to reduce dimensionality and then concatenate them, which can easily lead to the loss of temporal information. In addition, simply concatenating multiple features fails to effectively leverage their complementary advantages and limits the model's ability to recognize dynamic changes in respiratory sounds.

[0004] To this end, this study uses two features of the same spatial size in audio, namely the Log-Mel spectrum and its first-order difference feature (Delta), and designs a fusion enhancement module that can effectively fuse static and dynamic feature information while maintaining spatial alignment. Summary of the Invention

[0005] This invention addresses the problems of existing respiratory sound signal modeling methods that use averaging and dimensionality reduction techniques to concatenate different features over time, which can easily lead to the loss of temporal information, and the use of simple concatenation to fuse multiple features, which fails to effectively leverage their complementary advantages and limits the model's ability to recognize dynamic changes in respiratory sounds.

[0006] To solve the above-mentioned technical problems, the present invention is achieved through the following technical solution: Option 1: This invention proposes a feature extraction method based on multi-feature fusion enhancement of breath sound signals, the method comprising the following steps: Step 1: Acquire raw breath sound signal data and preprocess the raw breath sound signal data; Step 2: Extract the time-frequency features of the preprocessed raw breath sound signal data, including static features and dynamic features; Step 3: Based on the time-frequency features extracted in Step 2, the SpecAugment method is used to design a dual-channel synchronous random masking mechanism to simultaneously perform random masking of static and dynamic features in the time and frequency dimensions. Step 4: Construct an adaptive feature fusion and enhancement module for the static and dynamic features, and input the fused and enhanced features into the backbone network of the pre-trained AST to complete the feature extraction based on multi-feature fusion and enhancement of respiratory sound signals.

[0007] Furthermore, a preferred embodiment is provided, wherein the method for preprocessing the raw breath sound signal data in step 1 is as follows: Step 1.1: Perform frame segmentation on the raw breath sound signal data using a fixed time window to generate segments of 6 seconds in length, and discard any remaining segments that are less than 6 seconds long. Step 1.2: Resample the original breath sound signal data to 16kHz; Step 1.3: Use a frequency domain filtering method based on discrete Fourier transform to remove baseline drift. Select 1Hz as the cutoff frequency and perform normalization to ensure that the amplitude of all audio segments is uniform within the range of [-1,1].

[0008] Furthermore, a preferred embodiment is provided, wherein the method for extracting the time-frequency features of the preprocessed raw breath sound signal data in step 2 is as follows: Step 2.1: Convert each 6-second time segment into a time-frequency representation using the short-time Fourier transform method; Step 2.2: Map the linear frequency axis using a Mel scale, use a 64-Melb filter bank to perform a weighted summation of the STFT power spectrum, take the natural logarithm of the resulting Mel spectrum to form a 64×188 log-Mel feature matrix, and use the sliding window difference method in the time dimension to calculate the rate of change of the log-Mel feature matrix between adjacent frames; generate a Delta feature matrix with the same dimensions as the log-Mel feature matrix to supplement the dynamic information of the audio signal; Step 2.3: Concatenate the log-Mel feature matrix and the Delta feature matrix along the channel dimension to form a 2×64×188 dual-channel tensor, which is used to synchronize the input of the enhancement and fusion module in step 3.

[0009] Furthermore, a preferred embodiment is provided, wherein step 3 includes the following steps: Step 3.1: Set the maximum width of the time mask. Maximum width of frequency masking Set the synchronization masking trigger probability P; Step 3.2: Randomly sample the time window length on the channels of the Log-Mel feature matrix. (0, ) and start frame ,exist[ Set the interval to zero, and adjust the random sampling frequency window length. (0, ) and starting frequency band ,exist[ Set the interval to zero; Step 3.3: For each main feature masking, the probability P is used to determine whether to synchronously mask the channels of the Delta feature matrix. If not triggered, the original channel values ​​of the Delta special matrix are retained to compensate for the missing information of the Log-Mel special matrix. If synchronous masking is triggered, the channels of the Delta special matrix with the same time frame and frequency range on the Log-Mel special matrix are also set to zero.

[0010] Furthermore, a preferred embodiment is provided, wherein the method for constructing the adaptive feature fusion and enhancement module in step 4 is as follows: Step 4.1: Design a global perception modulation fusion module, and denote the two original input features as follows: and The channel is expanded to C=e dimensions using 1×1 convolutions, and each channel is independently convolved using depthwise convolutions with groups=e to learn the spatial features of each channel. Pointwise convolutions are used to mix the features extracted from each channel by depthwise convolutions, thus constructing a depthwise separable convolutional structure. Global average pooling is used to compress information, generating two weight coefficients, which are then weighted and summed over the two inputs to obtain the desired result. and ; Step 4.2: Design a multi-scale attention time-frequency aggregation module. This module extracts features of different receptive field sizes through convolutions with varying dilation rates, comprising four branches. These four branches are concatenated into two groups, and attention mechanisms are applied to each group of features. The processed two groups of features are then concatenated along the channel dimension to form a more unified feature map. Asymmetric convolution kernels of 1×k and k×1 are used, expanding along the time axis to capture temporal variation patterns and along the frequency axis to learn local correlations between frequency components. Finally, a 1×1 convolution is used to fuse all channels to generate a single, representative output, denoted as . The output of the adaptive feature fusion and enhancement module is then adaptively weighted and residually concatenated with two features of the original signal to obtain the final signal. This is used as input to the pre-trained model AST.

[0011] Option 2: A feature extraction system based on multi-feature fusion enhancement of breath sound signals, the system comprising: The data processing module is used to acquire raw breath sound signal data and preprocess the raw breath sound signal data. The time-frequency feature extraction module is used to extract the time-frequency features of the preprocessed raw breath sound signal data, wherein the time-frequency features include static features and dynamic features; The masking module is used to randomly mask the static and dynamic features based on the time-frequency features extracted by the time-frequency feature extraction module. It adopts the SpecAugment method and designs a dual-channel synchronous random masking mechanism to randomly mask the static and dynamic features in both time and frequency dimensions. The feature extraction module for multi-feature fusion and enhancement is used to construct an adaptive feature fusion and enhancement module for the static and dynamic features, and input the fused and enhanced features into the backbone network of the pre-trained AST to complete the feature extraction based on multi-feature fusion and enhancement of respiratory sound signals.

[0012] Option 3: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Option 1.

[0013] Option 4: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in Option 1.

[0014] The advantages of this invention are: This invention proposes a feature extraction method based on multi-feature fusion and enhancement of respiratory sound signals. Combining static features (Log-Mel spectrum) and dynamic features (Delta), a globally perceptive modulation fusion (PAFM) module and a multi-scale attention time-frequency aggregation (MSAA) module are designed to achieve adaptive feature fusion and multi-scale enhancement. Combined with a pre-trained audio spectrogram Transformer (AST) backbone network, the accuracy and robustness of respiratory disease classification are effectively improved.

[0015] This invention is also applicable to the field of audio multi-feature research. Attached Figure Description

[0016] Figure 1 This is an overall architecture diagram of the feature extraction method based on multi-feature fusion enhancement of respiratory sound signals described in Implementation Method 1.

[0017] Figure 2 This is a schematic diagram of the dual-channel masking enhancement described in Implementation Method 1. Figure 3 This is a schematic diagram of the global sensing modulation and fusion module structure described in Implementation Method 1.

[0018] Figure 4 This is a schematic diagram of the multi-scale attention time-frequency aggregation module structure described in Implementation Method 1.

[0019] Figure 5 This is a schematic diagram of the deep learning confusion matrix described in Implementation Method 1. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0021] Implementation Method 1: This implementation method proposes a feature extraction method based on multi-feature fusion enhancement of breath sound signals. The method includes the following steps: Step 1: Acquire raw breath sound signal data and preprocess the raw breath sound signal data; Step 2: Extract the time-frequency features of the preprocessed raw breath sound signal data, including static features and dynamic features; Step 3: Based on the time-frequency features extracted in Step 2, the SpecAugment method is used to design a dual-channel synchronous random masking mechanism to simultaneously perform random masking of static and dynamic features in the time and frequency dimensions. Step 4: Construct an adaptive feature fusion and enhancement module for the static and dynamic features, and input the fused and enhanced features into the backbone network of the pre-trained AST to complete the feature extraction based on multi-feature fusion and enhancement of respiratory sound signals.

[0022] Implementation Method Two: This implementation method further defines the feature extraction method based on multi-feature fusion enhancement of respiratory sound signals described in Implementation Method One. The method for preprocessing the original respiratory sound signal data in step 1 is as follows: Step 1.1: Perform frame segmentation on the raw breath sound signal data using a fixed time window to generate segments of 6 seconds in length, and discard any remaining segments that are less than 6 seconds long. Step 1.2: Resample the original breath sound signal data to 16kHz; Step 1.3: Use a frequency domain filtering method based on discrete Fourier transform to remove baseline drift. Select 1Hz as the cutoff frequency and perform normalization to ensure that the amplitude of all audio segments is uniform within the range of [-1,1].

[0023] Implementation Method 3: This implementation method further defines the feature extraction method based on multi-feature fusion enhancement of respiratory sound signals described in Implementation Method 1. The method for extracting the time-frequency features of the preprocessed original respiratory sound signal data in step 2 is as follows: Step 2.1: Convert each 6-second time segment into a time-frequency representation using the short-time Fourier transform method; Step 2.2: Map the linear frequency axis using a Mel scale, use a 64-Melb filter bank to perform a weighted summation of the STFT power spectrum, take the natural logarithm of the resulting Mel spectrum to form a 64×188 log-Mel feature matrix, and use the sliding window difference method in the time dimension to calculate the rate of change of the log-Mel feature matrix between adjacent frames; generate a Delta feature matrix with the same dimensions as the log-Mel feature matrix to supplement the dynamic information of the audio signal; Step 2.3: Concatenate the log-Mel feature matrix and the Delta feature matrix along the channel dimension to form a 2×64×188 dual-channel tensor, which is used to synchronize the input of the enhancement and fusion module in step 3.

[0024] Implementation Method Four: This implementation method further defines the feature extraction method based on multi-feature fusion enhancement of breath sound signals described in Implementation Method One. Step 3 includes the following steps: Step 3.1: Set the maximum width of the time mask. Maximum width of frequency masking Set the synchronization masking trigger probability P; Step 3.2: Randomly sample the time window length on the channels of the Log-Mel feature matrix. (0, ) and start frame ,exist[ Set the interval to zero, and adjust the random sampling frequency window length. (0, ) and starting frequency band ,exist[ Set the interval to zero; Step 3.3: For each main feature masking, the probability P is used to determine whether to synchronously mask the channels of the Delta feature matrix. If not triggered, the original channel values ​​of the Delta special matrix are retained to compensate for the missing information of the Log-Mel special matrix. If synchronous masking is triggered, the channels of the Delta special matrix with the same time frame and frequency range on the Log-Mel special matrix are also set to zero.

[0025] Implementation Method 5: This implementation method further defines the feature extraction method based on multi-feature fusion enhancement of respiratory sound signals described in Implementation Method 3. The method for constructing the adaptive feature fusion and enhancement module in step 4 is as follows: Step 4.1: Design a global perception modulation fusion module, and denote the two original input features as follows: and The channel is expanded to C=e dimensions using 1×1 convolutions, and each channel is independently convolved using depthwise convolutions with groups=e to learn the spatial features of each channel. Pointwise convolutions are used to mix the features extracted from each channel by depthwise convolutions, thus constructing a depthwise separable convolutional structure. Global average pooling is used to compress information, generating two weight coefficients, which are then weighted and summed over the two inputs to obtain the desired result. and ; Step 4.2: Design a multi-scale attention time-frequency aggregation module. This module extracts features of different receptive field sizes through convolutions with varying dilation rates, comprising four branches. These four branches are concatenated into two groups, and attention mechanisms are applied to each group of features. The processed two groups of features are then concatenated along the channel dimension to form a more unified feature map. Asymmetric convolution kernels of 1×k and k×1 are used, expanding along the time axis to capture temporal variation patterns and along the frequency axis to learn local correlations between frequency components. Finally, a 1×1 convolution is used to fuse all channels to generate a single, representative output, denoted as . The output of the adaptive feature fusion and enhancement module is then adaptively weighted and residually concatenated with two features of the original signal to obtain the final signal. This is used as input to the pre-trained model AST.

[0026] Implementation Method Six: This implementation method proposes a feature extraction system based on multi-feature fusion enhancement of breath sound signals. The system includes: The data processing module is used to acquire raw breath sound signal data and preprocess the raw breath sound signal data. The time-frequency feature extraction module is used to extract the time-frequency features of the preprocessed raw breath sound signal data, wherein the time-frequency features include static features and dynamic features; The masking module is used to randomly mask the static and dynamic features based on the time-frequency features extracted by the time-frequency feature extraction module. It adopts the SpecAugment method and designs a dual-channel synchronous random masking mechanism to randomly mask the static and dynamic features in both time and frequency dimensions. The feature extraction module for multi-feature fusion and enhancement is used to construct an adaptive feature fusion and enhancement module for the static and dynamic features, and input the fused and enhanced features into the backbone network of the pre-trained AST to complete the feature extraction based on multi-feature fusion and enhancement of respiratory sound signals.

[0027] Implementation Method Seven: This implementation method provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in any one of Implementation Methods One to Five. Implementation Method 8: This implementation method provides a computer device, including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the method described in any one of Implementation Methods 1 to 5.

[0028] Implementation Method Nine: This implementation method proposes an example, which is used to explain the above-described implementation methods one through eight. The specific example is as follows: See Figures 1 to 5 This embodiment illustrates that the purpose of this invention is to propose a feature extraction method based on multi-feature fusion and enhancement of respiratory sound signals. For example... Figure 1 As shown, this invention achieves efficient fusion and refined feature extraction of respiratory sound signals by simultaneously enhancing and adaptively fusing the static spectra and dynamic differential features of respiratory sounds, and by designing a multi-scale hierarchical attention enhancement module and using a pre-trained AST as the backbone network, thus significantly improving classification accuracy and model robustness.

[0029] To achieve the above objectives, the present invention adopts the following technical solution: Step 1: Acquire data and preprocess it to standardize data length and reduce noise.

[0030] Step 2: Extract two types of time-frequency features (static and dynamic) from the data, ensuring that the spatial dimensions of the two features are consistent, thus forming a dual-channel input.

[0031] Step 3: Based on the SpecAugment method, design a dual-channel synchronous random masking mechanism to simultaneously perform random masking of static and dynamic features in both time and frequency dimensions.

[0032] Step 4: Construct a feature fusion and enhancement module for the two types of features, and input the fused and enhanced features into the backbone network based on the pre-trained AST.

[0033] Furthermore, step 1 includes the following steps: Step 1.1: Frame segmentation is performed on the original breath sound signal using a fixed time window to generate segments of 6 seconds in length, and any remaining segments shorter than 6 seconds are discarded. Based on the typical periodic characteristics of the breath signal, to avoid information loss due to possible respiratory cycle truncation at the window boundaries, a 50% overlap strategy (i.e., 3-second overlap) is adopted to ensure smooth transitions and feature continuity between adjacent segments.

[0034] Step 1.2: Resample the original breath sound signal to 16kHz.

[0035] Step 1.3: Baseline drift is removed using a frequency domain filtering method based on Discrete Fourier Transform (DET), with 1Hz selected as the cutoff frequency. Normalization is then performed to ensure that the amplitude of all audio segments is uniformly within the range [-1, 1]. Furthermore, step 2 includes the following steps: Step 2.1: Convert each 6-second time segment into a time-frequency representation using Short Time Fourier Transform (STFT).

[0036] Step 2.2: Map the linear frequency axis using a Mel scale and perform a weighted summation of the STFT power spectrum using a 64-Melb filter bank. Take the natural logarithm of the resulting Mel spectrum to form a 64×188 log-Mel feature matrix. Calculate the rate of change of the log-Mel feature between adjacent frames using a sliding window differencing method in the time dimension. Generate a Delta feature matrix with the same dimensions as the log-Mel matrix to supplement the dynamic information of the audio signal.

[0037] Step 2.3: Concatenate the log-Mel and Delta features along the channel dimension to form a 2×64×188 dual-channel tensor, which will serve as the input for the subsequent synchronous enhancement and fusion module.

[0038] Furthermore, step 3 includes the following steps: Step 3.1: Set the maximum width of the time mask. Maximum width of frequency masking Set the synchronization masking trigger probability P.

[0039] Step 3.2: Randomly sample the time window length on the Log-Mel channel. (0, ) and start frame ,exist[ Set the interval to zero. Random sampling frequency window length. (0, ) and starting frequency band ,exist[ Set the interval to zero.

[0040] Step 3.3: For each main feature masking, determine whether to simultaneously mask the Delta channel with probability P. If not triggered, retain the original Delta channel value for the model to compensate for missing Log-Mel information; if simultaneous masking is triggered, set the Delta channel to zero for the same time frame and frequency range in Log-Mel. The masking effect is as follows: Figure 2 As shown.

[0041] Furthermore, step 4 includes the following steps: Step 4.1, as follows Figure 3 As shown, a globally perceptive modulation fusion (PAMF) module is designed. We denote the two features as follows: and The module expands the channels to C=e (expansion) dimensions using 1×1 convolutions, then performs independent convolutions on each channel using depthwise convolutions with groups=e to learn the spatial features of each channel. Finally, it uses pointwise convolutions to blend the features extracted from the depthwise convolutions, thus constructing a depthwise separable convolutional structure. The module performs independent depthwise separable convolutions on two input features to obtain... and We expand the number of channels of the two enhanced features to e (expansion) to enhance the expressive power of the original feature information; compared with standard convolution, this operation has significant advantages in terms of parameter quantity and computational cost. Then, we concatenate the two enhanced features to 2e channels, forming a joint feature space. We then perform Global Average Pooling (GAP) on the joint features to compress the entire joint feature map space to 1×1, retaining only channel-level global information. Finally, we use a 1×1 convolution to reduce the number of channels from 2e to e. This reduces the number of parameters and computational cost, extracts higher-order features between channels, and introduces non-linearity using the ReLU activation function, enabling the model to learn more complex feature representations. Then, a 1×1 convolution is used again to reduce the number of channels to 2, generating corresponding weight coefficients for the two input features, and the Softmax function is used to normalize the weight coefficients to ( ). , The weights are calculated such that their sum equals 1, thus determining the relative importance of each feature in the fusion process. Based on the obtained weight coefficients, the enhanced Log-Mel spectral features and Delta features are weighted and fused to obtain new fused features. This adaptive feature selection mechanism learns global information from two input features and dynamically adjusts their fusion ratio based on the input features, thereby making more efficient use of complementary information between features and improving model performance.

[0042] Step 4.2, as follows Figure 4As shown, a Multi-Scale Attention Time-Frequency Aggregation (MSAA) module is designed to enhance feature representation capabilities. Features of different receptive field sizes are extracted through convolutions with varying dilation rates: four branches are used, namely 1×1 convolutions and 3×3 convolutions with dilation rates of 1, 3, and 5. Each branch is followed by BatchNorm+GELU to maintain numerical stability and non-linear representation. The four branches are concatenated into two groups: the 1×1 convolution branch and the convolution branch with a dilation rate of 1 form the first group, which are then concatenated along the channels to obtain the desired result. Focusing on fine, small local features; the remaining two branches form the second group, which are then spliced ​​along the channel to obtain... We focus on broad feature information. Then, we apply two computationally efficient and structurally simple attention mechanisms to each of these two feature sets. For the first set of features, which represents more local details, the feature types represented by its channel dimensions are more critical, so we apply channel attention. First, we compress the spatial information of each channel into a global descriptor using GAP. Then, we use a lightweight multilayer perceptron (MLP) consisting of two 1×1 convolutions to learn the non-linear dependencies between channels, ultimately generating a channel importance weight. This weight is then compared with... Multiply to get This process recalibrates the features, amplifies the responses of key local feature channels, and suppresses secondary channels. For the second set of features, which represents more macroscopic information, we employ spatial attention. This is achieved through a minimal 1×1 convolution... The channels are compressed to 1, information from all channels is fused at each spatial location, and a two-dimensional spatial weight mask is generated using a sigmoid function. Regions with weights close to 1 in the mask have more important spatial information, while regions close to 0 have weaker spatial features. This weight is then combined with... Multiply to get This allows for effective focusing of spatial features. and The features are concatenated along the channel dimension to form a more unified feature map. Then, 1×k and k×1 asymmetric convolutional kernels are applied sequentially, expanding along the time axis to capture temporal variation patterns and along the frequency axis to learn local correlations between frequency components, respectively. Finally, a 1×1 convolution is used to fuse all channels to generate a single, representative output, denoted as […]. We then adaptively weighted the residuals of the module's output and the two features of the original signal to obtain... This is used as input to the pre-trained model AST.

[0043] Step 4.3: Use the fused and enhanced features as input to the AST model. AST's global modeling capability is more likely to capture features across the entire audio segment, thus providing strong support for the breath sound classification task. A model pre-trained on ImageNet is used, and this fine-tuned model is used as the baseline model. The AST model trained on a large dataset exhibits better generalization ability in the breath sound classification task.

[0044] Referring to Tables 1 and 2, the performance of different audio features was compared, and the Log-Mel feature was used as the benchmark feature to achieve the best performance.

[0045] Table 1 Comparison results of different features

[0046] Table 2 Comparison results between this application and existing methods

[0047] Table 1 shows the extraction of different time-frequency features from the preprocessed breath sound data, and compares their performance. The best performance was achieved when the Log-Mel feature was used as the baseline feature.

[0048] Table 2 compares the method described in this invention with commonly used methods in the prior art, including mainstream models such as CNN and RNN, as well as methods that fuse multiple features (Mel features, MFCC features, etc.). In summary, the method of this invention performs excellently in performance evaluation, achieving an average accuracy of 96.24%, specificity of 98.7%, precision of 94.11%, recall of 96.42%, and F1 score of 95.10% for four types of subjects.

[0049] This implementation first extracts static spectrograms and dynamic differential features from the breath sound signal. The PAFM module adaptively adjusts the fusion ratio of the two types of features using global perception information, fully leveraging their complementary advantages. Subsequently, the MSAA module extracts multi-scale features using dilated convolution and adaptively weights local and global features using channel and spatial attention mechanisms, enhancing key information and suppressing interference. Finally, the enhanced features are input into a pre-trained AST backbone network, achieving efficient fusion and refined feature extraction, significantly improving model performance.

[0050] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or technical solutions of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0051] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A feature extraction method based on multi-feature fusion enhancement of breath sound signals, characterized in that, The method includes the following steps: Step 1: Acquire raw breath sound signal data and preprocess the raw breath sound signal data; Step 2: Extract the time-frequency features of the preprocessed raw breath sound signal data, including static features and dynamic features; Step 3: Based on the time-frequency features extracted in Step 2, the SpecAugment method is used to design a dual-channel synchronous random masking mechanism to simultaneously perform random masking of static and dynamic features in the time and frequency dimensions. Step 4: Construct an adaptive feature fusion and enhancement module for the static and dynamic features, and input the fused and enhanced features into the backbone network of the pre-trained AST to complete the feature extraction based on multi-feature fusion and enhancement of respiratory sound signals.

2. The feature extraction method based on multi-feature fusion enhancement of respiratory sound signals according to claim 1, characterized in that, The method for preprocessing the raw breath sound signal data in step 1 is as follows: Step 1.1: Perform frame segmentation on the raw breath sound signal data using a fixed time window to generate segments of 6 seconds in length, and discard any remaining segments that are less than 6 seconds long. Step 1.2: Resample the original breath sound signal data to 16kHz; Step 1.3: Use a frequency domain filtering method based on discrete Fourier transform to remove baseline drift. Select 1 Hz as the cutoff frequency and perform normalization processing to ensure that the amplitude of all audio segments is uniform within the range of [-1,1].

3. The feature extraction method based on multi-feature fusion enhancement of respiratory sound signals according to claim 1, characterized in that, The method for extracting the time-frequency features of the preprocessed raw breath sound signal data in step 2 is as follows: Step 2.1: Convert each 6-second time segment into a time-frequency representation using the short-time Fourier transform method; Step 2.2: Map the linear frequency axis using a Mel scale, use a 64-Melb filter bank to perform a weighted summation of the STFT power spectrum, take the natural logarithm of the resulting Mel spectrum to form a 64×188 log-Mel feature matrix, and use the sliding window difference method in the time dimension to calculate the rate of change of the log-Mel feature matrix between adjacent frames; generate a Delta feature matrix with the same dimensions as the log-Mel feature matrix to supplement the dynamic information of the audio signal; Step 2.3: Concatenate the log-Mel feature matrix and the Delta feature matrix along the channel dimension to form a 2×64×188 dual-channel tensor, which is used to synchronize the input of the enhancement and fusion module in step 3.

4. The feature extraction method based on multi-feature fusion enhancement of breath sound signals according to claim 1, characterized in that, Step 3 includes the following steps: Step 3.1: Set the maximum width of the time mask. Maximum width of frequency masking Set the synchronization masking trigger probability P; Step 3.2: Randomly sample the time window length on the channels of the Log-Mel feature matrix. (0, ) and start frame ,exist[ Set the interval to zero, and adjust the random sampling frequency window length. (0, ) and starting frequency band ,exist[ Set the interval to zero; Step 3.3: For each main feature masking, the probability P is used to determine whether to synchronously mask the channels of the Delta feature matrix. If not triggered, the original channel values ​​of the Delta special matrix are retained to compensate for the missing information of the Log-Mel special matrix. If synchronous masking is triggered, the channels of the Delta special matrix with the same time frame and frequency range on the Log-Mel special matrix are also set to zero.

5. The feature extraction method based on multi-feature fusion enhancement of respiratory sound signals according to claim 1, characterized in that, The method for constructing the adaptive feature fusion and enhancement module in step 4 is as follows: Step 4.1: Design a global perception modulation fusion module, and denote the two original input features as follows: and The channel is expanded to C=e dimensions using 1×1 convolutions, and each channel is independently convolved using depthwise convolutions with groups=e to learn the spatial features of each channel. Pointwise convolutions are used to mix the features extracted from each channel by depthwise convolutions, thus constructing a depthwise separable convolutional structure. Global average pooling is used to compress information, generating two weight coefficients, which are then weighted and summed over the two inputs to obtain the desired result. and ; Step 4.2: Design a multi-scale attention time-frequency aggregation module. This module extracts features of different receptive field sizes through convolutions with varying dilation rates, comprising four branches. These four branches are concatenated into two groups, and attention mechanisms are applied to each group of features. The processed two groups of features are then concatenated along the channel dimension to form a more unified feature map. Asymmetric convolution kernels of 1×k and k×1 are used, expanding along the time axis to capture temporal variation patterns and along the frequency axis to learn local correlations between frequency components. Next, a 1×1 convolution is used to fuse all channels to generate a single-channel output that is the most representative. The output of the adaptive feature fusion and enhancement module is then adaptively weighted and residually concatenated with two features of the original signal to obtain the final result. This is used as input to the pre-trained model AST.

6. A feature extraction system based on multi-feature fusion enhancement of breath sound signals, characterized in that, The system includes: The data processing module is used to acquire raw breath sound signal data and preprocess the raw breath sound signal data. The time-frequency feature extraction module is used to extract the time-frequency features of the preprocessed raw breath sound signal data, wherein the time-frequency features include static features and dynamic features; The masking module is used to randomly mask the static and dynamic features based on the time-frequency features extracted by the time-frequency feature extraction module. It adopts the SpecAugment method and designs a dual-channel synchronous random masking mechanism to randomly mask the static and dynamic features in both time and frequency dimensions. The feature extraction module for multi-feature fusion and enhancement is used to construct an adaptive feature fusion and enhancement module for the static and dynamic features, and input the fused and enhanced features into the backbone network of the pre-trained AST to complete the feature extraction based on multi-feature fusion and enhancement of respiratory sound signals.

7. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-5.

8. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-5.

Images