Feature extraction method and system based on multi-feature fusion enhancement of respiratory sound signals

Abstract

The application relates to a multi-feature fusion enhanced feature extraction method and system based on respiratory sound signals, and relates to the field of machine learning. The method solves the problems that the modeling method of the prior art respiratory sound signal averages and reduces the dimension of different features in the time dimension, easily leads to the loss of time sequence information, adopts a simple splicing mode to fuse multiple features, cannot effectively play the complementary advantages, and limits the recognition ability of the model to the dynamic changes of the respiratory sound, etc. The method comprises the following steps: acquiring original respiratory sound signal data and carrying out pretreatment; extracting the time-frequency features of the pretreated original respiratory sound signal data; designing a double-channel synchronous random masking mechanism to simultaneously perform random masking in the time and frequency dimensions on the static features and the dynamic features; constructing an adaptive feature fusion and enhancement module, and inputting the fused and enhanced features into the backbone network of a pre-trained AST to complete the multi-feature fusion and enhancement of the features based on the respiratory sound signals.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a multi-feature fusion enhanced feature extraction method and system based on respiratory sound signals. BACKGROUND

[0002] In the face of the multi-level challenges brought by respiratory diseases, computed tomography (CT) and chest X-ray are commonly used methods for diagnosing respiratory diseases. Although these technologies can provide detailed anatomical structure information, their application is often limited by equipment constraints and potential health risks of radiation. Therefore, auscultation technology as a more economical, portable and non-invasive diagnostic method is attracting more and more attention. For the respiratory sound data collected by the stethoscope, effective preprocessing and feature extraction methods are needed to extract its key time-frequency information to provide a reliable input basis for subsequent disease classification models.

[0003] Although there have been attempts to model respiratory sound signals through multiple features, there are still certain limitations. For example, some methods average and reduce the dimension of different features in the time dimension, which can easily lead to the loss of time sequence information. In addition, the simple splicing method used to fuse multiple features cannot effectively play their complementary advantages, limiting the model's ability to recognize dynamic changes in respiratory sounds.

[0004] Therefore, this study uses two spatially identical features of audio, namely Log-Mel spectrogram and its first-order differential feature (Delta), and designs a fusion enhancement module that can effectively fuse static and dynamic feature information while maintaining spatial alignment. SUMMARY

[0005] The present application is to solve the problems of the existing respiratory sound signal modeling method, which averages and reduces the dimension of different features in the time dimension, which can easily lead to the loss of time sequence information, and the simple splicing method used to fuse multiple features cannot effectively play their complementary advantages, limiting the model's ability to recognize dynamic changes in respiratory sounds.

[0006] To solve the above technical problems, the present application is realized by the following technical scheme:

[0007] Scheme one, the present application proposes a multi-feature fusion enhanced feature extraction method based on respiratory sound signals, the method comprises the following steps:

[0008] Step 1, obtaining the original respiratory sound signal data, and preprocessing the original respiratory sound signal data;

[0009] Step 2, extracting the time-frequency features of the preprocessed original respiratory sound signal data, the time-frequency features include static features and dynamic features;

[0010] Step 3, based on the time-frequency features extracted in step 2, a double-channel synchronous random masking mechanism is designed using the SpecAugment method to randomly mask the static and dynamic features in time and frequency dimensions;

[0011] Step 4, an adaptive feature fusion and enhancement module is constructed for the static and dynamic features, and the fused and enhanced features are input into the backbone network of the pre-trained AST to complete the multi-feature fusion and enhancement based on the respiratory sound signal feature extraction.

[0012] Further, a preferred embodiment is provided, wherein the method for preprocessing the original respiratory sound signal data in step 1 is:

[0013] Step 1.1, frame segmentation is performed on the original respiratory sound signal data using a fixed time window to generate segments with a length of 6 seconds, and the remaining segments less than 6 seconds are discarded;

[0014] Step 1.2, resample the original respiratory sound signal data to 16 kHz;

[0015] Step 1.3, use the frequency domain filtering method based on discrete Fourier transform to remove baseline drift, select 1 Hz as the cutoff frequency, and perform normalization to ensure that the amplitudes of all audio segments are within the range of [-1, 1].

[0016] Further, a preferred embodiment is provided, wherein the method for extracting the time-frequency features of the preprocessed original respiratory sound signal data in step 2 is:

[0017] Step 2.1, convert each 6-second time segment to a time-frequency representation using the short-time Fourier transform method;

[0018] Step 2.2, map the linear frequency axis through the Mel scale, use 64 Mel filter banks, perform weighted summation on the STFT power spectrum, take the natural logarithm of the obtained Mel spectrogram, form a 64x188 log-Mel feature matrix, and use the sliding window difference method in the time dimension to calculate the change rate of the log-Mel feature matrix between adjacent frames; generate a Delta feature matrix with the same dimension as the log-Mel feature matrix to supplement the dynamic information of the audio signal;

[0019] Step 2.3, concatenate the log-Mel feature matrix and the Delta feature matrix in the channel dimension to form a 2x64x188 double-channel tensor, which is used as the input of the enhancement and fusion module in step 3.

[0020] Further, a preferred embodiment is provided, wherein step 3 includes the following steps:

[0021] Step 3.1, set the maximum width of time masking , set the maximum width of frequency masking , set the probability P of synchronous masking trigger;

[0022] Step 3.2, randomly sample the length of the time window on the channel of the Log-Mel feature matrix (0, ) and the starting frame , set to zero in the interval , randomly sample the length of the frequency window (0, ) and the starting frequency band , set to zero in the interval ;

[0023] Step 3.3, for each main feature masking, the probability P is used to determine whether to mask the channel of the Delta feature matrix synchronously, if not triggered, the original value of the channel of the Delta special matrix is retained, which is used to compensate for the missing information of the Log-Mel special matrix; if synchronous masking is triggered, the same time frame and frequency interval on the Log-Mel special matrix is also set to zero in the channel of the Delta special matrix.

[0024] Further, a preferred embodiment is provided, and the method for constructing an adaptive feature fusion and enhancement module in step 4 is:

[0025] Step 4.1, design a global perception modulation fusion module, let the two original input features be and , expand the channel to C=e dimensions through 1×1 convolution, and independently convolve each channel through deep convolution with groups=e, learn the spatial features of each channel, use point-by-point convolution to mix the features of each channel extracted by deep convolution, and build a depth separable convolution structure, use global average pooling to compress information, generate two weight coefficients, and get and ;

[0026] Step 4.2, design a multi-scale attention time-frequency aggregation module, extract features of different receptive field sizes through convolution with different dilation rates, including four branches, concatenate the four branches into two groups, and perform attention mechanism operation on the two groups of features respectively; concatenate the processed two groups of features along the channel dimension to form a more unified feature map; and use asymmetric convolution kernels of 1×k and k×1 to respectively expand along the time axis to capture the timing change pattern; and expand along the frequency axis to learn the local correlation between frequency components; secondly, a 1×1 convolution is used to fuse all channels to generate a single channel output that is most representative, denoted as Adaptive feature fusion and enhancement module output is adaptively weighted and connected with the residual connection of the two features of the original signal to obtain , as the input of the pre-training model AST.

[0027] Scheme two, a multi-feature fusion and enhancement feature extraction system based on respiratory sound signals, the system comprises:

[0028] A data processing module is configured to acquire original respiratory sound signal data and pre-process the original respiratory sound signal data.

[0029] A time-frequency feature extraction module is configured to extract time-frequency features of the pre-processed original respiratory sound signal data, wherein the time-frequency features include static features and dynamic features.

[0030] A masking module is configured to design a double-channel synchronous random masking mechanism based on the time-frequency features extracted by the time-frequency feature extraction module using the SpecAugment method, and simultaneously perform random masking in the time and frequency dimensions on the static features and dynamic features.

[0031] A multi-feature fusion and enhancement feature extraction module is configured to construct an adaptive feature fusion and enhancement module for the static features and dynamic features, and input the fused and enhanced features into the backbone network of the pre-training AST to complete multi-feature fusion and enhancement feature extraction based on respiratory sound signals.

[0032] Scheme three, a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by the processor to realize the steps of the method of scheme one.

[0033] Scheme four, a computer device, comprising a memory and a processor, the memory stores a computer program, when the processor runs the computer program stored in the memory, the processor executes the method of scheme one.

[0034] The present application has the advantages of:

[0035] The present application proposes a multi-feature fusion and enhancement feature extraction method based on respiratory sound signals, which combines static features (Log-Mel spectrum) and dynamic features (Delta), designs a global perception modulation fusion (PAFM) module and a multi-scale attention time-frequency aggregation (MSAA) module, and realizes adaptive feature fusion and multi-scale enhancement. Combined with the pre-training audio spectrum graph Transformer (AST) backbone network, the accuracy and robustness of the respiratory system disease classification are effectively improved.

[0036] The present application is also applicable to the field of audio multi-feature research. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The overall architecture diagram of the multi-feature fusion enhanced feature extraction method based on respiratory sound signals according to embodiment one.

[0038] Figure 2 The dual-channel masking enhancement schematic diagram according to embodiment one.

[0039] Figure 3 The global perception modulation fusion module structure schematic diagram according to embodiment one.

[0040] Figure 4 The multi-scale attention time-frequency aggregation module structure schematic diagram according to embodiment one.

[0041] Figure 5 The deep learning confusion matrix schematic diagram according to embodiment one. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application.

[0043] Embodiment one, the embodiment proposes a multi-feature fusion enhanced feature extraction method based on respiratory sound signals, the method comprising the following steps:

[0044] Step 1, obtaining original respiratory sound signal data, and pre-processing the original respiratory sound signal data;

[0045] Step 2, extracting the time-frequency features of the pre-processed original respiratory sound signal data, the time-frequency features including static features and dynamic features;

[0046] Step 3, based on the time-frequency features extracted in step 2, using the SpecAugment method, designing a dual-channel synchronous random masking mechanism, and simultaneously performing random masking in the time and frequency dimensions on the static features and dynamic features;

[0047] Step 4, constructing an adaptive feature fusion and enhancement module for the static features and dynamic features, and inputting the fused and enhanced features into the backbone network of the pre-trained AST, to complete the multi-feature fusion and enhancement based on the respiratory sound signal feature extraction.

[0048] Embodiment two, the embodiment is a further limitation of the multi-feature fusion enhanced feature extraction method based on respiratory sound signals according to embodiment one, the method for pre-processing the original respiratory sound signal data in step 1 is:

[0049] Step 1.1, frame segmentation is performed on the original respiratory sound signal data using a fixed time window to generate 6-second long segments, and the remaining segments less than 6 seconds are discarded;

[0050] Step 1.2, the original respiratory sound signal data is resampled to 16 kHz;

[0051] Step 1.3, the baseline drift is removed using a frequency domain filtering method based on discrete Fourier transform, 1 Hz is selected as the cutoff frequency, and normalization processing is performed to ensure that the amplitudes of all audio segments are within the range of [-1, 1].

[0052] Embodiment three, this embodiment is a further limitation of the multi-feature fusion enhanced feature extraction method based on respiratory sound signal of embodiment one, the method for extracting the time-frequency features of the preprocessed original respiratory sound signal data in step 2 is:

[0053] Step 2.1, convert each 6-second time segment into a time-frequency representation by short-time Fourier transform method;

[0054] Step 2.2, map the linear frequency axis through the Mel scale, use 64 Mel filter banks, perform weighted summation on the STFT power spectrum, take the natural logarithm of the obtained Mel spectrogram, form a 64x188 log-Mel feature matrix, and use a sliding window difference method in the time dimension to calculate the change rate of the log-Mel feature matrix between adjacent frames; generate a Delta feature matrix with the same dimension as the log-Mel feature matrix, which is used to supplement the dynamic information of the audio signal;

[0055] Step 2.3, concatenate the log-Mel feature matrix and the Delta feature matrix in the channel dimension to form a 2x64x188 two-channel tensor, which is used as the input of the enhancement and fusion module in step 3.

[0056] Embodiment four, this embodiment is a further limitation of the multi-feature fusion enhanced feature extraction method based on respiratory sound signal of embodiment one, step 3 includes the following steps:

[0057] Step 3.1, set the maximum width of time masking , the maximum width of frequency masking , and set the synchronous masking trigger probability P;

[0058] Step 3.2, randomly sample the time window length (0, ) and the starting frame on the channel of the log-Mel feature matrix, set to zero in the interval , and randomly sample the frequency window length (0, ) and the start band , set to zero in the interval [ ] ;

[0059] Step 3.3, for each main feature masking, the probability P is used to determine whether to mask the channel of the Delta feature matrix synchronously, if not triggered, the original value of the channel of the Delta special matrix is reserved for compensating for the missing information of the Log-Mel special matrix; if synchronous masking is triggered, the same time frame and frequency interval on the Log-Mel special matrix is also set to zero in the channel of the Delta special matrix.

[0060] Embodiment five, this embodiment is a further limitation of the multi-feature fusion enhanced feature extraction method based on respiratory sound signal of embodiment three, the method for constructing the adaptive feature fusion and enhancement module in step 4 is:

[0061] Step 4.1, design a global perception modulation fusion module, let the two original input features be and , expand the channel to C=e dimensions through 1x1 convolution, and independently convolve each channel through deep convolution with groups=e, learn the spatial features of each channel, and use point-by-point convolution to mix the features of each channel extracted by deep convolution, to build a depth separable convolution structure, use global average pooling to compress information, generate two weight coefficients, and get and ;

[0062] Step 4.2, design a multi-scale attention time-frequency aggregation module, extract features with different receptive field sizes through convolution with different dilation rates, including four branches, splice the four branches into two groups, and perform attention mechanism operation on the two groups of features respectively; splice the two groups of processed features along the channel dimension to form a more unified feature map; and use asymmetric convolution kernels of 1xk and kxl to respectively expand along the time axis to capture the timing change pattern; and expand along the frequency axis to learn the local correlation between frequency components; secondly, a 1x1 convolution is used to fuse all channels to generate a single channel output most representative, denoted as . The output of the adaptive feature fusion and enhancement module is again adaptively weighted and connected with the two features of the original signal to get , which is used as the input of the pre-training model AST.

[0063] Embodiment six, this embodiment proposes a multi-feature fusion enhanced feature extraction system based on respiratory sound signal, the system comprises:

[0064] A data processing module is configured to acquire original respiratory sound signal data and pre-process the original respiratory sound signal data.

[0065] A time-frequency feature extraction module is configured to extract time-frequency features of the pre-processed original respiratory sound signal data, wherein the time-frequency features include static features and dynamic features.

[0066] A masking module is configured to design a double-channel synchronous random masking mechanism based on the time-frequency features extracted by the time-frequency feature extraction module, and perform random masking in the time and frequency dimensions on the static features and the dynamic features by using a SpecAugment method.

[0067] A multi-feature fusion and enhanced feature extraction module is configured to construct an adaptive feature fusion and enhancement module for the static features and the dynamic features, and input the fused and enhanced features into a backbone network of a pre-trained AST to complete multi-feature fusion and enhanced feature extraction based on respiratory sound signals.

[0068] Embodiment seven, the embodiment proposes a computer readable storage medium, the computer readable storage medium stores a computer program, the computer program is executed by a processor to realize the steps of the method in any one of embodiments one to five.

[0069] Embodiment eight, the embodiment proposes a computer device, including a memory and a processor, the memory stores a computer program, when the processor runs the computer program stored in the memory, the processor executes the method in any one of embodiments one to five.

[0070] Embodiment nine, the embodiment proposes an example for explaining the above-mentioned embodiments one to eight, and the example is specifically:

[0071] Referring to Figures 1 to 5 It is explained that the purpose of the present application is to propose a multi-feature fusion and enhanced feature extraction method based on respiratory sound signals. As shown in Figure 1 The present application realizes efficient fusion, fine feature extraction and classification of respiratory sound signals by synchronously enhancing and adaptively fusing static spectrograms and dynamic differential features of respiratory sounds, and designing a multi-scale hierarchical attention enhancement module, and using a pre-trained AST as a backbone network, which significantly improves the classification accuracy and model robustness.

[0072] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0073] Step one, acquire data, pre-process the data to unify the data length and reduce the noise influence.

[0074] Step two, extract two time-frequency features of the data (static and dynamic), ensure the spatial size of the two features is consistent, and constitute a double-channel input.

[0075] Step three, based on the SpecAugment method, design a double-channel synchronous random masking mechanism to randomly mask the static and dynamic features in time and frequency dimensions.

[0076] Step four, construct a feature fusion and enhancement module for the two features, and input the fused and enhanced features to the backbone network based on the pre-trained AST.

[0077] Further, step 1 includes the following steps:

[0078] Step 1.1, frame segmentation is performed on the original respiratory sound signal using a fixed time window to generate 6-second segments, and the remaining less than 6-second segments are discarded. Based on the typical periodic characteristics of the respiratory signal, to avoid information loss caused by respiratory period truncation at the window boundary, a 50% overlap strategy (i.e. 3 seconds overlap) is used to ensure smooth transition and feature continuity between adjacent segments.

[0079] Step 1.2, resample the original respiratory sound signal to 16 kHz.

[0080] Step 1.3, use the discrete Fourier transform (DET) based frequency domain filtering method to remove baseline drift, and select 1 Hz as the cutoff frequency. And perform normalization to ensure that the amplitudes of all audio segments are uniform in the range of [-1, 1]

[0081] Further, step 2 includes the following steps:

[0082] Step 2.1, convert each 6-second time segment to a time-frequency representation through short-time Fourier transform (STFT).

[0083] Step 2.2, map the linear frequency axis through the Mel scale, use 64 Mel filter banks, and perform weighted summation on the STFT power spectrum. Take the natural logarithm of the resulting Mel spectrogram to form a 64x188 log-Mel feature matrix. In the time dimension, use the sliding window difference method to calculate the change rate of the log-Mel feature between adjacent frames. Generate a Delta feature matrix with the same dimension as the log-Mel matrix to supplement the dynamic information of the audio signal.

[0084] Step 2.3, concatenate the log-Mel and Delta features in the channel dimension to form a 2x64x188 double-channel tensor, which is used as the input of the subsequent synchronous enhancement and fusion module.

[0085] Further, step 3 includes the following steps:

[0086] Step 3.1, Set the maximum width of time masking , Set the maximum width of frequency masking , Set the probability P of synchronous masking trigger.

[0087] Step 3.2, Randomly sample the length of time window on Log-Mel channel (0, ) and the starting frame , set to zero in the interval . Randomly sample the length of frequency window (0, ) and the starting frequency band , set to zero in the interval .

[0088] Step 3.3, For each main feature masking, decide whether to synchronize mask the Delta channel with a probability P. If not triggered, keep the original value of the Delta channel for the model to compensate for the missing information of the Log-Mel; if the synchronous masking trigger is triggered, set the same time frame and frequency interval on the Log-Mel to zero in the Delta channel. The masking effect is shown in Figure 2 .

[0089] Further, step 4 includes the following steps

[0090] Step 4.1, as shown in Figure 3 , design a global perceptual modulation fusion (PAMF) module. We denote the two features as and , expand the channel to C = e (expansion) dimensions through 1 × 1 convolution, and then perform independent convolution on each channel through deep convolution with groups = e to learn the spatial features of each channel. Then use pointwise convolution to mix the features of each channel extracted by deep convolution to construct a depth separable convolution structure. The module performs independent depth separable convolution operations on the two input features to obtain and , and expands their channel number to e (expansion) to enhance the original feature information expression ability; compared with standard convolution, this operation has a significant advantage in parameter quantity and calculation amount. Then we concatenate the two enhanced features into 2e to form a joint feature space, and perform a global average pooling (GAP) operation on the joint features to compress the entire joint feature space information to 1 × 1, only retaining the global information at the channel level. Then use a 1 × 1 convolution to reduce the channel number from 2e to , to reduce the amount of parameters and computation, and extract high-order features between channels, and use the ReLU activation function to introduce nonlinearity, so that the model can learn more complex feature representation. Then use a 1x1 convolution again to reduce the channel to 2, generate the corresponding weight coefficients for the two input features, and use the Softmax function to normalize the weight coefficients , ) so that their sum is 1, so as to determine the relative importance of each feature in the fusion process. According to the obtained weight coefficients, the enhanced Log-Mel spectrum features and Delta features are weighted and fused to obtain new fusion features . This adaptive feature selection mechanism, through the learning of global information of two input features, dynamically adjusts their fusion ratio according to the situation of the input features, so as to more efficiently utilize the complementary information between features, and further improve the model performance.

[0091] Step 4.2, as shown in Figure 4 , a multi-scale attention time-frequency aggregation (MSAA) module is designed, which is used to enhance feature expression ability. Different receptive field size features are extracted by convolution with different dilation rates: a total of four branches, 1x1 convolution, 3x3 convolution with dilation rates of 1, 3 and 5. Each branch is connected with BatchNorm+GELU to keep numerical stability and nonlinear expression. The four branches are spliced into two groups, the 1x1 convolution branch and the convolution branch with a dilation rate of 1 as the first group, and the channel splicing , focusing on fine local features; the other two branches as the second group, channel splicing , focusing on large-scale feature information. Then, we apply two computationally efficient and structurally simple attention mechanism operations to these two groups of features. For the first group of features, which represent more local details, the channel dimension represents more critical feature types, so we apply channel attention to it. First, the spatial information of each channel is compressed into a global descriptor through GAP, and then a lightweight multi-layer perceptron (MLP) composed of two 1x1 convolutions is used to learn the nonlinear dependence between channels, and finally a channel importance weight is generated. Multiply to get , complete the recalibration of the features, amplify the response of the key local feature channels, and suppress the secondary channels. For the second group of features, which represent more macro information, we use spatial attention. Through a minimalist 1x1 convolution channel compression to 1, fuse all channel information on each spatial position, and generate a two-dimensional spatial weight mask using a Sigmoid function. The region with weight close to 1 in the mask is more important in spatial information, while the region close to 0 will be weakened in spatial features. Multiply this weight with to complete the effective focusing of spatial features. Concatenate and along the channel dimension to form a more unified feature map. Then, asymmetric convolution kernels of 1xk and kxl are used in turn to expand along the time axis to capture the temporal variation pattern, and along the frequency axis to learn the local correlation between frequency components. Subsequently, a 1x1 convolution is used to fuse all channels to generate a single most representative channel output, denoted as . We adaptively weight the residual connection between the output of the module and the two features of the original signal to obtain , which is used as the input of the pre-trained AST model.

[0092] Step 4.3, the enhanced features are fused and used as the input of the AST model. The global modeling capability of AST can capture features in the entire audio, thus providing strong support for the breath sound classification task. A model pre-trained on ImageNet is used as the benchmark model after fine-tuning. The AST model trained on a large dataset has better generalization ability in the breath sound classification task.

[0093] Referring to Tables 1-2, the performance of different audio features is compared, and the Log-Mel feature is used as the benchmark feature to achieve the best performance.

[0094] Table 1 Comparison results of different features

[0095]

[0096] Table 2 Comparison results of the present application and existing methods

[0097]

[0098] Table 1 compares the performance of different time-frequency features extracted from preprocessed breath sound data, and the Log-Mel feature is used as the benchmark feature to achieve the best performance.

[0099] ​Table 2 compares the method described in the present application with the methods commonly used in the prior art, including mainstream models such as CNN, RNN, and methods that adopt multiple feature fusion (Mel feature, MFCC feature, etc.). In summary, the method of the present application performs outstandingly in performance evaluation, achieving an average accuracy of 96.24%, a specificity of 98.7%, a precision of 94.11%, a recall of 96.42%, and an F1 score of 95.10% for the four types of subjects.

[0100] The present embodiment first extracts static spectrograms and dynamic differential features from the respiratory sound signal, and uses global perception information to adaptively adjust the fusion ratio of the two types of features through the PAFM module, fully utilizing the complementary advantages. Then, the MSAA module extracts multi-scale features using dilated convolution, and combines channel and spatial attention mechanisms to adaptively weight local and global features, respectively, to enhance key information and suppress interference. Finally, the enhanced features are input into the pre-trained AST backbone network to realize efficient fusion and fine feature extraction, significantly improving the model performance.

[0101] Those skilled in the art can understand that the above description is only preferred embodiments of the present application, and the features described in each embodiment and / or technical solution of the present disclosure can be combined or combined, even if such combination or combination is not explicitly described in the present disclosure. It is not intended to limit the present application, although the present application has been described in detail with reference to the foregoing embodiments, and those skilled in the art can modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features, any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0102] Although preferred embodiments of the present application have been described, those skilled in the art can make further changes and modifications to these embodiments once they know the basic inventive concept. Therefore, the appended claims are intended to include the preferred embodiments and all changes and modifications falling within the scope of the present application. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application also intends to include these modifications and variations.

Claims

1. A feature extraction method based on multi-feature fusion enhancement of respiratory sound signals, characterized in that, The method comprises the following steps: Step 1, obtaining original respiratory sound signal data and preprocessing the original respiratory sound signal data; Step 2, extracting time-frequency features of the preprocessed original respiratory sound signal data, the time-frequency features comprising static features and dynamic features; Step 3, based on the time-frequency features extracted in step 2, using a SpecAugment method, designing a double-channel synchronous random masking mechanism, and simultaneously performing random masking in the time and frequency dimensions on the static features and the dynamic features; Step 4, constructing an adaptive feature fusion and enhancement module for the static features and the dynamic features, and inputting the fused and enhanced features into the backbone network of a pre-trained AST to complete feature extraction based on multi-feature fusion and enhancement of respiratory sound signals; The static features are Log-Mel spectrograms, and the dynamic features are first-order differential features Delta of the Log-Mel spectrograms; Step 3 comprises the following steps: Step 3.1, set time mask maximum width , frequency mask maximum width , set synchronization mask trigger probability P; Step 3.2: Randomly sample the time window length on the channels of the Log-Mel feature matrix. (0, ) and start frame ,exist[ Set the interval to zero, and adjust the random sampling frequency window length. (0, ) and starting frequency band ,exist[ Set the interval to zero; Step 3.3, for each main feature masking, a probability P is used to determine whether to synchronously mask the channels of the Delta feature matrix, if not triggered, the original value of the channel of the Delta special matrix is retained to compensate for the missing information of the Log-Mel special matrix, if synchronous masking is triggered, the same time frame and frequency interval on the Log-Mel special matrix are also set to zero in the channel of the Delta special matrix.

2. The method of claim 1, wherein the method is enhanced feature extraction based on multi-feature fusion of breath sound signals. The method for preprocessing the original respiratory sound signal data in step 1 is: Step 1.1, frame segmentation is performed on the original respiratory sound signal data using a fixed time window to generate segments with a length of 6 seconds, and the remaining segments less than 6 seconds are discarded; Step 1.2, resampling the original respiratory sound signal data to 16 kHz; Step 1.3, using a frequency domain filtering method based on discrete Fourier transform to remove baseline drift, selecting 1 Hz as the cutoff frequency, and performing normalization processing to ensure that the amplitudes of all audio segments are uniform in the range of [-1, 1]. 3.The method of claim 1, wherein, The method for extracting time-frequency features of the preprocessed original respiratory sound signal data in step 2 is: Step 2.1, converting each 6-second time segment into a time-frequency representation through a short-time Fourier transform method; Step 2.2, mapping the linear frequency axis through a Mel scale, using a 64-Mel filter bank to perform weighted summation on the STFT power spectrum, taking the natural logarithm of the obtained Mel spectrogram to form a 64x188 log-Mel feature matrix, and using a sliding window difference method in the time dimension to calculate the change rate of the log-Mel feature matrix between adjacent frames; a Delta feature matrix with the same dimension as the log-Mel feature matrix is generated to supplement the dynamic information of the audio signal; Step 2.3, concatenating the log-Mel feature matrix and the Delta feature matrix in the channel dimension to form a 2x64x188 double-channel tensor, which is used as the input of the enhancement and fusion module in step 3. 4.The method of claim 1, wherein, The method for constructing an adaptive feature fusion and enhancement module in step 4 is: Step 4.1, design a global perception modulation fusion module, two original input features are respectively denoted as and , expand the channel to C = e dimensions through 1x1 convolution, and independently convolve each channel through depth convolution with groups = e to learn the spatial features of each channel, use pointwise convolution to mix the channel features extracted by depth convolution, and construct a depth separable convolution structure, use global average pooling to compress information, generate two weight coefficients, and obtain and ; Step 4.2, when designing a multi-scale attention time-frequency aggregation module, different field sizes of features are extracted through convolution with different expansion rates, including four branches in total, the four branches are spliced into two groups, and the attention mechanism operation is performed on the two groups of features respectively; the processed two groups of features are spliced along the channel dimension to form a more unified feature map; and asymmetric convolution kernels of 1xk and kxl are used to respectively expand along the time axis to capture the timing change pattern; and expand along the frequency axis to learn the local correlation between frequency components; Secondly, all channels are fused by a 1x1 convolution to generate a most representative single-channel output, denoted as The output of the adaptive feature fusion and enhancement module is adaptively weighted and connected with the residual of the two features of the original signal to obtain It is used as the input of the pre-training model AST.

5. A multi-feature fusion enhanced feature extraction system based on respiratory sound signals, characterized in that, The system comprises: A data processing module for obtaining original respiratory sound signal data and preprocessing the original respiratory sound signal data; A time-frequency feature extraction module for extracting time-frequency features of the preprocessed original respiratory sound signal data, the time-frequency features including static features and dynamic features; A masking module for designing a double-channel synchronous random masking mechanism based on the time-frequency features extracted by the time-frequency feature extraction module using the SpecAugment method, and simultaneously performing random masking in the time and frequency dimensions on the static features and the dynamic features; A multi-feature fusion and enhanced feature extraction module for constructing an adaptive feature fusion and enhancement module for the static features and the dynamic features, and inputting the fused and enhanced features into the backbone network of the pre-trained AST to complete multi-feature fusion and enhanced feature extraction based on respiratory sound signals; The static features are Log-Mel spectrograms, and the dynamic features are first-order differential features Delta of the Log-Mel spectrograms; The masking module includes the following steps: Step 3.1, set time mask maximum width , frequency mask maximum width , set synchronization mask trigger probability P; Step 3.2: Randomly sample the time window length on the channels of the Log-Mel feature matrix. (0, ) and start frame ,exist[ Set the interval to zero, and adjust the random sampling frequency window length. (0, ) and starting frequency band ,exist[ Set the interval to zero; Step 3.3, for each main feature masking, the probability P is used to determine whether to mask the channels of the Delta feature matrix synchronously, if not triggered, the original value of the channel of the Delta special matrix is retained to compensate for the missing information of the Log-Mel special matrix; if synchronous masking is triggered, the same time frame and frequency interval on the Log-Mel special matrix are also set to zero in the channel of the Delta special matrix.

6. A computer storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the method of any one of claims 1-4.

7. A computer device, comprising: Comprise: A memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-4.

Images