Double-path CNN heart sound classification method based on time-frequency and double-spectrum fusion features

Abstract

The invention relates to the technical field of audio signal processing and biomedical signal analysis, and still has a further optimized space for the recognition of anti-noise requirements, signal individual differences and complex pathological modes in a noise environment. The invention provides a double-path CNN heart sound classification method based on time-frequency and double-spectrum fusion features, and the method comprises the steps: carrying out the preprocessing of an original heart sound signal of a data set which is classified into a normal heart sound and an abnormal heart sound, and obtaining a to-be-recognized heart sound signal; based on dynamic continuous wavelet transform, adaptively selecting parameters to extract time-frequency characteristics, introducing bispectrum analysis, capturing nonlinear characteristics, generating a dual-channel characteristic pattern, and efficiently storing the dual-channel characteristic pattern in an HDF5 format; and based on a designed double-path convolutional neural network structure, respectively processing the extracted time-frequency and double-spectrum features, performing classification after fusion, and training a model in combination with category weighted loss and an optimization strategy to obtain a heart sound classification result. The heart sound recognition accuracy can be improved.

Description

Technical Field

[0001] This invention belongs to the field of audio signal processing and biomedical signal analysis technology, and more specifically, relates to a dual-path CNN heart sound classification method based on time-frequency and bispectral fusion features. Background Technology

[0002] Heart sounds are important biomedical signals for assessing heart health. With technological advancements, heart sound recognition has evolved from traditional auscultation to automated analysis methods based on signal processing and machine learning, supporting heart sound signal classification and remote monitoring. Despite the challenges of complex environments and diverse signals, heart sound recognition technology continues to evolve.

[0003] Traditional heart sound recognition and classification methods primarily extract features through signal processing. For example, time-domain analysis uses statistical features (such as amplitude and peak interval) to describe signal characteristics, while frequency-domain analysis obtains spectral information through Fourier transform. Wavelet transform, as a time-frequency analysis tool, can decompose signals and capture local features, and has been applied in heart sound analysis. In recent years, Mel-frequency cepstral coefficients (MFCCs), due to their success in speech processing, have also been introduced into the field of heart sound recognition. MFCCs, by simulating the nonlinear perception of frequency by the human ear, convert the signal into cepstral domain features, reflecting the spectral envelope characteristics of heart sounds, and are suitable for describing their low-frequency dominant patterns. These methods can all achieve preliminary heart sound recognition under specific conditions.

[0004] With the development of machine learning, methods such as Support Vector Machines (SVM) and Random Forests have improved classification capabilities by training classifiers to model extracted features (such as MFCC and wavelet coefficients). The rise of deep learning technology has further promoted the progress of heart sound recognition. Models such as Convolutional Neural Networks (CNNs) automatically extract features from inputs such as time-frequency maps or MFCCs through end-to-end learning, adapting to some complex signal patterns and demonstrating good performance.

[0005] In summary, heart sound recognition technology has made progress in signal processing, MFCC applications, machine learning, and deep learning. However, there is still room for further optimization to address the need for noise immunity in noisy environments, individual signal differences, and the recognition of complex pathological patterns. Therefore, exploring new heart sound recognition methods to improve performance and adaptability is of great significance. Summary of the Invention

[0006] To address the problems and shortcomings of existing technologies, this invention provides a dual-path CNN heart sound classification method based on time-frequency and bispectral fusion features. By combining the time-frequency local features of wavelet transform with the nonlinear coupling features of bispectral analysis, a dual-channel input is constructed to improve the accuracy of heart sound classification. It adaptively matches the frequency band characteristics of heart sound signals and enhances the ability to distinguish between low-frequency pathological components and high-frequency noise.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A dual-path CNN method for heart sound classification based on time-frequency and dual-spectrum fusion features preprocesses and extracts features from the heart sound signal, and then uses a hybrid neural network for classification. The method includes the following steps: Step 1: Heart sound signal preprocessing: The original heart sound signal is filtered, downsampled, normalized and segmented to obtain a standardized heart sound signal sample that is easy to extract features. Step 2, Time-Frequency and Bispectral Feature Extraction: Perform continuous wavelet transform on the preprocessed heart sound signal in Step 1, dynamically select the wavelet basis function, extract the amplitude of the wavelet coefficients, then perform bispectral analysis on the preprocessed heart sound signal, calculate the amplitude features of the bispectral matrix, and finally normalize the wavelet amplitude and bispectral amplitude features and stitch them together to form a two-channel heart sound sample feature map. Step 3, Hybrid Convolutional Neural Network Classification: Construct a dual-path convolutional network, with path 1 and path 2 processing time-frequency features and bispectral features respectively; concatenate the outputs of the two paths along the channel dimension to form a feature map containing fused time-frequency and bispectral information, and extract deep features through multi-scale convolution and global average pooling; the output layer uses the Sigmoid function to achieve binary classification of normal and abnormal heart sound signals with a set decision threshold.

[0008] Furthermore, in step 1, the preprocessing of the heart sound signal specifically includes the following steps: Step 1.1: Remove the data from the first and last seconds of the original heart sound signal and input it into a fifth-order Butterworth bandpass filter for filtering. Step 1.2: The filtered heart sound signal is downsampled by combining anti-aliasing filtering and interpolation decimation in a high-quality downsampling technique based on sinc interpolation to achieve sampling rate conversion; and the heart sound signal is normalized. Step 1.3: Based on the periodicity of the heart sound signal, perform frame segmentation on the normalized heart sound signal to obtain samples of the standardized heart sound signal.

[0009] Furthermore, in step 2, the extraction of time-frequency and bispectral features specifically includes the following steps: Step 2.1: Perform a Fast Fourier Transform on the preprocessed heart sound signal and calculate the dominant frequency of the spectrum. in Representing the discrete Fourier transform of heart sound signals, To input the original signal, =0, 1, ..., M-1, where M is the signal length; Select the complex Morlet wavelet basis function based on the dominant frequency: Analyze the wavelet basis function parameters, extract the center frequency ω, and determine the target frequency range [ , Calculate the scale parameters: in, The sampling rate of the heart sound signal. The lower limit frequency of the preset target frequency band. The upper limit frequency of the preset target frequency band; Generate geometrically spaced scale sequences: The generation of geometrically spaced scale sequences satisfies the following conditions: in, is the minimum scale parameter of the wavelet transform. The maximum scale parameter of the wavelet transform; N =num represents the number of scales. λ =0,1,…,N−1; The time-domain expression of the complex Morlet wavelet basis function is: in, For bandwidth parameters, The center frequency; Perform wavelet transform on the heart sound signal to generate a complex coefficient matrix: CWT ( , )= in, Let x(t) be the complex Morlet wavelet basis function, and x(t) be the preprocessed heart sound signal. For scale parameters, These are translation parameters; Finally, extract the wavelet amplitude value: ; Step 2.2: Perform a short-time Fourier transform on the preprocessed heart sound signal to generate a complex spectrum matrix: in, This is the original heart sound signal. The segment length, For window functions, m For time indexing, n The sample index of the signal; Calculate the bispectral matrix based on the third-order cumulants of the spectrogram: in, Indicates the first k The spectrum of the segment signal, K Total number of segments For frequency components, , The signal sampling rate; Extracting bispectral amplitude: Step 2.3: Based on the wavelet amplitude features and bispectral amplitude features extracted in Steps 2.1 and 2.2, create a two-channel blank matrix. Set the dimensions of the blank matrix to match the matrix size obtained from wavelet transform and bispectral analysis to ensure the consistency of the feature maps; perform channel filling and normalization on the blank matrix. Channel 1 is filled with wavelet amplitude values: in, The amplitude of the wavelet transform. and These are the mean and standard deviation of the wavelet transform amplitude, respectively; Channel 2 uses bispectral amplitude filling: in, For bispectral amplitude, and The mean and standard deviation of the bispectral amplitudes are respectively used as the inputs to the hybrid convolutional neural network.

[0010] Furthermore, in step 3, the constructed dual-path convolutional neural network classifies and identifies the feature maps of the heart sound signals, specifically including the following steps: Step 3.1: Using the heart sound time-frequency and bispectral fusion feature map as input to a hybrid convolutional neural network, a dual-path parallel convolutional neural network is constructed. Path 1 processes the time-frequency features, which are generated through wavelet transform to capture the linear spectral characteristics of the heart sound signal in the time and frequency dimensions. Path 2 processes the bispectral features, which are obtained through bispectral analysis to extract the nonlinear phase coupling information of the heart sound signal. Path 1 and Path 2 each have one convolutional layer and 32 convolutional kernels. Step 3.2, Feature concatenation: After concatenating the dual-path outputs along the passband dimension, the outputs are passed through two multi-scale convolutional layers and one fully connected layer, resulting in a single neuron in the output layer. Step 3.3: The output value of the Sigmoid activation function in the output layer represents the probability that the sample belongs to the positive category. According to the preset decision threshold, when the output probability is greater than or equal to the decision threshold, the sample is determined to be a positive category, i.e., an abnormal heart sound signal sample; if the output probability is less than the decision threshold, it is determined to be a negative category, i.e., a normal heart sound signal sample.

[0011] In summary, the invention has the following beneficial effects: (1) This invention combines the time-frequency local features of wavelet transform with the nonlinear coupling features of bispectral analysis to construct a dual-channel input including wavelet amplitude and bispectral amplitude, which can effectively improve the accuracy of heart sound classification; (2) The present invention uses a fifth-order Butterworth bandpass filter with a cutoff frequency of 20-500Hz and downsamples to 1000Hz, which can reduce data redundancy while retaining the key frequency band signal; (3) The present invention dynamically selects wavelet basis functions based on the main frequency of the signal, adaptively matches the frequency band characteristics of the heart sound signal, and enhances the ability to distinguish between low-frequency signal components and high-frequency noise; (4) This invention uses a dual-path parallel structure to independently process time-frequency features and bispectral nonlinear features to avoid modal interference. At the same time, it uses multi-scale convolution kernels (3×3, 5×5, 7×7) to capture local details and mid-to-high-level semantic information in layers, thereby enhancing feature specificity and realizing hierarchical expression. Attached Figure Description

[0012] Figure 1 This is a flowchart of the present invention; Figure 2 This is a flowchart of the heart sound signal preprocessing of the present invention; Figure 3 The normal heart sound signal of this invention is processed into a preprocessed image, a time-frequency feature image, and a bispectral feature image; Figure 4 The abnormal heart sound signal of this invention is processed into a preprocessed image, a time-frequency feature image, and a bispectral feature image; Figure 5 This is a diagram of the dual-path convolutional neural network structure of the present invention. Detailed Implementation

[0013] To facilitate understanding of the present invention, a more comprehensive description of the present invention will be given below with reference to the accompanying drawings.

[0014] like Figures 1-5 As shown, this invention discloses a dual-path CNN heart sound classification method based on time-frequency and dual-spectrum fusion features. The method preprocesses and extracts features from the heart sound signals, and then uses a hybrid neural network for classification. The specific steps are as follows: Step 1: Heart sound signal preprocessing: Select heart sound signals sampled at 2000Hz for preprocessing. Step 1.1: Remove the first and last second data of each heart sound signal to eliminate noise generated by friction when the receiving equipment contacts the auscultation site; use a fifth-order Butterworth bandpass filter with a cutoff frequency of 20-500Hz to digitally filter the heart sound signal, filtering out low-frequency artifacts and high-frequency noise. When a person breathes, the volume and pressure of the thoracic cavity change continuously. This change will cause slight movements of the surrounding tissues and organs, thus adding low-frequency artifact components during the heart sound signal acquisition process. This is the main source of low-frequency artifacts in the heart sound signal. Step 1.2: The Butterworth bandpass filter has a cutoff frequency of 20-50Hz. The filtered heart sound signal is downsampled using a combination of anti-aliasing filtering and interpolation decimation in high-quality downsampling techniques based on sinc interpolation. This effectively converts the sampling rate while ensuring signal quality and integrity. Step 1.3: The heart sound signal is normalized by downsampling the filtered heart sound signal to 1000Hz, achieving a sampling rate conversion from 2000Hz to 1000Hz. This satisfies the Nyquist sampling theorem requirement, where the target sampling rate of 1000Hz is twice the highest frequency of the filtered signal (500Hz), ensuring no aliasing distortion in the effective frequency band and reducing data redundancy. Due to significant differences in audio file scale across different datasets, the heart sound audio signal is normalized. Finally, based on the cardiac cycle characteristics, the signal is framed with a fixed sampling point length of 2500 points. To avoid truncating key pathological features, the inter-frame overlap rate is 50%, generating standardized heart sound signal samples that facilitate feature extraction.

[0015] Step 2, Time-Frequency and Bispectral Feature Extraction: Perform continuous wavelet transform feature extraction on the preprocessed heart sound signal and calculate the signal's FFT dominant frequency. : in Representing the discrete Fourier transform of heart sound signals, To input the original signal, =0, 1, ..., M-1, where M is the signal length.

[0016] Dynamically select the wavelet basis function (cmor1.5-1.0 or cmor2.5-1.5). Different signals have different frequency characteristics and time-domain features. For high-frequency and transient signals, choosing "cmor2.5-1.5" with a higher center frequency is more beneficial for capturing rapidly changing signal characteristics; while for low-frequency and relatively stable signals, "cmor1.5-1.0" provides better time-domain resolution. Select the complex Morlet wavelet basis based on the dominant frequency: Calculate the scale parameters: in, is the minimum scale parameter of the wavelet transform. The maximum scale parameter of the wavelet transform. The fundamental center frequency of the wavelet. The lower limit frequency of the preset target frequency band. The upper limit frequency of the preset target frequency band; in this embodiment, =20Hz, =500Hz.

[0017] Generate geometric interval scale: The generation of geometrically spaced scale sequences satisfies the following conditions: in, is the minimum scale parameter of the wavelet transform. The maximum scale parameter of the wavelet transform; N =num represents the number of scales. λ =0,1,…,N−1; the time-domain expression of the complex Morlet wavelet basis function is: in, For bandwidth parameters, The center frequency is used; wavelet transform is performed on the heart sound signal to generate a complex coefficient matrix: CWT ( , )= in, Let x(t) be the complex Morlet wavelet basis function, and x(t) be the preprocessed heart sound signal. For scale parameters, Using the translation parameters, perform a continuous wavelet transform to obtain the complex coefficient matrix; extract the wavelet amplitude (cwt_mag): Perform a short-time Fourier transform on the same heart sound signal to generate a complex spectrum matrix: in, This is the original heart sound signal. The segment length, For window functions, m For time indexing, n The sample index of the signal; Bispectral feature extraction was performed, followed by short-time Fourier transform, with a segment length of [missing information]. Calculate the bispectral matrix based on the third-order cumulants of the spectrogram: in, Indicates the first k The spectrum of the segment signal, K Total number of segments For frequency components, , The signal sampling rate; Extracting bispectral amplitude: Mean-mean normalization is performed on each channel, and wavelet transform and bispectral analysis are used to obtain feature maps with matching matrix sizes. The channel order is as follows: FeatureMap=[cwt_mag, bispec_mag].

[0018] Step 3: Hybrid Convolutional Neural Network Classification: Construct a dual-path convolutional network, with path 1 and path 2 processing time-frequency features and bispectral features respectively; concatenate the outputs of the two paths along the channel dimension to form a feature map containing fused time-frequency and bispectral information; extract deep features through multi-scale convolution and global average pooling; the output layer uses the Sigmoid function to achieve binary classification of normal and abnormal heart sound signals with a set decision threshold. In this embodiment, the decision threshold is set to 0.5.

[0019] Step 3.1: The heart sound time-frequency and bispectral fusion features are used as input to a dual-path convolutional neural network. Path 1 processes the time-frequency features, which are generated through wavelet transform to capture the linear spectral characteristics of the heart sound signal in the time and frequency dimensions. Path 2 processes the bispectral features, which are obtained through bispectral analysis to extract the nonlinear phase coupling information of the heart sound signal. Path 1 and Path 2 each have one convolutional layer with 32 convolutional kernels. The outputs of the two paths are concatenated along the channel dimension for feature fusion. Deep feature extraction consists of two multi-scale convolutional layers and one fully connected layer. The specific structure is as follows: Figure 5 As shown.

[0020] Step 3.2, Feature Concatenation: After concatenating the dual-path outputs along the passband dimension, the outputs are passed through two multi-scale convolutional layers and one fully connected layer. The output layer is a single neuron. The first convolutional layer has 64 kernels and a size of 5×5; the second convolutional layer has 128 kernels and a size of 7×7, forming a hierarchical feature representation.

[0021] Step 3.3: The output layer uses the Sigmoid activation function, whose output value represents the probability that the sample belongs to the positive category. According to the preset decision threshold, in this embodiment, the decision threshold is 0.5 to achieve binary classification. When the output probability is ≥0.5, the dual-path CNN heart sound classification model of the present invention, which integrates time-frequency and dual-spectrum features, determines that the sample is a positive category (abnormal heart sound signal sample); if the output probability is <0.5, it is determined to be a negative category (normal heart sound signal sample).

[0022] The training data used in this invention originates from internationally available medical databases, specifically the physiological acoustic classification dataset provided by the 2016 PhysioNet / Computing in Cardiology Challenge. This dataset includes samples from healthy individuals and patients with heart disease, focusing on heart sound signals from two typical pathological types: abnormal heart valve structure and coronary artery disease. A total of 3240 raw heart sound samples were acquired, standardized using various types of electronic auscultation devices. The signal duration ranged from 5 to 120 seconds, with 2575 normal heart sound signals and 665 abnormal heart sound signals. After processing in step 1, heart sound samples were obtained: 36515 normal samples and 11469 abnormal samples. Step 2 used time-frequency and bispectral analysis to convert both normal and abnormal heart sound signals into wavelet + bispectral feature maps, which were then used to train, validate, and test the dual-path CNN classification model. 70% of the normal and abnormal heart sound samples were randomly selected as the training set to train the model, 20% were used as the validation set to monitor the model's performance during training, and 10% were used as the test set to test the classification model's performance.

[0023] During training, a triple callback mechanism is introduced: automatically saving the optimal model based on validation loss, an early stopping strategy with a patience value, and dynamic learning rate decay. An inverse frequency-weighted algorithm is applied to balance class bias, with the weight calculation formula as follows: ( i (As a category index), effectively improving the detection sensitivity of abnormal heart sounds.

[0024] This invention achieves high-precision classification of heart sound signals by integrating the time-frequency characteristics of wavelet transform with the nonlinear coupling characteristics of bispectral analysis, combined with a dual-path hybrid convolutional neural network and dynamic category weight optimization. It features strong anti-interference capability, high computational efficiency, and easy deployment.

[0025] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A dual-path CNN heart sound classification method based on time-frequency and dual-spectral fusion features, characterized in that, The heart sound signals are preprocessed and feature extracted, and then classified using a hybrid neural network. The specific steps include: Step 1: Heart sound signal preprocessing: The original heart sound signal is filtered, downsampled, normalized and segmented to obtain a standardized heart sound signal sample that is easy to extract features; Step 2, Time-Frequency and Bispectral Feature Extraction: Perform continuous wavelet transform on the preprocessed heart sound signal in Step 1, dynamically select the wavelet basis function, extract the amplitude of the wavelet coefficients, then perform bispectral analysis on the preprocessed heart sound signal, calculate the amplitude features of the bispectral matrix, and finally normalize the wavelet amplitude and bispectral amplitude features and stitch them together to form a two-channel heart sound sample feature map. Step 3, Hybrid Convolutional Neural Network Classification: Construct a dual-path convolutional network, with path 1 and path 2 processing time-frequency features and bispectral features respectively; concatenate the outputs of the two paths along the channel dimension to form a feature map containing fused time-frequency and bispectral information, and extract deep features through multi-scale convolution and global average pooling; the output layer uses the Sigmoid function to achieve binary classification of normal and abnormal heart sound signals with a set decision threshold.

2. The dual-path CNN heart sound classification method based on time-frequency and dual-spectral fusion features according to claim 1, characterized in that, Step 1, specifically the preprocessing of the heart sound signal, includes the following steps: Step 1.1: Remove the data from the first and last seconds of the original heart sound signal and input it into a fifth-order Butterworth bandpass filter for filtering. Step 1.2: The filtered heart sound signal is downsampled by combining anti-aliasing filtering and interpolation decimation in a high-quality downsampling technique based on sinc interpolation to achieve sampling rate conversion; and the heart sound signal is normalized. Step 1.3: Based on the periodicity of the heart sound signal, perform frame segmentation on the normalized heart sound signal to obtain samples of the standardized heart sound signal.

3. The dual-path CNN heart sound classification method based on time-frequency and dual-spectral fusion features according to claim 1, characterized in that, Step 2, the extraction of time-frequency and bispectral features, specifically includes the following steps: Step 2.1: Perform a Fast Fourier Transform on the preprocessed heart sound signal and calculate the dominant frequency of the spectrum. in Representing the discrete Fourier transform of heart sound signals, To input the original signal, =0, 1, ..., M-1, where M is the signal length; Select the complex Morlet wavelet basis function based on the dominant frequency: Analyze the wavelet basis function parameters, extract the center frequency ω, and determine the target frequency range [ , Calculate the scale parameters: in, The sampling rate of the heart sound signal. The lower limit frequency of the preset target frequency band. The upper limit frequency of the preset target frequency band; Generate geometrically spaced scale sequences: The generation of geometrically spaced scale sequences satisfies the following conditions: in, is the minimum scale parameter of the wavelet transform. The maximum scale parameter of the wavelet transform; N =num represents the number of scales. λ =0,1,…,N−1; The time-domain expression of the complex Morlet wavelet basis function is: in, For bandwidth parameters, The center frequency; Perform wavelet transform on the heart sound signal to generate a complex coefficient matrix: CWT ( , )= in, Let x(t) be the complex Morlet wavelet basis function, and x(t) be the preprocessed heart sound signal. For scale parameters, These are translation parameters; Finally, extract the wavelet amplitude value: ； Step 2.2: Perform a short-time Fourier transform on the preprocessed heart sound signal to generate a complex spectrum matrix: in, This is the original heart sound signal. The segment length is... For window functions, m For time indexing, n The sample index of the signal; Calculate the bispectral matrix based on the third-order cumulants of the spectrogram: in, Indicates the first k The spectrum of the segment signal, K Total number of segments For frequency components, , The signal sampling rate; Extracting bispectral amplitude: Step 2.3: Based on the wavelet amplitude features and bispectral amplitude features extracted in Steps 2.1 and 2.2, create a two-channel blank matrix. Set the dimensions of the blank matrix to match the matrix size obtained from wavelet transform and bispectral analysis to ensure the consistency of the feature maps; perform channel filling and normalization on the blank matrix. Channel 1 is filled with wavelet amplitude values: in, The amplitude of the wavelet transform. and These are the mean and standard deviation of the wavelet transform amplitude, respectively; Channel 2 uses bispectral amplitude filling: in, For bispectral amplitude, and The mean and standard deviation of the bispectral amplitudes are respectively used as the inputs to the hybrid convolutional neural network.

4. The dual-path CNN heart sound classification method based on time-frequency and dual-spectral fusion features according to claim 1, characterized in that, In step 3, the constructed dual-path convolutional neural network classifies and identifies the feature maps of the heart sound signals, specifically including the following steps: Step 3.1: Using the heart sound time-frequency and bispectral fusion feature map as input to a hybrid convolutional neural network, a dual-path parallel convolutional neural network is constructed. Path 1 processes the time-frequency features, which are generated through wavelet transform to capture the linear spectral characteristics of the heart sound signal in the time and frequency dimensions. Path 2 processes the bispectral features, which are obtained through bispectral analysis to extract the nonlinear phase coupling information of the heart sound signal. Path 1 and Path 2 each have one convolutional layer and 32 convolutional kernels. Step 3.2, Feature concatenation: After concatenating the dual-path outputs along the passband dimension, the outputs are passed through two multi-scale convolutional layers and one fully connected layer, resulting in a single neuron at the output layer. Step 3.3: The output value of the Sigmoid activation function in the output layer represents the probability that the sample belongs to the positive category. According to the preset decision threshold, when the output probability is greater than or equal to the decision threshold, the sample is determined to be a positive category, i.e., an abnormal heart sound signal sample; if the output probability is less than the decision threshold, it is determined to be a negative category, i.e., a normal heart sound signal sample.

Images