Heart sound signal identification method

Through the dual-branch feature extraction network, traditional audio and higher-order spectral features are extracted in parallel, combined with the channel attention mechanism, the limitations of the existing technology in capturing complex features of the heart sound signal are solved, and the accuracy of heart sound signal recognition and model performance are significantly improved. .

CN119007758AInactive Publication Date: 2024-11-22NANTONG INST OF TECH

Patent Information

Application Number
CN202411282983.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-13
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing heart sound signal classification method has limitations in capturing complex nonlinear features and their complex interactions, and cannot fully reflect the characteristics of heart sound signal, limiting the performance of the model.

Method used

Two-branch feature extraction network is adopted to extract traditional audio features and higher-order spectral features in parallel, and use one-dimensional convolutional neural network and two-dimensional convolutional neural network to process different types of features, combining the channel attention mechanism to dynamically pay attention to audio. Important areas in the signal reduce errors and improve feature representation capabilities.

Benefits of technology

The complex frequency relationship and nonlinear characteristics of the heart sound signal are effectively captured, and the recognition accuracy is improved. The accuracy of the model and F1 score on the public data set are significantly improved, proving the effectiveness of the method.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007758A_ABST
    Figure CN119007758A_ABST
Patent Text Reader

Abstract

The invention discloses a heart sound signal identification method, which relates to the technical field of signal processing, and comprises the following steps: S1, preprocessing an original audio signal; s2, extracting two groups of features from the audio signal, wherein the two groups of features are respectively traditional audio features and high-order spectrum features; s3, building a model, and constructing a double-branch feature extraction network; s4, training and verifying the model; according to the method, the complex frequency relation and key features in the audio signals are effectively captured in a mode of extracting the high-order spectrum features and the traditional audio features in parallel, the high-order spectrum features can reveal nonlinear frequency coupling in the signals, and the performance of the model on the independent test set is evaluated. And the traditional audio features capture the overall time-frequency characteristics of the signals, and the model can obtain more comprehensive signal description by combining the different types of features, so that the recognition accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a heart sound signal recognition method. Background Art

[0002] Cardiovascular diseases pose a great threat to human health, so their early diagnosis is particularly important. Heart sound signals are key indicators of heart function and are widely used in the diagnosis of cardiovascular diseases. By analyzing abnormal sounds in heart sound signals, a variety of heart diseases can be identified. The complexity and diversity of heart sound signals, such as variable waveform patterns, potential noise interference, and significant individual differences, increase the difficulty of effective detection of heart sound signals.

[0003] At present, the classification methods of heart sound signals are mainly concentrated in two directions: one is based on traditional machine learning methods, which rely on artificially designed features extracted from the signal, such as time domain features and frequency domain features. For example, some studies extracted the Shannon entropy and spectral entropy of each frequency band and used them as inputs to the support vector machine; other studies extracted power features and compared the classification performance of support vector machines, random forests, and multilayer perceptrons. However, the traditional feature extraction and selection process is still insufficient in capturing the complexity and nonlinear relationship of heart sound signals, and cannot fully reflect the characteristics of heart sound signals, which in turn limits the performance of the model.

[0004] The second is the method based on deep learning. This type of method can automatically extract features, especially convolutional neural networks, which show obvious advantages in processing the nonlinear characteristics of heart sound signals. In order to enhance the information flow of convolutional neural networks, some studies have introduced a bidirectional connection structure and achieved good classification results. Other studies have explored the time-frequency distribution as the input of convolutional neural networks, thus providing new ideas for deep learning automatic diagnosis of heart sound signals. However, although these methods have shown superior performance, they still have certain limitations in capturing the nonlinear characteristics of heart sound signals and their complex interactions.

[0005] Therefore, a heart sound signal recognition method is proposed, which can better capture the complex characteristics of heart sound signals. Summary of the invention

[0006] The purpose of the present invention is to solve the problems in the prior art and to propose a heart sound signal recognition method.

[0007] A heart sound signal recognition method comprises the following steps:

[0008] S1, preprocessing the original audio signal;

[0009] S2, extracting two sets of features from the audio signal, the two sets of features are traditional audio features and high-order spectrum features;

[0010] S3, model building, constructing a dual-branch feature extraction network;

[0011] S4, training and validating the model;

[0012] S5. Evaluate the performance of the model on an independent test set.

[0013] Preferably, in step S1, preprocessing the original audio signal comprises the following steps:

[0014] (1) Apply a bandpass filter to the original audio data, retaining the frequency range between 25 Hz and 400 Hz;

[0015] (2) resampling the filtered audio signal;

[0016] (3) Standardize the audio signal so that its maximum absolute value is 1 and unify the amplitude range of the data;

[0017] (4) The dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0018] Preferably, in step S2, performing traditional audio feature extraction on the audio signal comprises the following steps:

[0019] (1) Select the zero-crossing rate, spectral centroid, spectral bandwidth, spectral roll-off, and Mel-frequency cepstral coefficients of the audio signal.

[0020] (2) Calculate the mean and standard deviation of the above traditional audio features and integrate them to form the final feature vector.

[0021] Preferably, in step S2, extracting high-order spectral features from the audio signal comprises the following steps:

[0022] (1) First, the frequency spectrum of the audio signal is calculated by short-time Fourier transform. The short-time Fourier transform formula is defined as:

[0023]

[0024] Where x(τ) is the input signal and ω(t-τ) is the window function;

[0025] (2) Next, select the frequency range and set the two frequencies to the maximum frequency value minus the sum of all frequency offsets, defined as:

[0026]

[0027] Where max(f) represents the maximum frequency, ∑ofreqs represents the sum of all frequency offsets;

[0028] (3) The spectrum data obtained by short-time Fourier transform is used to calculate the product between each frequency component. The nonlinear interaction between different frequencies in the signal is analyzed by the product of the frequency components. The calculation of the spectrum product is defined as:

[0029] temp(f1,f2)=X(t,f1)·X(t,f2)·X * (f1+f2+f ofreqs ,t)

[0030] Where f1 and f2 are the frequency components within the frequency range determined in the previous step, X(t,f1) is the short-time Fourier transform component of the signal at frequency f1, X(t,f2) is the short-time Fourier transform component of the signal at frequency f2, and X * (f1+f2+f ofreqs ,t) is the conjugate spectrum of the sum of these frequencies;

[0031] (4) Perform normalization processing to ensure the rationality of the coherence results. Finally, the normalized result is defined as:

[0032]

[0033] Where T is the number of samples and temp2 is the normalization factor calculated by the modulus of the spectral product, defined as:

[0034]

[0035] Preferably, in step S3, constructing a dual-branch feature extraction network comprises the following steps:

[0036] (1) A one-dimensional convolutional neural network is used to extract traditional audio features. Block 1 includes a convolution layer, batch normalization, an activation function, and a pooling layer. The traditional feature is represented as A. The feature definition obtained after block 1 is:

[0037] X channel =Maxpool(ReLU(BN1D(Conv1d(A))))

[0038] Among them, Maxpool represents the maximum pooling operation, ReLU represents the rectified linear unit, Conv1d represents the one-dimensional convolution operation, channel represents different channels, and then the weight of the feature is readjusted in the channel dimension through attention, so that the network pays more attention to important features. The feature map obtained after three blocks is X, and the squeeze operation is defined as:

[0039]

[0040] Among them, H and W are the height and width of the feature map, z C is the global average of the cth channel, X C (i, j) is the value of the input feature map at position (i, j) on channel C. The excitation operation consists of two fully connected layers, defined as:

[0041] s=σ(W2·ReLU(W1·z))

[0042] where z is the sum of all z C W1 and W2 are the weight matrices of the two fully connected layers. σ(·) represents the Sigmoid function. The scaling factor s generated by the excitation operation is re-adjusted to the feature map of each channel. The recalibrated feature map on channel C is defined as:

[0043]

[0044] Among them, s C is the scaling factor of the Cth channel, reshapes the output of SENet to adapt to global information processing, and uses a fully connected layer to distribute the mapping of the input features to the fully connected layer to obtain the output feature F′;

[0045] (2) A two-dimensional convolutional neural network is used to extract high-order spectral features. Block 2 includes a convolution layer, batch normalization, an activation function, and a pooling layer. Assume that the traditional feature input is B. The feature map obtained after block 2 is defined as:

[0046] X c ' hannel =Maxpool(ReLU(BN2D(Conv2d(B))))

[0047] Among them, Conv2d represents a two-dimensional convolution operation, channel represents different channels, and the convolution layer is used to further extract high-level features of high-order features, perform batch normalization, and flatten the multi-dimensional feature map into a one-dimensional vector for easy input to the fully connected layer. The fully connected layer maps the flattened features to the target dimension to obtain the output feature F″

[0048] (3) Dual-branch network fusion: feature fusion of the output F′ of branch 1 and the output F″ of branch 2 to obtain F fusion , F fusion It is expressed as:

[0049] F fusion =Cat(F′,F″)

[0050] Apply the Dropout operation to prevent overfitting, and the final output F is defined as:

[0051] F = Dropout (Ffusion ).

[0052] Preferably, in step S4, the constructed model is trained, and the accuracy, recall, precision and F1 score indicators are evaluated on the validation set.

[0053] Compared with the prior art, the present invention has the following advantages:

[0054] 1. The present invention effectively captures the complex frequency relationships and key features in audio signals by extracting high-order spectral features and traditional audio features in parallel. High-order spectral features can reveal the nonlinear frequency coupling in the signal, while traditional audio features capture the overall time-frequency characteristics of the signal. By combining these different types of features, the model can obtain a more comprehensive signal description, thereby improving recognition accuracy.

[0055] 2. The present invention innovatively adopts two groups of convolutional neural networks to process different types of features respectively. Among them, the one-dimensional convolutional neural network for traditional audio features introduces a channel attention mechanism. This architectural design allows the model to dynamically focus on important areas in the audio signal, thereby improving the feature representation capability, reducing errors caused by irrelevant or noisy features, and further improving the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flow chart of the present invention.

[0057] Figure 2 This is a diagram of the audio signal preprocessing process in the present invention.

[0058] Figure 3 It is a schematic diagram of the double-branch network structure in the present invention.

[0059] Figure 4 Comparison of the actual value and predicted value in the test set of the present invention Figure 1 .

[0060] Figure 5 Comparison of the actual value and predicted value in the test set of the present invention Figure 2 . DETAILED DESCRIPTION

[0061] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the present invention is further explained below in conjunction with specific implementation methods.

[0062] Reference Figure 1-3 As shown, a heart sound signal recognition method comprises the following steps:

[0063] S1, preprocessing the original audio signal;

[0064] S2, extracting two sets of features from the audio signal, the two sets of features are traditional audio features and high-order spectrum features;

[0065] S3, model building, constructing a dual-branch feature extraction network;

[0066] S4, training and validating the model;

[0067] S5. Evaluate the performance of the model on an independent test set.

[0068] In step S1, preprocessing the original audio signal includes the following steps:

[0069] (1) Apply a bandpass filter to the original audio data to retain the frequency range between 25 Hz and 400 Hz to remove noise interference;

[0070] (2) resampling the filtered audio signal to reduce the data volume and the computational load of subsequent processing;

[0071] (3) Standardize the audio signal so that its maximum absolute value is 1 and unify the amplitude range of the data;

[0072] (4) The dataset is divided into training set, validation set and test set in a ratio of 8:1:1.

[0073] In step S2, performing traditional audio feature extraction on the audio signal includes the following steps:

[0074] (1) Select the zero-crossing rate, spectral centroid, spectral bandwidth, spectral roll-off, and Mel-frequency cepstral coefficients of the audio signal.

[0075] (2) Calculate the mean and standard deviation of the above traditional audio features and integrate them to form the final feature vector.

[0076] In step S2, extracting high-order spectral features from the audio signal comprises the following steps:

[0077] (1) First, the frequency spectrum of the audio signal is calculated by short-time Fourier transform. The short-time Fourier transform formula is defined as:

[0078]

[0079] Where x(τ) is the input signal and ω(t-τ) is the window function;

[0080] (2) Next, select the frequency range and set the two frequencies to the maximum frequency value minus the sum of all frequency offsets, defined as:

[0081]

[0082] Where max(f) represents the maximum frequency, ∑ofreqs represents the sum of all frequency offsets;

[0083] (3) The spectrum data obtained by short-time Fourier transform is used to calculate the product between each frequency component. The nonlinear interaction between different frequencies in the signal is analyzed by the product of the frequency components. The calculation of the spectrum product is defined as:

[0084] temp(f1,f2)=X(t,f1)·X(t,f2)·X * (f1+f2+f ofreqs ,t)

[0085] Where f1 and f2 are the frequency components within the frequency range determined in the previous step, X(t,f1) is the short-time Fourier transform component of the signal at frequency f1, X(t,f2) is the short-time Fourier transform component of the signal at frequency f2, and X * (f1+f2+f ofreqs ,t) is the conjugate spectrum of the sum of these frequencies;

[0086] (4) Perform normalization processing to ensure the rationality of the coherence results. Finally, the normalized result is defined as:

[0087]

[0088] Where T is the number of samples and temp2 is the normalization factor calculated by the modulus of the spectral product, defined as:

[0089]

[0090] In step S3, constructing a dual-branch feature extraction network includes the following steps:

[0091] (1) A one-dimensional convolutional neural network is used to extract traditional audio features. Block 1 includes a convolution layer, batch normalization, an activation function, and a pooling layer. The traditional feature is represented as A. The feature definition obtained after block 1 is:

[0092] X channel =Maxpool(ReLU(BN1D(Conv1d(A))))

[0093] Among them, Maxpool represents the maximum pooling operation, ReLU represents the rectified linear unit, Conv1d represents the one-dimensional convolution operation, channel represents different channels, and then the weight of the feature is readjusted in the channel dimension through attention, so that the network pays more attention to important features. The feature map obtained after 3 blocks 1 is X, and the squeeze operation is defined as:

[0094]

[0095] Among them, H and W are the height and width of the feature map, z C is the global average of the cth channel, X C (i, j) is the value of the input feature map at position (i, j) on channel C. The excitation operation consists of two fully connected layers, defined as:

[0096] s=σ(W2·ReLU(W1·z))

[0097] where z is the sum of all z C W1 and W2 are the weight matrices of the two fully connected layers. σ(·) represents the Sigmoid function. The scaling factor s generated by the excitation operation is re-adjusted to the feature map of each channel. The recalibrated feature map on channel C is defined as:

[0098]

[0099] Among them, s C is the scaling factor of the Cth channel, reshapes the output of SENet to adapt to global information processing, and uses a fully connected layer to distribute the mapping of the input features to the fully connected layer to obtain the output feature F′;

[0100] (2) A two-dimensional convolutional neural network is used to extract high-order spectral features. Block 2 includes a convolution layer, batch normalization, an activation function, and a pooling layer. Assume that the traditional feature input is B. The feature map obtained after block 2 is defined as:

[0101] X c ' hannel =Maxpool(ReLU(BN2D(Conv2d(B))))

[0102] Among them, Conv2d represents a two-dimensional convolution operation, channel represents different channels, and the convolution layer is used to further extract high-level features of high-order features, perform batch normalization, and flatten the multi-dimensional feature map into a one-dimensional vector for easy input to the fully connected layer. The fully connected layer maps the flattened features to the target dimension to obtain the output feature F″

[0103] (3) Dual-branch network fusion: feature fusion of the output F′ of branch 1 and the output F″ of branch 2 to obtain F fusion , F fusion It is expressed as:

[0104] F fusion =Cat(F′,F″)

[0105] Apply the Dropout operation to prevent overfitting, and the final output F is defined as:

[0106] F = Dropout (F fusion ).

[0107] In step S4, the constructed model is trained, and the accuracy, recall, precision and F1 score indicators are evaluated on the validation set.

[0108] Example

[0109] The present invention selects two public data sets for experiments. Data set A contains 1 normal class and 4 abnormal classes. The four abnormal types are: aortic stenosis, mitral stenosis, mitral regurgitation and mitral valve prolapse, with 200 samples in each class, a total of 1000 samples. Data set B contains 1 normal class and 1 abnormal class, of which the normal class has 183 samples and the abnormal class has 1958 samples. Figure 4 and Figure 5 A comparison chart of the true value and the predicted value in two test sets is shown. After experimental verification, the accuracy of the method proposed in the present invention on the two public data sets reached 99.00% and 99.53%, and the average F1 score reached 99.00% and 98.45% respectively, which proves the effectiveness of the method proposed in the present invention in processing heart sound signals.

[0110] In summary, the present invention extracts traditional audio and high-order features from the preprocessed heart sound signal to comprehensively capture its spectral and nonlinear characteristics. Then, in order to make full use of these features, a dual-branch feature extraction network is designed. One branch focuses on processing the high-order spectral features of the heart sound signal to capture the nonlinear relationship within the signal, and the other branch processes the traditional audio features. By combining these two complementary feature sets, the network can provide a more comprehensive understanding of the signal characteristics. In addition, a channel attention mechanism is introduced in the traditional audio feature branch to further enhance the channel importance of key features, thereby significantly improving the overall performance of the model.

[0111] It is known from common technical knowledge that the present invention can be implemented by other embodiments that do not deviate from its spirit or essential features. Therefore, the above disclosed embodiments are only illustrative in all respects and are not exclusive. All changes within the scope of the present invention or within the scope equivalent to the present invention are included in the present invention.

Claims

1. A heart sound signal recognition method, characterized in that: The following steps are involved: S1, preprocessing the original audio signal; S2, extracting two sets of features from the audio signal, the two sets of features are traditional audio features and high-order spectrum features; S3, model building, constructing a dual-branch feature extraction network; S4, training and validating the model; S5. Evaluate the performance of the model on an independent test set.

2. A heart sound signal recognition method according to claim 1, characterized in that: In step S1, preprocessing the original audio signal includes the following steps: (1) Apply a bandpass filter to the original audio data, retaining the frequency range between 25 Hz and 400 Hz; (2) resampling the filtered audio signal; (3) Standardize the audio signal so that its maximum absolute value is 1 and unify the amplitude range of the data; (4) The dataset is divided into training set, validation set and test set in a ratio of 8:1:

1.

3. A heart sound signal recognition method according to claim 1, characterized in that: In step S2, performing traditional audio feature extraction on the audio signal includes the following steps: (1) Select the zero-crossing rate, spectral centroid, spectral bandwidth, spectral roll-off, and Mel-frequency cepstral coefficients of the audio signal. (2) Calculate the mean and standard deviation of the above traditional audio features and integrate them to form the final feature vector.

4. A heart sound signal recognition method according to claim 1, characterized in that: In step S2, extracting high-order spectral features from the audio signal comprises the following steps: (1) First, the frequency spectrum of the audio signal is calculated by short-time Fourier transform. The short-time Fourier transform formula is defined as: Where x(τ) is the input signal and ω(t-τ) is the window function; (2) Next, select the frequency range and set the two frequencies to the maximum frequency value minus the sum of all frequency offsets, defined as: Where max(f) represents the maximum frequency, ∑ofreqs represents the sum of all frequency offsets; (3) The spectrum data obtained by short-time Fourier transform is used to calculate the product between each frequency component. The nonlinear interaction between different frequencies in the signal is analyzed by the product of the frequency components. The calculation of the spectrum product is defined as: temp(f1,f2)=X(t,f1)·X(t,f2)·X * (f1+f2+f ofreqs ,t) Where f1 and f2 are the frequency components within the frequency range determined in the previous step, X(t,f1) is the short-time Fourier transform component of the signal at frequency f1, X(t,f2) is the short-time Fourier transform component of the signal at frequency f2, and X * (f1+f2+f ofreqs ,t) is the conjugate spectrum of the sum of these frequencies; (4) Perform normalization processing to ensure the rationality of the coherence results. Finally, the normalized result is defined as: Where T is the number of samples and temp2 is the normalization factor calculated by the modulus of the spectral product, defined as:

5. The heart sound signal recognition method according to claim 1, characterized in that: In step S3, constructing a dual-branch feature extraction network includes the following steps: (1) A one-dimensional convolutional neural network is used to extract traditional audio features. Block 1 includes a convolution layer, batch normalization, an activation function, and a pooling layer. The traditional feature is represented as A. The feature definition obtained after block 1 is: X channel =Maxpool(ReLU(BN1D(Conv1d(A)))) Among them, Maxpool represents the maximum pooling operation, ReLU represents the rectified linear unit, Conv1d represents the one-dimensional convolution operation, channel represents different channels, and then the weight of the feature is readjusted in the channel dimension through attention, so that the network pays more attention to important features. The feature map obtained after 3 blocks 1 is X, and the squeeze operation is defined as: Among them, H and W are the height and width of the feature map, z C is the global average of the cth channel, X C (i, j) is the value of the input feature map at position (i, j) on channel C. The excitation operation consists of two fully connected layers, defined as: s=σ(W2·ReLU(W1·z)) where z is the sum of all z C W1 and W2 are the weight matrices of the two fully connected layers. σ(·) represents the Sigmoid function. The scaling factor s generated by the excitation operation is re-adjusted to the feature map of each channel. The recalibrated feature map on channel C is defined as: Among them, s C is the scaling factor of the Cth channel, reshapes the output of SENet to adapt to global information processing, and uses a fully connected layer to distribute the mapping of the input features to the fully connected layer to obtain the output feature F′; (2) A two-dimensional convolutional neural network is used to extract high-order spectral features. Block 2 includes a convolution layer, batch normalization, an activation function, and a pooling layer. Assume that the traditional feature input is B. The feature map obtained after block 2 is defined as: X c ′ hannel =Maxpool(ReLU(BN2D(Conv2d(B)))) Among them, Conv2d represents a two-dimensional convolution operation, channel represents different channels, and the convolution layer is used to further extract high-level features of high-order features, perform batch normalization, and flatten the multi-dimensional feature map into a one-dimensional vector for easy input to the fully connected layer. The fully connected layer maps the flattened features to the target dimension to obtain the output feature F″ (3) Dual-branch network fusion: feature fusion of the output F′ of branch 1 and the output F″ of branch 2 to obtain F fusion , F fusion It is expressed as: F fusion =Cat(F′,F″) Apply the Dropout operation to prevent overfitting, and the final output F is defined as: F=Dropout(F fusion )。 6. A heart sound signal recognition method according to claim 1, characterized in that: In step S4, the constructed model is trained, and the accuracy, recall, precision and F1 score indicators are evaluated on the validation set.

Citation Information

Patent Citations

  • Heart sound signal classification method based on convolutional recurrent neural network

    CN109961017A

  • Heart sound classification method based on convolutional recurrent neural network

    CN115486865A

  • Heart sound classification method based on deep residual neural network

    CN116030829A

  • Heart sound signal classification method

    CN116304785A

  • Heart sound classification algorithm based on bispectral feature extraction and CNN

    CN118245870A

Cited By

  • Industrial field high-frequency voice recognition method and storage medium

    CN120496577A