Abnormal heart sound detection method based on spatio-temporal attention feature fusion model

Abstract

The application relates to the technical field of disease screening, in particular to an abnormal heart sound detection method based on a space-time attention feature fusion model. The abnormal heart sound detection method first acquires heart sound signal data, pre-processes the heart sound signal data, and obtains pre-processed heart sound signal data; each heart sound signal feature data is determined by performing feature extraction on the pre-processed heart sound signal data; each heart sound signal feature data is subjected to multi-source feature fusion and heart sound classification through a constructed CNN-TCN-Attention network model, and a classification result of the heart sound signal data is obtained. The application directly extracts features from the heart sound signal data, utilizes the CNN-TCN-Attention network model to perform heart sound signal classification, effectively improves the accuracy of abnormal heart sound recognition, and is mainly applied to the field of abnormal heart sound detection.

Description

Technical Field

[0001] This invention relates to the field of disease screening technology, specifically to an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model. Background Technology

[0002] Cardiovascular diseases (CVD), including arrhythmias, hypertension, coronary heart disease, heart failure, and cerebrovascular diseases, are among the leading causes of death worldwide. According to the "China Cardiovascular Disease Report 2018" compiled by the National Center for Cardiovascular Diseases, there were approximately 290 million cardiovascular disease patients in 2017; the "China Cardiovascular Health and Disease Report 2020" disclosed that the number of cardiovascular disease patients in China reached approximately 330 million in 2019. In just two years, the number of patients increased by 40 million, demonstrating a rapid rise in the prevalence of cardiovascular disease. Moreover, cardiovascular disease ranks first among all causes of death, far exceeding other causes of death such as cancer and respiratory diseases. Furthermore, the cost of treating cardiovascular diseases is exorbitant. In terms of total cost, the total cost of treatment in 2018 was nearly 600 billion yuan, ranking first among all diseases and accounting for 16.92% of the total cost, equivalent to 0.65% of the GDP that year. In terms of growth, it accounted for 16.38% of the cost in 2012, reaching 287 billion yuan, far exceeding my country's GDP growth rate. Furthermore, the cost of treating cardiovascular diseases increases with age. Therefore, if cardiovascular diseases are not intervened in a timely manner, the treatment costs will increase significantly, which will be a huge disaster, especially for ordinary families.

[0003] In summary, cardiovascular diseases not only have extremely high morbidity, mortality, and treatment costs, but their prevalence continues to rise, seriously threatening human life and safety, and urgently requiring corresponding solutions.

[0004] Currently, stethoscope auscultation is the most common method for diagnosing heart disease. When blood flows through the heart, doctors can use a stethoscope to listen to the sounds produced by the heart valves and determine if any abnormalities are present. However, stethoscope auscultation has many limitations. It relies entirely on the doctor's judgment; the doctor's hearing ability, knowledge, and experience greatly influence the diagnostic results. Therefore, stethoscope auscultation places extremely high demands on doctors, and those with this skill are not many, especially in underdeveloped counties and towns, making diagnostic errors highly likely. Furthermore, due to the inherent characteristics of the human ear, it can only hear sounds within a certain frequency range and cannot distinguish excessively low or high frequencies, often missing important information during auscultation and potentially leading to misdiagnosis. Therefore, it is essential to research a method that can overcome the limitations of auscultation and accurately identify heart disease.

[0005] Heart sounds are weak physiological signals produced by the mechanical movement of the heart. Changes in these signals can indirectly reflect abnormalities in the heart and arteries. When abnormalities occur in the heart and arteries, the heart sound signal can reveal corresponding pathological information. Therefore, analyzing heart sound signals can help differentiate cardiovascular diseases. Heart sound signals are presented in audio or graphical form, recording the time series of heart sounds. They contain frequency information inaudible to the human ear, overcoming the limitations of auscultation and offering convenient storage and analysis. Therefore, using computer tools to analyze heart sound signals allows for accurate classification of normal and abnormal heart sound signals, enabling the detection of cardiac lesions and thus allowing for early intervention and treatment.

[0006] Currently proposed heart sound classification algorithms can be divided into two categories: those including segmentation steps and those not including segmentation steps. One approach preprocesses 361 randomly selected heart sound signals from congenital heart disease patients, performing denoising, envelope extraction, and segmentation localization to obtain each cardiac cycle. Then, S-transform is used to extract feature parameters for each cardiac cycle, and finally, a backpropagation (BP) neural network is used for classification, achieving a recognition rate of 80.4%. Another approach, based on the Challenge 2016 dataset, uses cyclic autocorrelation to segment cardiac cycles and employs bilinear interpolation to extract the power spectral density time-frequency features of each cardiac cycle. These features are then input into a convolutional neural network for classification, achieving an accuracy of 0.8472, demonstrating good overall performance. The above studies focus on heart sound signal classification that includes segmentation steps. However, heart sound segmentation has certain limitations. These limitations stem from the non-stationarity and susceptibility to noise interference inherent in heart sound signals, making accurate segmentation difficult. Most importantly, the fundamental purpose of heart sound recognition is not simply to segment heart sounds, but to accurately distinguish between normal and abnormal heart sounds.

[0007] The shortcomings of existing methods for detecting abnormal heart sounds include:

[0008] (1) Diagnosing heart disease by auscultation with a stethoscope relies entirely on the doctor’s own judgment, and some important information is often missed during auscultation, which may lead to diagnostic errors and low accuracy in detecting abnormal heart sounds.

[0009] (2) The classification method of heart sound segmentation is easily affected by the accuracy of heart sound segmentation. Moreover, heart sound segmentation is generally time-consuming, labor-intensive, and difficult to achieve accurate segmentation. The efficiency and accuracy of abnormal heart sound detection are poor.

[0010] (3) The classification results are generally affected by a variety of factors, and the influence of features is particularly important. However, commonly used features such as time-frequency features, frequency domain features and wavelet packet energy features have some shortcomings, resulting in low accuracy of classification results, that is, low accuracy of abnormal heart sounds recognition. Summary of the Invention

[0011] To address the low accuracy of existing abnormal heart sound detection methods, the present invention aims to provide an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model. The specific technical solution adopted is as follows:

[0012] One embodiment of the present invention provides an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model, the method comprising the following steps:

[0013] Acquire heart sound signal data, preprocess the heart sound signal data, and obtain preprocessed heart sound signal data;

[0014] Feature extraction is performed on the preprocessed heart sound signal data to determine the feature data of each heart sound signal;

[0015] By constructing a CNN-TCN-Attention network model, multi-source feature fusion and heart sound classification are performed on the various heart sound signal feature data to obtain the classification results of the heart sound signal data.

[0016] Further, the step of extracting features from the preprocessed heart sound signal data to determine the feature data of each heart sound signal includes:

[0017] By using wavelet scattering transform, features of the preprocessed heart sound signal data are extracted to obtain two-dimensional heart sound signal features;

[0018] The features of the preprocessed heart sound signal data are extracted using the Mel frequency cepstral coefficient algorithm to obtain the first one-dimensional heart sound signal features;

[0019] The features of the preprocessed heart sound signal data are extracted using Hilbert-Huang transform to obtain the second one-dimensional heart sound signal features;

[0020] The heart sound signal feature data includes the two-dimensional heart sound signal features, the first one-dimensional heart sound signal features, and the second one-dimensional heart sound signal features.

[0021] Furthermore, the process of using the constructed CNN-TCN-Attention network model to perform multi-source feature fusion and heart sound classification on the various heart sound signal feature data to obtain the classification results of the heart sound signal data includes:

[0022] Two-dimensional heart sound signal features are input into a two-dimensional CNN. The first one-dimensional heart sound signal features and the second one-dimensional heart sound signal features are combined and input into a TCN. The outputs of the two-dimensional CNN and the TCN are combined to obtain a multi-source fusion feature vector.

[0023] The multi-source fusion feature vector is input into the Dropout layer, and the Dropout layer is used to process the multi-source fusion feature vector to obtain the feature vector output by the Dropout layer.

[0024] The feature vector output by the Dropout layer is input into the fully connected layer, and the fully connected layer learns the global features of the feature vector output by the Dropout layer to obtain the feature vector output by the fully connected layer.

[0025] The feature vector output by the fully connected layer is input into the attention layer, and the feature vector output by the fully connected layer is weighted by an attention mechanism to obtain the feature vector output by the attention layer.

[0026] The feature vector output by the attention layer is input into the softmax layer, and the feature vector output by the attention layer is subjected to heart sound classification processing to obtain the classification result of the heart sound signal data.

[0027] Further, the preprocessing of the heart sound signal data to obtain preprocessed heart sound signal data includes:

[0028] The heart sound signal data is denoised using Butterworth bandpass filtering to obtain denoised heart sound signal data.

[0029] The denoised heart sound signal data is segmented into segments of equal duration to obtain each heart sound signal segment.

[0030] Normalization is performed on each sampling point in each heart sound signal segment to obtain normalized heart sound signal segments, and the normalized heart sound signal segments are determined as preprocessed heart sound signal data.

[0031] Furthermore, the expression for normalizing each sampling point in each heart sound signal segment is as follows:

[0032]

[0033] In the formula, x ′ (n) is the normalized value corresponding to the nth sampling point in each heart sound signal segment, x(n) is the nth sampling point in each heart sound signal segment, min(x) is the minimum amplitude value in each heart sound signal segment, and max(x) is the maximum amplitude value in each heart sound signal segment.

[0034] The present invention has the following beneficial effects:

[0035] This invention provides an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model. This abnormal heart sound detection method uses computer tools to analyze heart sound signals to identify cardiovascular diseases and achieves accurate classification of heart sound signals, that is, it can accurately identify abnormal heart sound signals. This abnormal heart sound detection method provides a certain reference for clinical diagnosis in medicine. Preprocessing heart sound signal data can effectively avoid the influence of external factors on the data, helping to obtain more accurate heart sound signal data and providing some assistance for accurate classification of heart sound signals. Determining the feature data of each heart sound signal and analyzing the features of the preprocessed heart sound signal data from multiple different perspectives can yield multiple different heart sound signal feature data. Combining the advantages of different features helps to obtain more accurate heart sound classification results. The constructed CNN-TCN-Attention network model, combining convolutional neural networks and temporal convolutional neural networks, helps to fuse multi-source features of heart sound signal feature data obtained from different perspectives, preserving the temporal and frequency characteristics of heart sound signals while processing temporal and frequency domain information. Furthermore, the CNN-TCN-Attention network model introduces an attention mechanism, which helps to adaptively extract features, significantly improving the flexibility and robustness of the network model. By constructing the CNN-TCN-Attention network model to perform multi-source feature fusion and heart sound classification on various heart sound signal feature data, the complementarity between multiple features can be fully explored, helping to improve the accuracy of abnormal heart sound recognition, and is mainly applied in the field of abnormal heart sound detection. Attached Figure Description

[0036] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model according to the present invention;

[0038] Figure 2 This is a schematic diagram of the two sets of wavelet filter banks corresponding to the scattering network in the embodiment of the present invention;

[0039] Figure 3 This is a schematic diagram illustrating the correspondence between Mel-scale frequencies and linear frequencies in an embodiment of the present invention.

[0040] Figure 4 This is a flowchart of MFCC extraction in an embodiment of the present invention;

[0041] Figure 5 This is a schematic diagram of the heart sound signal after EMD decomposition in an embodiment of the present invention.

[0042] Figure 6 The Hilbert marginal spectrum of the heart sound signal in this embodiment of the invention;

[0043] Figure 7 A schematic diagram of the network structure corresponding to the CNN-TCN-Attention network model in this embodiment of the invention. Detailed Implementation

[0044] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the specific implementation methods, structures, features, and effects of the technical solution proposed according to the present invention are described in detail below with reference to the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0046] This embodiment addresses the following application scenario: a heart sound classification method that does not require heart sound segmentation, i.e., directly extracting features from heart sound signal segments to study heart sound classification algorithms. First, the heart sound signal is preprocessed to ensure consistent signal intensity. Then, the advantages of different features are comprehensively utilized to extract multi-source features of the heart sound signal. A multi-source feature fusion model is then used for abnormal heart sound recognition. Specifically, this embodiment provides an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model, such as... Figure 1 As shown, it includes the following steps:

[0047] S1, acquire heart sound signal data, preprocess the heart sound signal data, and obtain preprocessed heart sound signal data. Specific implementation steps may include:

[0048] The first step is to acquire heart sound signal data.

[0049] In this embodiment, an electronic stethoscope can be used to acquire the heart sound signals of the human body under test at a fixed sampling frequency, obtaining discrete heart sound signals. These discrete heart sound signals are then used as the audio signal data for subsequent data analysis. The fixed sampling frequency can be set by the implementer according to specific circumstances and is not specifically limited here. Of course, the implementer can also acquire the heart sound signals of the human body under test using other acquisition devices.

[0050] The second step involves preprocessing the heart sound signal data to obtain preprocessed heart sound signal data. Specific implementation steps may include:

[0051] The first sub-step involves using Butterworth bandpass filtering to denoise the heart sound signal data, thereby obtaining the denoised heart sound signal data.

[0052] It should be noted that during the acquisition process, it is difficult to obtain pure heart sound signals, as they are inevitably affected by various noises, such as circuit noise from the acquisition equipment, shaking of the subject, faint breathing sounds from the subject, ambient noise, and friction noise between the heart sound sensor and the skin. This noise can negatively impact subsequent heart sound signal analysis; therefore, denoising processing of the heart sound signal data obtained in the first step is necessary.

[0053] In this embodiment, to reduce interference from low-frequency and high-frequency noise, a fifth-order Butterworth bandpass filter is used to filter the heart sound signal, with cutoff frequencies set at 25 Hz and 400 Hz. The implementation process of the Butterworth bandpass filter is prior art and is not within the scope of this invention; therefore, it will not be described in detail here.

[0054] The second sub-step involves performing equal-duration segmentation on the denoised heart sound signal data to obtain each heart sound signal segment.

[0055] It should be noted that heart sound signals are quasi-periodic signals. Currently, some researchers perform periodic segmentation of heart sound signal data during the preprocessing stage, dividing the data into multiple cardiac cycles. However, heart sound segmentation has certain limitations. These limitations stem from the fact that the segmentation operation is not only cumbersome, but many methods also require heart sound signal data from the same patient, making it difficult to obtain suitable data. Overall, heart sound segmentation is very challenging. Furthermore, when the durations of the cardiac cycles obtained from segmentation are not entirely identical, it hinders subsequent feature extraction and heart sound signal classification. Based on these shortcomings of heart sound segmentation, this embodiment proposes a simpler method of equal-duration segmentation to replace the complex segmentation operation.

[0056] In this embodiment, when performing equal-duration segmentation of heart sounds, it is necessary to set the length of the heart sound signal segment. According to the set length, the denoised heart sound signal data is segmented into equal-duration segments to obtain each heart sound signal segment. Regarding the setting of the heart sound signal segment length, the heart sound signal can be an approximately periodic signal, with no heart murmur occurring within each cardiac cycle. Therefore, the length of the heart sound signal segment should at least include one cardiac cycle; and since the cardiac cycle is approximately 0.8 seconds, the length of the heart sound signal segment should be greater than 0.8 seconds. Considering the sample size after segmentation, this embodiment sets the length of the heart sound signal segment to 5 seconds. That is, starting from the zero moment of each heart sound signal, the heart sound signal is segmented by sliding across the heart sound signal with a fixed time window of 5 seconds.

[0057] Performing equal-duration segmentation on the denoised heart sound signal data is not only simple to operate and helps improve the efficiency of abnormal heart sound detection, but also does not damage the global information of the cardiac cycle and affect the accuracy of subsequent feature extraction.

[0058] The third sub-step involves normalizing each sampling point in each heart sound signal segment to obtain normalized heart sound signal segments, and then determining the normalized heart sound signal segments as preprocessed heart sound signal data.

[0059] In this embodiment, the intensity of the heart sound signals varies. To ensure the consistency of the heart sound signal intensity and facilitate subsequent data analysis, each sampling point in each heart sound signal segment is normalized. The expression for normalization can be:

[0060]

[0061] In the formula, x ′ (n) represents the normalized value corresponding to the nth sampling point in each heart sound signal segment, x ′ The value of (n) ranges from 0 to 1, x(n) is the nth sampling point in each heart sound signal segment, min(x) is the minimum amplitude value in each heart sound signal segment, and max(x) is the maximum amplitude value in each heart sound signal segment.

[0062] It is worth noting that normalization is the last step in the preprocessing of heart sound signal data. To maintain consistency with the description above, each normalized heart sound signal segment is identified as the preprocessed heart sound signal data.

[0063] Thus, this embodiment has obtained preprocessed heart sound signal data.

[0064] S2, extract features from the preprocessed heart sound signal data to determine the feature data of each heart sound signal. Specific implementation steps may include:

[0065] The first step is to use wavelet scattering transform to extract features from the preprocessed heart sound signal data and obtain two-dimensional heart sound signal features.

[0066] It should be noted that wavelet scattering transform is a novel time-frequency analysis method based on wavelet transform. Through cascaded wavelet decomposition and low-pass filtering averaging, it enables the model to simultaneously possess translation invariance and deformation stationarity. Furthermore, wavelet scattering transform uses both wavelet functions and scaling functions, which helps the model extract both low-frequency and high-frequency information of the signal, ensuring the completeness of the extracted information. Before using wavelet scattering transform to extract features from preprocessed heart sound signal data, the order of the wavelet scattering network needs to be set. Since the energy of the scattering coefficient converges rapidly with increasing network depth, reaching approximately 1% after two layers of network architecture, this embodiment chooses to construct a second-order wavelet scattering network, setting quality factors of 8 and 1 respectively. A schematic diagram of the two sets of wavelet filter banks corresponding to the scattering network is shown below. Figure 2 As shown.

[0067] In this embodiment, based on the characteristics of the preprocessed heart sound signal data, the number of sampling points for wavelet scattering transform is set to 10,000, and the sampling frequency is set to 2000 Hz. After being input into the scattering network, wavelet scattering transform is performed layer by layer to extract the wavelet scattering features of the preprocessed heart sound signal data. The logarithm of the wavelet scattering features is used as the final feature, that is, the logarithm of the wavelet scattering features is determined as the two-dimensional heart sound signal feature. The implementation process of wavelet scattering transform is prior art and is not within the scope of protection of this invention, and will not be described in detail here.

[0068] Based on the set parameters of the preprocessed heart sound signal data in this embodiment, 67 first-order scattering paths and 195 second-order scattering paths can be obtained, which together with the zero-order scattering path constitute 263 scattering paths. Since the wavelet scale is 10, the resulting scattering coefficient feature matrix has a size of scattering path multiplied by the wavelet scale, i.e., 263×10.

[0069] The second step is to use the Mel frequency cepstral coefficient algorithm to extract the features of the preprocessed heart sound signal data and obtain the first one-dimensional heart sound signal features.

[0070] It should be noted that, firstly, the Mel frequency cepstrum coefficient (MFCC) scale is a non-linear frequency scale based on the human ear's sensory judgment of equidistant pitch changes. It can effectively reflect the human ear's ability to recognize sound, meaning MFCC can extract the distinctive components of an audio signal. The heart sound signal acquired in this embodiment shares similar characteristics with the audio signal; both are vibration-generated signals and are unstable. Therefore, MFCC can be used for feature extraction of the heart sound signal. Secondly, the Mel filter is a key step in MFCC. It is a non-linearly distributed filter bank that is sparse in the high-frequency range but dense in the low-frequency range. Therefore, the Mel filter has better recognition capabilities in the low-frequency band, and the main components of the heart sound signal are also in the low-frequency range. Thus, MFCC is suitable for analyzing heart sound signals. Therefore, the Mel frequency cepstrum coefficient has good recognition performance and robustness for heart sound signals.

[0071] In this embodiment, based on the auditory characteristics of the human ear, the conversion relationship between Mel-scale frequencies and linear frequencies can be:

[0072]

[0073] In the formula, Mel(f) is the Mel-scale frequency, the unit of Mel(f) is Hz, f is the linear frequency, and log is the logarithmic function.

[0074] A schematic diagram illustrating the correspondence between Mel-scale frequencies and linear frequencies is shown below. Figure 3 As shown, in Figure 3 In the process of converting linear frequencies to Mel-scale frequencies, the intervals between linear frequencies gradually increase, while the intervals between Mel-scale frequencies are of equal width.

[0075] The flowchart for MFCC extraction is as follows: Figure 4 As shown, in Figure 4 In the process, the MFCC extraction steps include pre-emphasis, framing, windowing, accelerated Fourier transform, modulus square operation, Mel frequency filtering, logarithmic operation, and discrete cosine transform.

[0076] When extracting MFCCs, two aspects need to be considered: first, how to select the MFCC dimension; and second, whether to use framed MFCCs. First, regarding the selection of the MFCC dimension, based on historical experience, a 13-dimensional MFCC is chosen. The most essential information of the original signal is concentrated in certain components of the MFCC, and the first few dimensions of the MFCC have good discriminative performance. Therefore, this embodiment extracts the first 13 dimensions of the MFCC. Next, regarding whether to use framed MFCCs, MFCCs include framed and unframed MFCCs. For each heart sound signal segment, the features corresponding to the unframed MFCC are in vector form, while the features corresponding to the framed MFCC are in matrix form. To determine whether to use framed MFCCs, this embodiment compares the classification results of the two methods. According to the classification results, the unframed MFCC has the best classification effect. Therefore, the unframed 13-dimensional MFCC is finally extracted from the heart sound signal and determined as the first one-dimensional heart sound signal feature. The specific implementation process of MFCC extraction is existing technology and is not within the scope of protection of this invention, so it will not be described in detail here.

[0077] The third step is to use Hilbert-Huang transform to extract features from the preprocessed heart sound signal data and obtain the second one-dimensional heart sound signal features.

[0078] It should be noted that the Hilbert-Huang transform consists of two steps: Empirical Mode Decomposition (EMD) and Hilbert Transform. The specific process is as follows: First, the heart sound signal data is subjected to EMD to obtain a series of Intrinsic Mode Functions (IMFs). Then, the Hilbert transform is performed on these IMFs to obtain the instantaneous frequency of each IMF.

[0079] In this embodiment, the preprocessed heart sound signal data undergoes Hilbert-Huang Transform. A schematic diagram of the heart sound signal after EMD decomposition is shown below. Figure 5 As shown, in Figure 5 In the diagram, the horizontal axis represents time in seconds, and the vertical axis represents amplitude. (a) shows a normal heart sound and its first five components after passing through the EMD; (b) shows the last five components of the normal heart sound EMD and the residual. Figure 5As can be seen, after EMD processing, the complex heart sound signal is decomposed into ten IMF components and one residual component. The amplitudes of the IMF components decrease sequentially from the first to the tenth. The residual component essentially does not contain the main information of the heart sound signal, so it can be ignored in subsequent analysis. Further Hilbert transform of each IMF component yields the instantaneous frequency and amplitude. These instantaneous frequencies and amplitudes can then be used to determine the Hilbert marginal spectrum. The Hilbert marginal spectrum of the heart sound signal is shown below. Figure 6 As shown. In Figure 6 In the diagram, the horizontal axis represents frequency, and the vertical axis represents amplitude. Figure 6 It can display the cumulative amplitude of each frequency component over the entire time domain, i.e., the contribution of each frequency to the energy. It can also be found that the energy is roughly concentrated between 0-200 Hz, with the highest energy between 20-40 Hz. The implementation process of the Hilbert-Huang transform is prior art and is not within the scope of protection of this invention, so it will not be described in detail here.

[0080] Finally, it is worth noting that the various heart sound signal feature data include two-dimensional heart sound signal features, first one-dimensional heart sound signal features, and second one-dimensional heart sound signal features.

[0081] Thus, this embodiment has obtained the characteristic data of each heart sound signal corresponding to the heart sound signal data.

[0082] S3, through the constructed CNN-TCN-Attention network model, performs multi-source feature fusion and heart sound classification on various heart sound signal feature data to obtain the classification results of the heart sound signal data. Specific implementation steps may include:

[0083] The first step is to input the two-dimensional heart sound signal features into a two-dimensional CNN, combine the first and second one-dimensional heart sound signal features and input them into a TCN, and combine the outputs of the two-dimensional CNN and TCN to obtain a multi-source fusion feature vector.

[0084] In this embodiment, considering that one two-dimensional feature and two one-dimensional features were extracted through step S2—namely, the two-dimensional heart sound signal feature, the first one-dimensional heart sound signal feature, and the second one-dimensional heart sound signal feature—a parallel network architecture needs to be constructed. Since feedforward neural networks (CNNs) have proven effective in learning large-scale and high-dimensional data, and temporal convolutional networks (TCNs) outperform sequence analysis models such as Rapid Response Networks (RRNs) in one-dimensional sequence learning, this embodiment combines CNNs and TCNs, that is, combines the outputs of two-dimensional CNNs and TCNs to obtain a multi-source fused feature vector. This multi-source fused feature vector integrates different types of feature data, effectively representing the characteristics of the heart sound signal data, facilitating more accurate heart sound signal classification in the future.

[0085] The CNN module uses two convolutional layers: the first has 16 kernels and the second has 32 kernels, each 15×3 in size, activated by the ReLU activation function. After each convolutional layer, max pooling layers are used for dimensionality reduction, with the two max pooling layers using 2×2 and 4×2 windows respectively. The TCN module uses three residual blocks, each consisting of two convolutional layers. The number of kernels in the three residual blocks are set to 32, 32, and 16 respectively, with each kernel size being 1×3 and dilation factors of 1, 2, and 4 respectively.

[0086] The second step involves inputting the multi-source fusion feature vector into the Dropout layer, processing it, and obtaining the feature vector output by the Dropout layer. This feature vector is then input into a fully connected layer, which learns the global features of the Dropout layer's output feature vector, resulting in the fully connected layer's output feature vector. Finally, the fully connected layer's output feature vector is input into an attention layer, where an attention mechanism is applied to weight it, yielding the attention layer's output feature vector. Finally, the attention layer's output feature vector is input into a softmax layer, where it is used for heart sound classification, resulting in the classification results for the heart sound signal data.

[0087] In this embodiment, the Dropout rate of the Dropout layer can be 0.5. The Dropout layer can be used to reduce the number of neurons, improve computational efficiency, and prevent overfitting. The fully connected layer can be a fully connected layer with 64 neurons, which can be used to learn global features. The attention layer can be used to add an attention mechanism to achieve weighting. The softmax layer can be used to derive the probabilities of normal and abnormal heart sounds and classify them based on the probability. The network structure diagram of the CNN-TCN-Attention network model is shown below. Figure 7 As shown.

[0088] Thus, this embodiment has obtained the classification results corresponding to the collected heart sound signal data.

[0089] The CNN-TCN-Attention network model was established. The training data for the CNN-TCN-Attention network model came from the PhysioNet / CinC 2016 challenge database. 1330 heart sound signals were selected from the PhysioNet / CinC 2016 heart sound dataset, with 655 normal and 655 abnormal heart sound signals. After preprocessing the heart sound signals, 5722 heart sound signal segments with a duration of 5 seconds were obtained. Multi-source fusion feature vectors corresponding to these 5722 heart sound signal segments were extracted and input into the CNN-TCN-Attention network model for classification and recognition training. The CNN-TCN-Attention network model was trained using a binary cross-entropy loss function and the Adam optimizer. The learning rate was set to 0.01, the number of training iterations was 100, and the training and test sets were divided in an 8:2 ratio. One-fifth of the data in the training set was used for validation, and the accuracy (Acc) metric was used to measure the model's classification performance.

[0090] A comparison of classification results before and after the addition of attention. To investigate the impact of the attention mechanism on classification results, validation was performed on both CNN-TCN and CNN-TCN-Attention models. The validation classification results are shown in Table 1:

[0091] Table 1

[0092]

[0093] As shown in Table 1, the classification accuracy of the CNN-TCN network model without the attention mechanism is 93.18%, while the classification accuracy of the CNN-TCN-Attention network model with the attention mechanism is 94.49%. The comparison shows that the classification result with the attention mechanism is better, which indicates that the attention mechanism plays a certain role in the classification process and verifies the effectiveness of the attention mechanism.

[0094] Comparison with existing research results. To verify the effectiveness and accuracy of the CNN-TCN-Attention feature fusion model proposed in this invention, existing methods using deep learning algorithms for classification were selected and compared using the same PhysioNet / CinC2016 dataset. The comparison results are shown in Table 2:

[0095] Table 2

[0096]

[0097] As shown in Table 2, the CNN-TCN-Attention network model in this invention outperforms other classification models, indicating that the CNN-TCN-Attention network model in this invention is very effective for classifying heart sound signals and has a certain competitiveness in heart sound signal classification research.

[0098] This invention provides an abnormal heart sound detection method based on a spatiotemporal attention feature fusion model. This method utilizes computer tools to analyze heart sound signals to identify cardiovascular diseases. Simultaneously, it employs a heart sound classification method that does not require heart sound segmentation, directly extracting features from heart sound segments for signal classification. By utilizing a CNN-TCN-Attention network model to achieve heart sound signal classification, it effectively improves the accuracy of abnormal heart sound detection while enhancing the model's flexibility and robustness, thus significantly improving the accuracy of abnormal heart sound recognition.

[0099] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An abnormal heart sound detection method based on a spatiotemporal attention feature fusion model, characterized in that, Includes the following steps: Acquire heart sound signal data, preprocess the heart sound signal data, and obtain preprocessed heart sound signal data; Feature extraction is performed on the preprocessed heart sound signal data to determine the feature data of each heart sound signal; By constructing a CNN-TCN-Attention network model, multi-source feature fusion and heart sound classification are performed on the various heart sound signal feature data to obtain the classification results of the heart sound signal data; The step of extracting features from the preprocessed heart sound signal data to determine the feature data of each heart sound signal includes: By using wavelet scattering transform, features of the preprocessed heart sound signal data are extracted to obtain two-dimensional heart sound signal features; The features of the preprocessed heart sound signal data are extracted using the Mel frequency cepstral coefficient algorithm to obtain the first one-dimensional heart sound signal features; The features of the preprocessed heart sound signal data are extracted using Hilbert-Huang transform to obtain the second one-dimensional heart sound signal features; The heart sound signal feature data includes the two-dimensional heart sound signal features, the first one-dimensional heart sound signal features, and the second one-dimensional heart sound signal features; The constructed CNN-TCN-Attention network model performs multi-source feature fusion and heart sound classification on the various heart sound signal feature data to obtain the classification results of the heart sound signal data, including: Two-dimensional heart sound signal features are input into a two-dimensional CNN. The first one-dimensional heart sound signal features and the second one-dimensional heart sound signal features are combined and input into a TCN. The outputs of the two-dimensional CNN and the TCN are combined to obtain a multi-source fusion feature vector. The multi-source fusion feature vector is input into the Dropout layer, and the Dropout layer is used to process the multi-source fusion feature vector to obtain the feature vector output by the Dropout layer. The feature vector output by the Dropout layer is input into the fully connected layer, and the fully connected layer learns the global features of the feature vector output by the Dropout layer to obtain the feature vector output by the fully connected layer. The feature vector output by the fully connected layer is input into the attention layer, and the feature vector output by the fully connected layer is weighted by an attention mechanism to obtain the feature vector output by the attention layer. The feature vector output by the attention layer is input into the softmax layer, and the feature vector output by the attention layer is subjected to heart sound classification processing to obtain the classification result of the heart sound signal data.

2. The abnormal heart sound detection method based on a spatiotemporal attention feature fusion model according to claim 1, characterized in that, The preprocessing of the heart sound signal data to obtain preprocessed heart sound signal data includes: The heart sound signal data is denoised using Butterworth bandpass filtering to obtain denoised heart sound signal data. The denoised heart sound signal data is segmented into segments of equal duration to obtain each heart sound signal segment. Normalization is performed on each sampling point in each heart sound signal segment to obtain normalized heart sound signal segments, and the normalized heart sound signal segments are determined as preprocessed heart sound signal data.

3. The abnormal heart sound detection method based on a spatiotemporal attention feature fusion model according to claim 2, characterized in that, The expression for normalizing each sampling point in each heart sound signal segment is as follows: In the formula, This represents the normalized value corresponding to the nth sampling point in each heart sound signal segment. For the nth sampling point in each heart sound signal segment, This represents the minimum amplitude value in each heart sound signal segment. This represents the maximum amplitude value in each heart sound signal segment.

Images