Heart sound anomaly detection method and device, electronic equipment and storage medium

Abstract

The invention provides a heart sound anomaly detection method and device, electronic equipment and a storage medium. An original heart sound signal is acquired and preprocessed into a standard heart sound signal, a plurality of cardiac cycles are divided to form a periodic heart sound signal, and corresponding Mel filter bank coefficient features and Mel frequency cepstral coefficient features are extracted; fusing the Mel filter bank coefficient features and the Mel frequency cepstrum coefficient features, inputting the fused Mel filter bank coefficient features and Mel frequency cepstrum coefficient features into an encoder in a pre-trained MobileNetV2 network, capturing a long-distance dependency relationship in the periodic heart sound signals by using a self-attention mechanism, and outputting a potential space representation vector; and inputting the potential space representation vector into a classifier of the MobileNetV2 network, and outputting a prediction result through a Softmax function. According to the method, the model complexity can be greatly reduced while the model precision is guaranteed, effective features can be extracted while low calculation overhead is kept, and the recognition capability and generalization performance of the model on abnormal heart sounds can be enhanced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of signal processing, and in particular, to a heart sound abnormality detection method, apparatus, electronic device, and storage medium. Background Art

[0002] As one of the important physiological signals of the human vital signs, the heart sound signal can effectively reflect the opening and closing state of the heart valves and the characteristics of blood flow, and is of great significance in the early screening and auxiliary diagnosis of heart diseases. Traditional heart sound auscultation mainly relies on doctors' experience and subjective judgment, and has problems such as low detection accuracy, high missed diagnosis rate, and large individual differences among doctors. Especially in primary medical institutions, due to professional ability and medical equipment limitations, it is difficult to detect heart abnormalities in a timely and accurate manner.

[0003] With the development of digital signal processing and artificial intelligence technologies, heart sound abnormality detection methods based on machine learning and deep learning have gradually become a research hotspot. Existing methods mostly use convolutional neural networks (CNNs), recurrent neural networks (RNNs), or their combined structures to classify heart sound signals. Some models only utilize single information in the time domain or frequency domain, and it is difficult to comprehensively represent the multi-scale characteristics of heart sound signals, resulting in limited classification performance. Due to the complex distribution of heart sound data and low signal-to-noise ratio, traditional networks often show poor robustness when facing different types of murmurs or noise interferences. At the same time, some deep model structures are complex and have a large number of parameters, making it difficult to be efficiently deployed in edge devices or mobile terminals, which limits their clinical popularization and application. Summary of the Invention

[0004] At least one embodiment of the present disclosure provides a heart sound abnormality detection method, apparatus, electronic device, and storage medium. Based on the MobileNetV2 structure and introducing the self-attention mechanism, while ensuring the model accuracy, the model complexity is greatly reduced. It can not only extract effective features while maintaining a low computational cost, but also enhance the model's recognition ability and generalization performance for abnormal heart sounds.

[0005] An embodiment of the present disclosure provides a heart sound abnormality detection method, including:

[0006] Obtain an original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, and divide a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal;

[0007] Extract the Mel filter bank coefficient features and Mel frequency cepstrum coefficient features corresponding to the periodic heart sound signal;

[0008] Fuse the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features, and input the fused features into the encoder of a pre-trained MobileNetV2 network. Then, use the self-attention mechanism to capture the long-range dependencies in the periodic heart sound signal, and output a latent space representation vector.

[0009] Input the latent space representation vector into the classifier of the MobileNetV2 network, and output the prediction result of the heart sound abnormality category through the Softmax function.

[0010] In an alternative implementation, divide the multiple cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal. Specifically, it includes:

[0011] Determine the mean and variance corresponding to each cardiac state, and use the Gaussian distribution to describe the duration probability corresponding to each cardiac state according to the mean and the variance.

[0012] Extract the multi-dimensional signal features corresponding to the standard heart sound signal, input the multi-dimensional signal features as observations into a hidden semi-Markov model, calculate the observation probability corresponding to each cardiac state using a logistic regression model, and correct the observation probability through the Gaussian distribution.

[0013] Decode the optimal state sequence corresponding to the standard heart sound signal according to the duration probability and the observation probability, divide the cardiac cycle according to the optimal state sequence, and extract each cardiac cycle as an independent unit and standardize it to a fixed length to form the periodic heart sound signal.

[0014] In an alternative implementation, fuse the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features based on the following steps:

[0015] During the generation of the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features, use the same frame division parameters to align the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features in the time dimension.

[0016] Stack the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features in the channel dimension to form a two-dimensional stacked feature map.

[0017] Perform zero-mean unit-variance normalization on the two-dimensional stacked feature map, apply the spectral masking technique to randomly mask specific frequency bands, and randomly stretch or compress within a preset range in the time dimension.

[0018] In an alternative implementation, output the latent space representation vector based on the following steps:

[0019] Input the two-dimensional stacked feature map into the encoder, and perform convolution operations by the inverted residual blocks in the MobileNetV2 network;

[0020] Enhance the ability of the MobileNetV2 network to capture long-range dependencies in the periodic heart sound signal through the self-attention layer, and fuse the attention output with the two-dimensional stacked feature map;

[0021] Compress the fused attention output and the two-dimensional stacked feature map into a multi-dimensional latent space representation vector through the global average pooling layer and the fully connected layer.

[0022] In an optional implementation manner, before inputting the latent space representation vector into the classifier of the MobileNetV2 network, the method further includes:

[0023] Input the latent space representation vector into the decoder in the MobileNetV2 network;

[0024] Map the initial dimension of the latent space representation vector through the fully connected layer;

[0025] Gradually restore the temporal resolution of the latent space representation vector through multi-layer transposed convolution processing, and reconstruct the feature representations of the mel filter bank coefficients feature and the mel-frequency cepstral coefficients feature at the output layer.

[0026] In an optional implementation manner, train the MobileNetV2 network based on the following steps:

[0027] Obtain sample heart sound signals to construct a training set, and label each sample heart sound signal in the training set with normal heart sounds, pathological abnormal heart sounds, and noise as labels;

[0028] Use the mean square error to calculate the difference between the original feature and the reconstructed feature to construct a reconstruction loss function, and use the label smoothing classification loss function to calculate the difference between the predicted class and the true class to construct a classification loss function;

[0029] Perform weighted summation on the reconstruction loss function and the classification loss function according to a preset balance coefficient to determine the final loss function;

[0030] Use the training set and the corresponding labels as sample data, and train the MobileNetV2 network under the guidance of the final loss function until the MobileNetV2 network has the ability to classify normal heart sounds, pathological abnormal heart sounds, and noise.

[0031] The embodiments of the present disclosure also provide a heart sound abnormality detection device, including:

[0032] A signal processing module, configured to obtain an original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, and divide a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal;

[0033] A feature extraction module, configured to extract Mel filter bank coefficient features and Mel frequency cepstral coefficient features corresponding to the periodic heart sound signal;

[0034] An encoding module, configured to fuse the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features and input them into an encoder in a pre-trained MobileNetV2 network, and capture long-range dependencies in the periodic heart sound signal by using a self-attention mechanism, and output a latent space representation vector;

[0035] A classification module, configured to input the latent space representation vector into a classifier of the MobileNetV2 network, and output a prediction result of a heart sound abnormality category through a Softmax function.

[0036] An embodiment of the present disclosure further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the above-mentioned heart sound abnormality detection method, or steps in any possible implementation manner of the above-mentioned heart sound abnormality detection method are executed.

[0037] An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the above-mentioned heart sound abnormality detection method, or steps in any possible implementation manner of the above-mentioned heart sound abnormality detection method are executed.

[0038] An embodiment of the present disclosure further provides a computer program product, including a computer program / instructions. When the computer program and instructions are executed by a processor, the above-mentioned heart sound abnormality detection method, or steps in any possible implementation manner of the above-mentioned heart sound abnormality detection method are implemented.

[0039] A heart sound abnormality detection method, device, electronic device and storage medium provided by an embodiment of the present disclosure obtain an original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, divide a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal; extract Mel filter bank coefficient features and Mel frequency cepstral coefficient features corresponding to the periodic heart sound signal; fuse the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features and input them into an encoder in a pre-trained MobileNetV2 network, and use a self-attention mechanism to capture long-range dependencies in the periodic heart sound signal, and output a latent space representation vector; input the latent space representation vector into a classifier of the MobileNetV2 network, and output a prediction result of the heart sound abnormality category through a Softmax function. Based on the MobileNetV2 structure and introducing a self-attention mechanism, while ensuring the model accuracy, the model complexity is greatly reduced. It can not only extract effective features while maintaining a low computational cost, but also enhance the model's recognition ability and generalization performance for abnormal heart sounds.

[0040] To make the above objects, features and advantages of the present disclosure more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] To more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required to be used in the embodiments. The drawings here are incorporated into the specification and constitute a part of this specification. These drawings show embodiments consistent with the present disclosure and, together with the specification, are used to explain the technical solutions of the present disclosure. It should be understood that the following drawings only show some embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0042] Figure 1 Shows a flowchart of a heart sound abnormality detection method provided by an embodiment of the present disclosure;

[0043] Figure 2 Shows a flowchart of a training method for a MobileNetV2 network provided by an embodiment of the present disclosure;

[0044] Figure 3 Shows a schematic diagram of a heart sound abnormality detection device provided by an embodiment of the present disclosure;

[0045] Figure 4 Shows a schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are only some, but not all, of the embodiments of the present disclosure. The components of the embodiments of the present disclosure generally described and illustrated in the figures herein can be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but merely represents selected embodiments of the present disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative efforts fall within the scope of the present disclosure.

[0047] It should be noted that like reference numerals and letters denote like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.

[0048] As used herein, the term "and / or" merely describes an association relationship and indicates that three relationships may exist. For example, A and / or B may represent the following three cases: A alone, both A and B present, and B alone. Additionally, the term "at least one" as used herein means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C may represent any one or more elements selected from the set consisting of A, B, and C.

[0049] Through research, it has been found that with the development of digital signal processing and artificial intelligence technologies, heart sound anomaly detection methods based on machine learning and deep learning have gradually become a research hotspot. Existing methods mostly use convolutional neural networks (CNNs), recurrent neural networks (RNNs), or their combined structures to classify heart sound signals. Some models only utilize single information in the time domain or frequency domain, making it difficult to comprehensively represent the multi-scale features of heart sound signals, resulting in limited classification performance. Due to the complex distribution of heart sound data and low signal-to-noise ratio, traditional networks often exhibit poor robustness when facing different types of murmurs or noise interferences. At the same time, some deep model structures are complex and have a large number of parameters, making it difficult to be efficiently deployed on edge devices or mobile terminals, which limits their clinical popularization and application.

[0050] Based on the above research, the present disclosure provides a method, apparatus, electronic device, and storage medium for detecting abnormal heart sounds. The method includes: obtaining an original heart sound signal, preprocessing the original heart sound signal into a standard heart sound signal, dividing a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal; extracting Mel filter bank coefficient features and Mel frequency cepstral coefficient features corresponding to the periodic heart sound signal; fusing the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features and inputting the fused features into an encoder in a pre-trained MobileNetV2 network, and using a self-attention mechanism to capture long-range dependence relationships in the periodic heart sound signal to output a latent space representation vector; inputting the latent space representation vector into a classifier of the MobileNetV2 network, and outputting a prediction result of the abnormal heart sound category through a Softmax function. Based on the MobileNetV2 structure and introducing a self-attention mechanism, while ensuring the model accuracy, the model complexity is greatly reduced. It can not only extract effective features while maintaining a low computational cost, but also enhance the model's recognition ability and generalization performance for abnormal heart sounds.

[0051] To facilitate the understanding of this embodiment, first, a method for detecting abnormal heart sounds disclosed in the embodiments of the present disclosure will be introduced in detail. The execution subject of the method for detecting abnormal heart sounds provided in the embodiments of the present disclosure is generally a computer device with certain computing capabilities, such as a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the method for detecting abnormal heart sounds may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0052] See Figure 1 As shown in the flowchart of a method for detecting abnormal heart sounds provided in the embodiments of the present disclosure, the method includes steps S101 to S104, where:

[0053] S101. Obtain an original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, and divide a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal.

[0054] In a specific implementation, an electronic stethoscope or a high-sensitivity microphone is used to collect the original heart sound signal of the subject's chest area. The original heart sound signal can be one-dimensional time series data, and the sampling frequency can be set to 1000 Hz or 2000 Hz, and the sampling time is not less than 5 seconds to ensure that multiple complete cardiac cycles are included. Then, denoising and normalization operations are performed on the original heart sound signal to make it have a clearer heart sound structure feature.

[0055] Here, a Butterworth filter or a wavelet denoising method is adopted to limit the signal frequency within the range of typical heart sound frequencies (such as 20 Hz to 600 Hz) to remove high-frequency interference and low-frequency drift. A linear transformation or Z-score normalization is performed on the overall amplitude of the signal to make its mean 0 and standard deviation 1, and the possible pseudo-signals at the beginning and end are removed, such as the shock wave at the moment of device startup or the environmental background sound segment.

[0056] Exemplarily, first, a second-order Butterworth filter is used to remove the noise below 25 Hz and above 400 Hz to eliminate environmental and high-frequency interference. Then, the signal is resampled to the standard sampling rate of 8 kHz to ensure data consistency. The amplitude of the downsampled signal is normalized and adjusted to the range of [-1, 1] to enhance the comparability between different signals.

[0057] Furthermore, the mean and variance corresponding to each cardiac state are determined, and the duration probability corresponding to each cardiac state is described by a Gaussian distribution according to the mean and variance; the multi-dimensional signal features corresponding to the standard heart sound signal are extracted, and the multi-dimensional signal features are used as observation values to be input into the hidden semi-Markov model. A logistic regression model is used to calculate the observation probability corresponding to each cardiac state, and the observation probability is corrected by a Gaussian distribution; the optimal state sequence corresponding to the standard heart sound signal is decoded according to the duration probability and the observation probability, the cardiac cycle is divided according to the optimal state sequence, and each cardiac cycle is extracted as an independent unit and normalized to a fixed length to form a periodic heart sound signal.

[0058] In a specific implementation, to convert a standard heart sound signal into multiple independent periodic units, it is necessary to identify the start and end points of each cardiac cycle, establish a heart sound state sequence model composed of four typical cardiac states: S1, systolic phase, S2, and diastolic phase, statistically calculate the mean and standard deviation of the duration of each state in a large number of training samples, construct a Gaussian distribution model, obtain the prior duration probability of each state, extract multi-dimensional features such as short-time energy, instantaneous frequency, and spectral entropy from the standard heart sound signal, input the multi-dimensional features into a hidden semi-Markov model (HSMM), and use a method combining the forward-backward algorithm and the Viterbi decoding algorithm to output the optimal state label sequence corresponding to each sampling point. Taking the interval between two consecutive S1s as a cardiac cycle, extract the periodic segment and uniformly standardize it to a fixed length (such as 1024 frames) through linear interpolation or time stretching and compression methods to ensure the consistency of the subsequent model training input.

[0059] Here, according to the principles of heart sound auscultation, the heart sound signal is divided into four states: S1, systolic phase, S2, and diastolic phase. The hidden semi-Markov model combined with the dynamic programming algorithm is used to perform periodic segmentation on the heart sound signal. This algorithm first models the heart sound signal as a Markov process with four physiological states (S1, systolic phase, S2, and diastolic phase), and uses the Gaussian distribution to accurately model the duration characteristics of each state.

[0060] Among them, the algorithm extracts multi-dimensional features (mainly including homomorphic envelope and Hilbert envelope) from the signal as the observation sequence, and then combines the logistic regression model and Gaussian correction to calculate the observation probability. Finally, by integrating the state transition constraints, observation probability, and duration probability, the improved Viterbi algorithm can calculate the most likely state sequence, realizing the accurate positioning and segmentation of key heart sound events such as S1 and S2.

[0061] Specifically, for each state (S1, systolic phase, S2, and diastolic phase), calculate its mean μ and variance σ 2 , for example, the mean duration of S1 is mean_s1 and the variance is std_s1. The following Gaussian distribution formula is used to describe the probability of the state duration:

[0062]

[0063] Among them, P(duration|μ,σ 2 ) represents the duration probability; μ represents the mean; σ 2 represents the variance.

[0064] It should be noted that according to the actual data and physiological laws of the heart sound signal, a reasonable duration range (minimum and maximum values) is set for each state. For example, the maximum / minimum duration of S1 is the mean ± 3 times the standard deviation. In this way, it can be ensured that the duration of each state conforms to the physiological laws and is reasonably restricted in the model.

[0065] Furthermore, the hidden semi-Markov model combines the duration probability of each state with the signal features for calculation. Using the signal features (Hilbert envelope, power spectral density) as the observed values to input into the hidden semi-Markov model, a logistic regression model is used to calculate the observation probability of each state, and these probabilities are corrected by the Gaussian distribution to ensure that the features conform to the Gaussian distribution:

[0066]

[0067] Among them, P obs (x t ,s i ) represents the observation probability; x t represents the observed value at time t; s i represents a specific heart sound state; P logit (x t ,s i ) represents the observation probability output by the logistic regression model, indicating the probability that the current observed value belongs to a certain state; P gauss (x t ) represents the Gaussian correction term; π i represents the initial state probability.

[0068] After that, the improved Viterbi algorithm is used to decode the optimal state sequence of the heart sound signal, and dynamic programming calculation is performed in combination with the state duration and the observation probability. First, calculate the initial probability of each state at time 0, in combination with the initial state probability π and the observation probability:

[0069] δ[0,:] = log(π) + log(P obs (x0,s i ))

[0070] Among them, δ[0,:] represents the initial probability of each state at time 0; P obs (x t ,s i ) represents the observation probability.

[0071] Finally, for each subsequent time point, calculate the optimal probability of each state and update the backtracking matrix:

[0072]

[0073] Among them, represents the state transition probability, P obs (x t , s j ) represents the observation probability; P(Duration|μ, σ 2 ) represents the duration probability. Starting from the last time point, the optimal state sequence is gradually recovered using the backtracking matrix, thereby realizing the division of the cycle components of heart sounds. Each complete cardiac cycle (from one S1 to before the next S1) is extracted as a separate unit and then normalized to a fixed length (2 seconds).

[0074] S102. Extract the Mel filter bank coefficient features and Mel frequency cepstral coefficient features corresponding to the periodic heart sound signal.

[0075] In a specific implementation, two complementary features, namely Mel frequency cepstral coefficient features (MFCC) and Mel filter bank coefficients (FBANK), are combined to comprehensively capture the acoustic characteristics of heart sound signals. These two types of features focus on different aspects of the spectral structure, and more comprehensive heart sound features can be obtained through fusion. The periodic heart sound signal is divided into multiple frame segments with a fixed length and overlap to extract local features in the time domain and frequency domain subsequently. Exemplarily, the frame length is 25 milliseconds (corresponding to approximately 25 sampling points at a sampling rate of 1000 Hz); the window function used is the Hamming window or the Blackman window to reduce spectral leakage.

[0076] Here, first, an emphasis filter is applied to the preprocessed heart sound signal to enhance the high-frequency part and compensate for the natural attenuation of the vocalization system; the signal is framed, with each frame having a length of 25 ms and an overlap of 15 ms (step size 10 ms) between adjacent frames to ensure time continuity; a Hamming window function is applied to each frame signal to reduce spectral leakage; the fast Fourier transform is performed to convert the time-domain signal into a frequency-domain representation; the power spectrum is calculated to reflect the energy distribution at different frequencies; a Mel filter bank is applied to simulate the non-linear perception of different frequencies by the human auditory system; the logarithm is taken of the filter output to compress the dynamic range; the discrete cosine transform (DCT) is performed to remove the correlation between features; the first 40 DCT coefficients are retained as the final MFCC features, and the higher-order coefficients are discarded to reduce the influence of noise.

[0077] Among them, the discrete cosine transform (DCT) is performed on the sequence of logarithmic Mel filter bank energy values to convert it into the cepstral representation of the spectral envelope, and the coefficients of the first 13 dimensions or other specified dimensions are taken as the final MFCC features to capture the overall contour and pronunciation structure information of the audio.

[0078] Here, zero-padding is performed on each frame of the signal and fast Fourier transform is carried out to convert it from the time domain to the frequency domain, obtaining the amplitude spectrum or power spectrum corresponding to this frame. In the frequency domain, a set of triangular filters is used to simulate the resolution ability of the human auditory system to construct a Mel-scale filter bank with evenly distributed center frequencies on the Mel scale. The power spectrum is multiplied by the response of each filter respectively, and weighted integration is performed on the product to obtain the energy value corresponding to each filter. Taking the logarithm of each energy value, the logarithmic Mel filter bank energy value, that is, the FBANK feature, is obtained.

[0079] Among them, the filter bank energy feature retains more original spectral details, is more sensitive to short-term spectral changes and local spectral structures, and can capture subtle spectral changes that MFCC may ignore. The extraction process is as follows: Similar to the preprocessing of MFCC, frame segmentation, windowing, and Fourier transform are performed on the heart sound signal; calculate the power spectral density, representing the energy distribution per unit frequency; apply a 64-channel Mel filter bank to cover the main frequency band of heart sounds from 25 to 400 Hz; sum the outputs of each filter to obtain the energy of this frequency band; take the logarithm of the filter energy to obtain the 64-dimensional FBANK feature; perform median filtering to reduce random fluctuations and noise interference.

[0080] Optionally, mean-variance normalization is applied to the extracted FBANK and MFCC features respectively to standardize their distributions, and sliding window average filtering or first-order difference processing can be further used to improve robustness.

[0081] Furthermore, the FBANK feature and MFCC feature corresponding to each frame are respectively saved as two-dimensional feature matrices (time × feature dimension), and subsequently, the FBANK and MFCC features are stacked along the channel dimension to form a unified input feature map for use by the neural network.

[0082] S103. After fusing the Mel filter bank coefficient feature and the Mel-frequency cepstral coefficient feature, input them into the encoder in the pre-trained MobileNetV2 network, and use the self-attention mechanism to capture the long-range dependencies in the periodic heart sound signal, and output the latent space representation vector.

[0083] In a specific implementation, for the extracted Mel filter bank coefficient feature (FBANK) and Mel-frequency cepstral coefficient feature (MFCC), first ensure their alignment in the time dimension, that is, use the same frame length and frame shift parameters for feature extraction. Then perform a fusion operation in the channel dimension, stack the two types of features into two channels to form a three-dimensional feature map in the form of [number of time steps × frequency dimension × number of channels] as the input tensor for subsequent network processing.

[0084] Specifically, during the generation of Mel filter bank coefficient features and Mel-frequency cepstral coefficient features, the same framing parameters are used to align the Mel filter bank coefficient features and Mel-frequency cepstral coefficient features in the time dimension; the Mel filter bank coefficient features and Mel-frequency cepstral coefficient features are stacked in the channel dimension to form a two-dimensional stacked feature map; zero-mean unit-variance normalization is performed on the two-dimensional stacked feature map, the spectral masking technique is applied to randomly mask specific frequency bands, and random stretching or compression within a preset range is performed in the time dimension.

[0085] Here, time alignment is first performed to ensure that the MFCC and FBANK features are precisely aligned in the time dimension, and the same framing parameters (window length 25ms, step size 10ms) are used for both features; then, dimensional matching is carried out. MFCC provides 40-dimensional features and FBANK provides 64-dimensional features, a total of 104 dimensions; finally, MFCC and FBANK are stacked in the channel dimension to form a two-dimensional feature map of 104×T, where T is the number of time frames (usually 200, corresponding to 2 seconds of heart sound).

[0086] Optionally, zero-mean unit-variance normalization is performed on the two-dimensional stacked feature map according to the following formula to improve training stability:

[0087]

[0088] where μ represents the feature mean, σ represents the standard deviation, and ε is a small constant (e -10 ) to prevent division by zero.

[0089] Optionally, the spectral masking technique is applied to randomly mask specific frequency bands (about 15%) to enhance the model's robustness to missing features; random stretching or compression (±10%) is performed in the time dimension to simulate the changes in heart sound features at different heart rates.

[0090] Furthermore, the two-dimensional stacked feature map is input into the encoder, and convolution operations are performed by the inverted residual blocks in the MobileNetV2 network; the self-attention layer is used to enhance the ability of the MobileNetV2 network to capture long-range dependencies in the periodic heart sound signal, and the attention output is fused with the two-dimensional stacked feature map; the fused attention output and the two-dimensional stacked feature map are compressed into a multi-dimensional latent space representation vector through the global average pooling layer and the fully connected layer.

[0091] In a specific implementation, the above-mentioned fused features are input into the encoder part of the pre-trained MobileNetV2 network. The encoder of MobileNetV2 consists of multiple inverted residual structures, and each structure includes a 1×1 channel expansion convolution (pointwise convolution), a 3×3 depthwise separable convolution, and a 1×1 channel compression convolution.

[0092] Here, at the end of the MobileNetV2 encoder, an embedded self-attention mechanism is used to enhance the network's ability to model long-term dependencies. The output feature map of the encoder is flattened into a two-dimensional sequence in the form of [number of time steps, feature dimension], and query (Q), key (K), and value (V) matrices are constructed and linearly transformed. Then, the attention weight matrix is calculated based on the query matrix, key matrix, and value matrix:

[0093]

[0094] where Q, K, and V are the query matrix, key matrix, and value matrix respectively, and d k is the scaling factor used to prevent the problem of gradient vanishing; Q = W Q X, K = W K X, V = W V X, where W Q 、W K 、W V are learnable weight matrices.

[0095] Finally, the attention output is combined with the original features: X att = X + γ·Attention(Q, K, V), where γ is a learnable parameter with an initial value of 0 and gradually increases during training. The self-attention mechanism enables the model to adaptively focus on critical moments in the heart sound signal, such as systolic murmurs or abnormal S1 / S2, significantly improving the sensitivity to pathological features. The last part of the encoder is the global average pooling layer and the fully connected layer, which compress the features into a 128-dimensional latent space. This low-dimensional representation contains the core features of the heart sound signal, which are used for subsequent classification tasks and are also the input to the decoder.

[0096] As a possible implementation, the latent space representation vector is input into the decoder in the MobileNetV2 network; the initial dimension of the latent space representation vector is mapped through the fully connected layer; the temporal resolution of the latent space representation vector is gradually restored through multi-layer transposed convolution processing, and the feature representations of the mel filter bank coefficients and mel-frequency cepstral coefficients are reconstructed at the output layer.

[0097] Here, for the decoder in the MobileNetV2 network, the input layer receives a 128-dimensional latent vector, the fully connected layer maps the latent vector back to the initial dimension, the temporal resolution of the features is gradually restored through multi-layer transposed convolution, and a dual feature representation of 104×T dimensions is reconstructed at the output layer.

[0098] S104. The latent space representation vector is input into the classifier of the MobileNetV2 network, and the prediction results of the heart sound abnormality categories are output through the Softmax function.

[0099] In a specific implementation, the input layer receives the latent feature vector representation of the heart sound signal from the encoder. The intermediate layer adopts a multi-layer fully connected network structure and gradually extracts the complex relationships between features through non-linear transformations. To prevent overfitting and improve the generalization ability of the model, an activation function and Dropout regularization are equipped after each hidden layer. The output layer uses the Softmax activation function to map the final features into the probability distributions of three categories, facilitating subsequent decision-making analysis.

[0100] Here, the heart sound signal processed by the MobileNetV2 encoder and the self-attention module is compressed into a fixed-length latent space representation vector. This vector synthesizes the time-frequency features, structural dependencies, and periodic patterns of the input heart sound signal, and has good separability and discriminability. The latent representation vector is input into the classifier module in the MobileNetV2 network. The classifier includes one or more fully connected layers for further transforming and compressing the latent vector; an output layer, fully connected to the final classification dimension (e.g., 3 categories: normal heart sound, pathological abnormal heart sound, noise), inputs the above output vector into the Softmax function, calculates the prediction probability of each category, and selects the category with the highest prediction probability as the final prediction result of the heart sound abnormality category.

[0101] As a possible implementation, the difference between the original features and the reconstructed features can be calculated through the reconstruction error analysis module to quantify the reconstruction quality of the autoencoder. By statistically analyzing the reconstruction error distributions of different category samples in the training set, an error threshold model is established. In a well-trained model, normal heart sounds usually have a lower reconstruction error, pathological abnormal heart sounds have a medium reconstruction error, and noise samples usually exhibit significantly higher reconstruction errors due to their irregularity and unstructured characteristics. This pattern provides important clues for distinguishing noise from real abnormalities.

[0102] A heart sound abnormality detection method provided by an embodiment of the present disclosure obtains an original heart sound signal, preprocesses the original heart sound signal into a standard heart sound signal, divides a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal; extracts Mel filter bank coefficient features and Mel frequency cepstrum coefficient features corresponding to the periodic heart sound signal; fuses the Mel filter bank coefficient features and the Mel frequency cepstrum coefficient features and inputs them into an encoder in a pre-trained MobileNetV2 network, and uses a self-attention mechanism to capture long-range dependencies in the periodic heart sound signal, and outputs a latent space representation vector; inputs the latent space representation vector into a classifier of the MobileNetV2 network, and outputs a prediction result of the heart sound abnormality category through a Softmax function. Based on the MobileNetV2 structure and introducing a self-attention mechanism, while ensuring the model accuracy, the model complexity is greatly reduced. It can not only extract effective features while maintaining a low computational cost, but also enhance the model's recognition ability and generalization performance for abnormal heart sounds.

[0103] See Figure 2 As shown, it is a flowchart of a training method for a MobileNetV2 network provided by an embodiment of the present disclosure. The method includes steps S201 to S204, where:

[0104] S201. Obtain sample heart sound signals to construct a training set, and label each sample heart sound signal in the training set with normal heart sounds, pathological abnormal heart sounds, and noise as labels.

[0105] S202. Calculate the difference between the original features and the reconstructed features using mean square error to construct a reconstruction loss function, and calculate the difference between the predicted category and the true category using a label smoothing classification loss function to construct a classification loss function.

[0106] S203. Perform weighted summation on the reconstruction loss function and the classification loss function according to a preset balance coefficient to determine the final loss function.

[0107] S204. Use the training set and the corresponding labels as sample data, and train the MobileNetV2 network under the guidance of the final loss function until the MobileNetV2 network has the ability to classify normal heart sounds, pathological abnormal heart sounds, and noise.

[0108] In specific implementation, a training set is constructed through publicly available heart sound datasets (such as PhysioNet Challenge, PASCAL heart sound dataset) and clinically collected heart sound samples. The training set includes three types of samples: normal heart sounds, pathological abnormal heart sounds, and noise samples. To ensure the generalization ability of the model, the dataset contains samples of different ages, different health conditions, and different acquisition conditions.

[0109] Here, the backbone network of the self-attention enhanced MobileNetV2 autoencoder adopts an improved MobileNetV2 architecture, and a self-attention mechanism is introduced to enhance the feature capture ability. The encoder part consists of multiple inverted residual blocks, and each block includes a 1×1 convolution to expand the channels, a depthwise separable 3×3 convolution, and a 1×1 convolution to compress the channels. The decoder part adopts a structure symmetric to the encoder, and gradually restores the signal resolution through transposed convolution. The classifier adopts a multi-layer fully connected network, including two hidden layers and an output layer.

[0110] Among them, the loss function required for model training includes two parts, and the final loss function is the weighted sum of the two loss functions: Reconstruction loss: The mean square error is used to calculate the difference between the original features and the reconstructed features, ensuring that the autoencoder can effectively reconstruct the input signal. Classification loss: The label smoothing classification loss function is used to calculate the difference between the predicted class and the true class. Final loss function: L total = αL recon + βL class , where α and β are balance coefficients, which are dynamically adjusted according to the performance of the validation set.

[0111] Furthermore, optimizers such as ADAM and SGD can be used, and fixed learning rates or learning rate strategies such as exponential decay and cosine annealing can be used for model training.

[0112] As a possible implementation, overall performance metrics are adopted: including evaluation metrics such as accuracy, sensitivity, specificity, and confusion matrix, which are used to evaluate the detection performance of the model. The high-dimensional latent features are mapped to a two-dimensional plane through dimensionality reduction technology to show the distribution relationship of different types of heart sound samples. The system adopts the t-SNE algorithm, and this non-linear dimensionality reduction method can preserve the local structural characteristics of the data. In the generated scatter plot, each point represents a heart sound sample, and different colors identify different classes. Samples of the same class form a tight cluster, and there are obvious boundaries between different classes. By analyzing the clustering pattern, the classification ability of the model can be evaluated: clear class boundaries indicate that the model has good discrimination ability; overlapping samples in the boundary area are usually critical cases that are difficult to classify and require special attention from doctors; some abnormal samples may form secondary clusters, suggesting the existence of multiple pathological types, providing a basis for further fine classification of classes.

[0113] Those skilled in the art can understand that in the above method of the specific implementation manner, the writing order of each step does not mean a strict execution order and does not impose any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0114] A heart sound abnormality detection method provided by an embodiment of the present disclosure includes: acquiring an original heart sound signal, preprocessing the original heart sound signal into a standard heart sound signal, dividing a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal; extracting Mel filter bank coefficient features and Mel frequency cepstrum coefficient features corresponding to the periodic heart sound signal; fusing the Mel filter bank coefficient features and the Mel frequency cepstrum coefficient features and inputting the fused features into an encoder in a pre-trained MobileNetV2 network, and using a self-attention mechanism to capture long-range dependencies in the periodic heart sound signal to output a latent space representation vector; inputting the latent space representation vector into a classifier of the MobileNetV2 network, and outputting a prediction result of a heart sound abnormality category through a Softmax function. Based on the MobileNetV2 structure and introducing a self-attention mechanism, while ensuring the model accuracy, the model complexity is greatly reduced. It can not only extract effective features while maintaining a low computational cost, but also enhance the model's recognition ability and generalization performance for abnormal heart sounds.

[0115] Based on the same inventive concept, an embodiment of the present disclosure also provides a heart sound abnormality detection device corresponding to the heart sound abnormality detection method. Since the principle of solving problems by the device in the embodiment of the present disclosure is similar to the above heart sound abnormality detection method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and repeated parts will not be described again.

[0116] Please refer to Figure 3 , FIG. Heart sound abnormality detection is a schematic diagram of a heart sound abnormality detection device provided by an embodiment of the present disclosure. As Figure 3 shown in, the heart sound abnormality detection device 300 provided by an embodiment of the present disclosure includes:

[0117] A signal processing module 310, configured to acquire an original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, and divide a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal.

[0118] A feature extraction module 320, configured to extract Mel filter bank coefficient features and Mel frequency cepstrum coefficient features corresponding to the periodic heart sound signal.

[0119] An encoding module 330 is configured to fuse the Mel filter bank coefficient features and the Mel-frequency cepstral coefficient features and input the fused features into an encoder in a pre-trained MobileNetV2 network, and capture long-range dependencies in the periodic heart sound signal by using a self-attention mechanism, and output a latent space representation vector.

[0120] A classification module 340 is configured to input the latent space representation vector into a classifier of the MobileNetV2 network, and output a prediction result of a heart sound anomaly category through a Softmax function.

[0121] Descriptions of the processing flows of the various modules in the device and the interaction flows between the various modules may refer to the relevant descriptions in the above method embodiments, and will not be elaborated here.

[0122] A heart sound anomaly detection device provided by an embodiment of the present disclosure obtains an original heart sound signal, preprocesses the original heart sound signal into a standard heart sound signal, divides a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal; extracts Mel filter bank coefficient features and Mel-frequency cepstral coefficient features corresponding to the periodic heart sound signal; fuses the Mel filter bank coefficient features and the Mel-frequency cepstral coefficient features and inputs the fused features into an encoder in a pre-trained MobileNetV2 network, and captures long-range dependencies in the periodic heart sound signal by using a self-attention mechanism, and outputs a latent space representation vector; inputs the latent space representation vector into a classifier of the MobileNetV2 network, and outputs a prediction result of a heart sound anomaly category through a Softmax function. Based on the MobileNetV2 structure and introducing a self-attention mechanism, while ensuring the model accuracy, the model complexity is greatly reduced. It can not only extract effective features while maintaining a low computational cost, but also enhance the model's recognition ability and generalization performance for abnormal heart sounds.

[0123] Corresponding to Figure 1 the heart sound anomaly detection method in Figure 4 as shown, which is a schematic structural diagram of an electronic device 400 provided by an embodiment of the present disclosure, including:

[0124] A processor 41, a memory 42, and a bus 43; the memory 42 is used to store execution instructions, including an internal memory 421 and an external memory 422; here, the internal memory 421 is also called a main memory, and is used to temporarily store operation data in the processor 41 and data exchanged with the external memory 422 such as a hard disk. The processor 41 exchanges data with the external memory 422 through the internal memory 421. When the electronic device 400 runs, communication is carried out between the processor 41 and the memory 42 through the bus 43, so that the processor 41 executesFigure 1 Steps of the heart sound abnormality detection method in the center.

[0125] The embodiments of the present disclosure also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the heart sound abnormality detection method described in the above method embodiments. Among them, the storage medium can be a volatile or non-volatile computer-readable storage medium.

[0126] The embodiments of the present disclosure also provide a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can execute the steps of the heart sound abnormality detection method described in the above method embodiments. For details, please refer to the above method embodiments and will not be elaborated here.

[0127] Among them, the above computer program product can be specifically implemented in the form of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is specifically embodied as a computer storage medium. In another optional embodiment, the computer program product is specifically embodied as a software product, such as a Software Development Kit (SDK), etc.

[0128] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described device can refer to the corresponding process in the foregoing method embodiments and will not be elaborated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other forms.

[0129] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0130] In addition, in each embodiment of the present disclosure, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0131] When the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such an understanding, the technical solution of the present disclosure, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present disclosure. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs that can store program codes.

[0132] Finally, it should be noted that: the above-mentioned embodiments are only specific implementation manners of the present disclosure, used to illustrate the technical solutions of the present disclosure, rather than limiting them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: any person skilled in the art within the technical scope disclosed by the present disclosure can still modify the technical solutions recorded in the foregoing embodiments, or can easily think of changes, or make equivalent replacements for some of the technical features; and these modifications, changes, or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should all be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims.

Claims

1. A method for detecting abnormal heart sounds, characterized in that, Including: Obtain the original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, divide multiple cardiac cycles included in the standard heart sound signal, and form a periodic heart sound signal; Extract the Mel filter bank coefficient features and Mel frequency cepstral coefficient features corresponding to the periodic heart sound signal; After fusing the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features, input them into the encoder in the pre-trained MobileNetV2 network, and use the self-attention mechanism to capture the long-range dependencies in the periodic heart sound signal, and output a latent space representation vector; Input the latent space representation vector into the classifier of the MobileNetV2 network, and output the prediction result of the heart sound abnormality category through the Softmax function.

2. The method according to claim 1, wherein Dividing multiple cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal specifically includes: Determine the mean and variance corresponding to each cardiac state, and use a Gaussian distribution to describe the duration probability corresponding to each cardiac state according to the mean and the variance; Extract the multi-dimensional signal features corresponding to the standard heart sound signal, input the multi-dimensional signal features as observations into a hidden semi-Markov model, use a logistic regression model to calculate the observation probability corresponding to each cardiac state, and correct the observation probability through a Gaussian distribution; Decode the optimal state sequence corresponding to the standard heart sound signal according to the duration probability and the observation probability, divide the cardiac cycle according to the optimal state sequence, extract each cardiac cycle as an independent unit and standardize it to a fixed length to form the periodic heart sound signal.

3. The method according to claim 1, characterized in that, Fuse the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features based on the following steps: During the generation process of the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features, use the same frame division parameters to align the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features in the time dimension; Stack the Mel filter bank coefficient features and the Mel frequency cepstral coefficient features in the channel dimension to form a two-dimensional stacked feature map; Perform zero-mean unit-variance normalization on the two-dimensional stacked feature map, apply the spectral masking technique to randomly mask specific frequency bands, and randomly stretch or compress a preset range in the time dimension.

4. The method according to claim 3, characterized in that, Output the latent space representation vector based on the following steps: Input the two-dimensional stacked feature map into the encoder, and perform convolution operations by the inverted residual blocks in the MobileNetV2 network; Enhance the ability of the MobileNetV2 network to capture long-range dependencies in the periodic heart sound signal through a self-attention layer, and fuse the attention output with the two-dimensional stacked feature map; Compress the fused attention output and the two-dimensional stacked feature map into a multi-dimensional latent space representation vector through a global average pooling layer and a fully connected layer.

5. The method according to claim 1, wherein Before inputting the latent space representation vector into the classifier of the MobileNetV2 network, the method further includes: Input the latent space representation vector into the decoder in the MobileNetV2 network; Map the initial dimension of the latent space representation vector through a fully connected layer; Gradually restore the temporal resolution of the latent space representation vector through multi-layer transposed convolution processing, and reconstruct the feature representations of the mel filter bank coefficients feature and the mel-frequency cepstral coefficients feature at the output layer.

6. The method according to claim 1, characterized in that, Train the MobileNetV2 network based on the following steps: Obtain sample heart sound signals to construct a training set, and label each sample heart sound signal in the training set with normal heart sounds, pathological abnormal heart sounds, and noise as labels; Use the mean square error to calculate the difference between the original features and the reconstructed features to construct a reconstruction loss function, and use the label smoothing classification loss function to calculate the difference between the predicted class and the true class to construct a classification loss function; Perform weighted summation on the reconstruction loss function and the classification loss function according to a preset balance coefficient to determine the final loss function; Use the training set and the corresponding labels as sample data, and train the MobileNetV2 network under the guidance of the final loss function until the MobileNetV2 network has the ability to classify normal heart sounds, pathological abnormal heart sounds, and noise.

7. A heart sound abnormality detection device, characterized in that, Comprising: A signal processing module, configured to obtain an original heart sound signal, preprocess the original heart sound signal into a standard heart sound signal, and divide a plurality of cardiac cycles included in the standard heart sound signal to form a periodic heart sound signal; A feature extraction module, configured to extract the mel filter bank coefficients feature and the mel-frequency cepstral coefficients feature corresponding to the periodic heart sound signal; An encoding module, configured to fuse the mel filter bank coefficients feature and the mel-frequency cepstral coefficients feature and input them into an encoder in a pre-trained MobileNetV2 network, and use the self-attention mechanism to capture the long-range dependence relationship in the periodic heart sound signal, and output a latent space representation vector; A classification module, configured to input the latent space representation vector into a classifier in the MobileNetV2 network, and output a prediction result of the heart sound abnormality category through a Softmax function.

8. An electronic device, characterized in that, Comprising: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the electronic device runs, the processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps of the heart sound abnormality detection method according to any one of claims 1 to 6 are executed.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is run by a processor, the steps of the heart sound abnormality detection method according to any one of claims 1 to 6 are executed.

10. A computer program product, comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, the steps of the heart sound abnormality detection method according to any one of claims 1 to 6 are implemented.

Images