A Heart Sound Intelligent Classification System and Method Based on Embedded NPU

Abstract

This invention discloses a method for accelerating heart sound classification based on an embedded NPU. First, raw heart sound signals are acquired using a heart sound sensor and preprocessed. Simultaneously, pre-trained improved generative adversarial network (GAN) and dual-branch feature fusion model parameters are loaded onto the embedded NPU platform. Data is transmitted to the NPU processing unit via a USB interface to perform denoising and classification tasks: an improved GAN network combined with STFT mask filtering is used to suppress high-frequency noise, and feature extraction is achieved through parallel temporal and frequency domain branches to realize fusion classification. INT8 fixed-point quantization technology is used to compress model parameters, and multi-core parallel acceleration and pipelined scheduling of the NPU are combined to improve processing efficiency. The final classification result is output to the display terminal via a MIPI interface, achieving high-precision real-time classification on the RK3588 platform (accuracy 96.1%, latency ≤3.97ms, power consumption ≤4W), significantly reducing system resource consumption and power consumption. This method is suitable for primary healthcare heart sound-assisted diagnosis scenarios, improving diagnostic efficiency and robustness in noisy environments.

Description

Technical Field

[0001] This invention belongs to the fields of medical electronics technology and embedded artificial intelligence technology, specifically relating to a heart sound signal denoising and classification system and method based on an embedded NPU (neural network processor), which is particularly suitable for early screening and auxiliary diagnosis of cardiovascular diseases. Background Technology

[0002] Cardiovascular diseases (CVDs) are one of the leading causes of death worldwide, and early diagnosis is crucial for reducing mortality. Heart auscultation is an important means of screening and diagnosing cardiovascular diseases, but traditional auscultation relies heavily on the subjective experience of doctors, and faces the challenge of a shortage of professional doctors in primary healthcare and home health monitoring scenarios.

[0003] With the development of electronic technology and artificial intelligence, intelligent electronic stethoscopes and automatic heart sound analysis technology have emerged. However, existing technologies still face the following bottlenecks:

[0004] (1) Severe signal noise interference: Heart sound signals are easily affected by environmental noise (such as breath sounds and equipment friction sounds) and physiological noise during the acquisition process, and the signal-to-noise ratio is often lower than 10dB. Traditional denoising methods (such as wavelet transform and bandpass filtering) rely on manual parameter adjustment, which has poor adaptability to non-stationary noise and is prone to losing the pathological characteristics of the effective frequency band of heart sounds (70-120Hz).

[0005] (2) Insufficient accuracy of classification models: Existing deep learning models mostly rely on single features (time domain or frequency domain), making it difficult to fully capture the temporal dynamics and spectral characteristics of heart sound signals. For example, the accuracy of a single CNN model on the PhysioNet dataset is generally below 90%, and its sensitivity to pathologies such as aortic stenosis (AS) is less than 85%.

[0006] (3) Low embedded deployment efficiency: GPU solutions have high power consumption (≥40W), while low-power platforms such as Raspberry Pi have limited computing power and cannot support real-time inference of complex models. Existing FPGA solutions are limited by on-chip resources and cannot deploy deep networks with more than 1 million parameters. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies. This invention proposes a heart sound intelligent classification system and method based on an embedded NPU. By improving the generative adversarial network (GAN) denoising algorithm, the dual-branch feature fusion classification model, and NPU hardware acceleration technology, it achieves highly robust and low-power real-time heart sound analysis.

[0008] The technical solution of the present invention is as follows:

[0009] The first aspect is the overall system framework:

[0010] It consists of a heart sound signal acquisition module, an improved GAN denoising module, a dual-branch feature fusion classification module, and an NPU accelerated deployment module. The workflow is as follows:

[0011] (1) Signal acquisition: The raw signal was acquired by the HKY-06B+ heart sound sensor (sampling rate 2kHz) and transmitted to the RK3588 platform via the USB interface.

[0012] (2) Preprocessing: Normalize the signal (amplitude scaled to [-1, 1]) and segment it (each segment lasts 2.5s, covering 3-4 cardiac cycles).

[0013] (3) Noise reduction: Improve the GAN network to suppress noise and output a signal with PSNR≥24.15dB.

[0014] (4) Classification and identification: The dual-branch model outputs pathological classification results (such as "normal" and "aortic stenosis").

[0015] (5) Results showed that the heart sound waveform and diagnostic labels were displayed in real time on a 7-inch screen via the MIPI interface.

[0016] The second aspect is to improve the GAN denoising algorithm:

[0017] The core innovation in this area is the combination of CBAM attention mechanism, LSTM timing modeling, and STFT mask filtering. The generator structure consists of the following:

[0018] (1) Extract local features using 4-layer one-dimensional convolution (kernel size 5, stride 2);

[0019] (2) CBAM Attention Module: Focuses on S1 / S2 heart sound events through a dual-path approach using channel weights (Mc) and spatial weights (Ms).

[0020] Mc=σ(MLP(AvgPool(F))+MLP(MaxPool(F)))Ms=σ(f7×7([AvgPool(F);MaxPool(F)]))

[0021] (3) LSTM module: 64 hidden units, modeling the temporal dependence of the heart sound cycle.

[0022] The discriminator optimization employs least squares loss (LSGAN) to improve training stability.

[0023]

[0024] The STFT mask filter in the algorithm serves to suppress high-frequency noise above 200Hz while preserving pathological features.

[0025]

[0026] The third aspect is the dual-branch feature fusion classification model:

[0027] The core innovation in this area lies in the complementary fusion of time-domain and frequency-domain features.

[0028] The time-domain branch is:

[0029] (1) 1D-CNN (4 convolutional layers, number of channels 128→64→32→16) extracts local waveform features;

[0030] (2) BiLSTM (64 hidden units) captures long-range periodic dependencies and outputs the feature vector Ftime.

[0031] Frequency domain branching is:

[0032] (1) Improved MFCC features: frame length 50ms, frame shift 25ms, calculate first-order / second-order difference enhancement dynamic characterization;

[0033] (2) Variance score screening: The top 13 features with the highest discriminative power are retained (redundancy compression of 40%):

[0034]

[0035] (3) 2D-CNN extracts spectral spatial features and outputs Ffreq.

[0036] Feature fusion and classification are as follows:

[0037] (1) Concatenate Ftime and Ffreq, and input them into a fully connected layer (256 units) and a Softmax classifier;

[0038] (2) Loss function: cross-entropy; optimizer: Adam (learning rate 0.0001).

[0039] The fourth aspect is NPU accelerated deployment:

[0040] The core innovation in this area is INT8 quantization combined with a multi-threaded pipeline. The model quantization uses RKNN-Toolkit2 to convert the FP32 model to INT8 format.

[0041]

[0042] The model size was compressed from 334.5MB to 19.7MB (compression ratio 94.1%).

[0043] Multithreading is designed as follows:

[0044] (1) Acquisition thread: The ALSA library controls the sensor to sample in real time;

[0045] (2) NPU inference thread: calls the librknnrt.so library to perform denoising and classification;

[0046] (3) GUI thread: QT framework development interface, which displays waveforms and results in real time.

[0047] Compared with the prior art, the present invention adopts the above technical solution and has the following significant advantages:

[0048] (1) Excellent noise reduction performance: The improved GAN network combined with STFT mask filtering can specifically suppress high-frequency noise in the heart sound signal. Under 10dB Gaussian noise environment, the peak signal-to-noise ratio (PSNR) of the output signal can reach more than 24.15dB and the mean square error (MSE) is as low as 0.007, which effectively improves the signal quality.

[0049] (2) Extremely high classification accuracy: Through time-frequency domain dual-branch feature fusion, the model can fully utilize heart sound information. On the publicly available PhysioNet / 2016 dataset, the binary classification (normal / abnormal) accuracy reaches 96.1%, with sensitivity and specificity reaching 92.58% and 96.13%, respectively; on the more complex Yaseen four-class dataset, the average recognition accuracy for various pathologies can reach 99%.

[0050] (3) High-efficiency embedded real-time processing: Through INT8 model quantization and NPU hardware acceleration, extremely low inference latency (≤3.97ms) and power consumption (≤4W) are achieved on the RK3588 platform, enabling high-precision intelligent heart sound diagnosis to run on portable devices, meeting the strict requirements of real-time performance and low power consumption in primary healthcare scenarios.

[0051] (4) Complete system integration solution: This invention provides a complete technology chain from algorithm innovation and model optimization to hardware deployment, which has high practicality and scalability, and provides a powerful technical tool for the early screening of cardiovascular diseases. Attached image description:

[0052] Figure 1 System overall architecture diagram (heart sound sensor → NPU processing → display terminal)

[0053] Figure 2 Improved GAN network architecture (including CBAM, LSTM, and STFT mask modules)

[0054] Figure 3 : Diagram of dual-branch feature fusion model (temporal domain 1D-CNN+BiLSTM, frequency domain MFCC+2D-CNN)

[0055] Figure 4Comparison of time-domain images of heart sounds before and after denoising (original signal, denoised signal, residual noise)

[0056] Figure 5 Heart sound classification confusion matrix (normal / abnormal binary and four-class classification results)

[0057] Figure 6 RK3588 Platform Deployment Flowchart (Data Acquisition → NPU Inference → GUI Display) Detailed Implementation

[0058] Example 1: Software Implementation and Training of Heart Sound Denoising and Classification Algorithm

[0059] This embodiment details how to train a high-performance denoising and classification model. For example... Figure 1 As shown, the overall workflow of the system includes signal acquisition, preprocessing, noise reduction, classification and recognition, and result display, which provides a framework for the algorithm training in this embodiment.

[0060] First, data preparation was performed: several publicly available heart sound datasets were collected and integrated, including PhysioNet / 2016 (for binary classification training) and the Yaseen dataset (for quadruple classification training). To simulate real-world noisy environments, noise samples (such as wind, conversation, and equipment hum) from the ESC-50 ambient sound library were added to the clean heart sounds at different signal-to-noise ratios (SNR) ranging from 0dB to 20dB, constructing noisy sample pairs for training the denoising model.

[0061] In the training phase of improving the GAN denoising model, the network structure is built based on the TensorFlow framework. For example... Figure 2 As shown, the generator takes a noisy heart sound segment (2.5s in length, 5000 sampling points) as input, passes it through four cascaded CCL_Blocks (containing convolution, batch normalization, LeakyReLU activation, LSTM units, and CBAM attention modules), and finally outputs a preliminary denoised signal through a fully connected layer and Tanh activation. The discriminator uses a 5-layer fully connected network to distinguish between real, clean heart sounds and the generated signal. The Adam optimizer is used (learning rate of 0.0001 for both generator and discriminator), with a batch size of 16, and training for 200 epochs. The loss function used is LSGAN loss.

[0062]

[0063] After training, apply an STFT mask filter to the generator output, such as... Figure 1 As shown, the final denoised signal is obtained.

[0064] During the training phase of the dual-branch feature fusion classification model, such as Figure 3As shown, the 1D-CNN using the temporal branch is configured with 4 convolutional layers (number of filters 128→64→32→16, kernel size 4, stride 2), followed by max pooling after each layer, and the number of BiLSTM hidden units is set to 64. In the frequency branch, the MFCC feature frame length is 50ms, the frame shift is 25ms, the number of Mel filters is 24, and 39-dimensional features are calculated (13-dimensional static MFCC + first / second order difference). The top 13 high-discriminative features are retained by using variance scores.

[0065]

[0066] The time-domain and frequency-domain features are concatenated and input into a 256-unit fully connected layer, which then outputs the class probabilities via Softmax. Cross-entropy loss and the Adam optimizer (learning rate 0.0001, batch size 32) are used to train until the accuracy on the validation set converges.

[0067] Example 2: Deployment of an Embedded System Based on the RK3588 Platform

[0068] This embodiment details the deployment and real-time operation of the model on an embedded device. During the model conversion and quantization stage, the RKNN-Toolkit2 toolchain is installed on the Ubuntu system. After importing the TensorFlow model (.pb format), asymmetric quantization parameters are configured. The quantization formula is as follows:

[0069]

[0070] The dynamic range was determined using calibration data, and the FP32 model was converted to INT8 format, compressing the model size from over 300MB to approximately 20MB. Finally, it was exported as a RKNN format file specific to the RK3588 NPU. In the embedded software development phase, a Linux system was deployed on the RK3588, and the RKNN Runtime library was installed. Multithreaded applications were written in C++, such as... Figure 6 As shown, the deployment process includes: First, thread A (the acquisition thread) reads sensor data in real time through the ALSA library and stores it in a circular buffer; second, thread B (the NPU inference thread) calls the RKNN Runtime API to sequentially execute the denoising and classification models; third, thread C (the GUI thread) displays the heart sound waveform and diagnostic results (such as "normal" or "suspected aortic stenosis") in real time based on the QT framework. Thread synchronization is achieved through mutexes and semaphores to ensure the efficient operation of the pipeline for data acquisition, NPU inference, and GUI display.

[0071] Example 3: System Performance Testing and Verification

[0072] This embodiment verifies system performance through comprehensive testing. In the denoising performance test, for 100 heart sound samples with 10dB Gaussian noise added, the average PSNR after improved GAN denoising reached 24.15dB, and the MSE was 0.007. Figure 4 As shown, the time-domain comparison before and after denoising clearly demonstrates that the noise in the original signal is effectively suppressed, and the heart sound cycle characteristics of the denoised signal are more obvious, verifying the effectiveness of the algorithm.

[0073] Classification accuracy tests showed that on the PhysioNet / 2016 test set, the binary classification accuracy reached 96.1% (specificity 96.13%, sensitivity 92.58%); on the Yaseen four-class test set, the accuracy in identifying pathological conditions such as aortic stenosis (AS) and mitral stenosis (MS) was close to 100%. Figure 5 As shown, the confusion matrix ( Figure 5 (a) shows the binary classification result. Figure 5 (b) The four-class classification results quantitatively demonstrate the model's high-precision classification ability, with almost no confusion between abnormal categories.

[0074] Embedded real-time performance testing shows that the system's power consumption on the RK3588 platform is stable at 3.8W-4.2W, with an average end-to-end processing latency of only 3.97ms (far below the 100ms real-time requirement), and the classification accuracy remains at 94.2% even in a 10dB dialogue noise environment. Figure 6 The deployment process and hardware connections ensure the system's practicality and stability in complex scenarios. Key scenario performance comparisons are as follows: In a quiet environment, classification accuracy is 96.1%, latency is 3.97ms, and power consumption is 3.8W; in a 10dB fan noise environment, accuracy is 94.2%, latency is 19.41ms, and power consumption is 4.0W; under device movement interference, accuracy is 92.5%, latency is 21.30ms, and power consumption is 4.2W, verifying the system's practicality and stability in complex scenarios.

Claims

1. A heart sound intelligent diagnostic system based on an embedded NPU, characterized in that, It includes a heart sound signal acquisition module, an improved generative adversarial network denoising module, a dual-branch feature fusion classification module, and an NPU accelerated deployment module; The heart sound signal acquisition module acquires raw heart sound signals (sampling rate 2kHz) through the HKY-06B+ sensor and performs preprocessing, including signal normalization and segmentation (each segment lasts 2.5s). The improved generative adversarial network denoising module integrates CBAM attention mechanism (channel and spatial dual-path weighting) and LSTM timing modeling unit in the generator, and is connected to STFT mask filter (cutoff frequency 200Hz, attenuation coefficient α=0.5) at the back end to suppress high-frequency noise in a targeted manner. The dual-branch feature fusion classification module extracts temporal features through the temporal branch (1D-CNN convolution kernel size 4, number of channels 128 / 64 / 32 / 16+BiLSTM), and extracts spectral features based on the improved MFCC features (including first-order / second-order differences, and the variance score is used to select the top 13 high-discriminative features) through the frequency domain branch. The fusion results are then output as classification results. The NPU acceleration deployment module implements model INT8 quantization and multi-threaded pipeline scheduling on the RK3588 platform, supporting real-time inference (latency ≤ 3.97ms).

2. The system according to claim 1, characterized in that, The improved generative adversarial network denoising method specifically includes: Step 1: Perform a short-time Fourier transform on the input heart sound signal to generate a time-frequency spectrum; Step 2: The generator uses the CBAM module to weight key feature regions (focusing on S1 / S2 events), and the LSTM module to model the temporal dependence of the heart sound cycle; Step 3: The discriminator uses the least squares loss function (formula: Optimize generator output; Step 4: Apply STFT mask filtering to attenuate frequencies above 200Hz (retaining the effective heart sound frequency band of 70-120Hz); Step 5: Reconstruct the denoised signal through inverse STFT, and output PSNR≥24.15dB.

3. The system according to claim 1, characterized in that, The dual-branch feature fusion classification method includes: Temporal branch: 1D-CNN extracts local waveform features, BiLSTM captures long-range periodic dependencies, and outputs feature vectors; Frequency domain branch: Calculate MFCC features (frame length 50ms, frame shift 25ms), introduce variance scoring method to filter the top 13 features (compress redundancy 40%), splice first-order / second-order difference to enhance dynamic representation, and output through 2D-CNN; Feature fusion: F time With F freq Concatenate according to channel dimension to form F fusion The input is a fully connected layer and a Softmax classifier, optimized with a cross-entropy loss function (learning rate 0.0001). On the PhysioNet dataset, the binary classification accuracy is 96.1% (sensitivity 92.58%, specificity 96.13%).

4. The system according to claim 1, characterized in that, The NPU accelerated deployment method includes: INT8 Quantization: The FP32 model is quantized to INT8 format using RKNN-Toolkit2. The quantization formula is as follows: Where r is the original parameter, s is the scaling factor, and z is the zero-point offset (z = round(q)). max -r max The model size was compressed from 334.5MB to 19.7MB (compression ratio 94.1%). Multithreaded pipeline: Acquisition thread: The ALSA library controls the sensor to sample in real time; NPU inference thread: calls the librknnrt.so library to perform denoising and classification in parallel; GUI thread: QT framework development interface, which displays heart sound waveforms and classification labels (such as "normal" and "aortic stenosis") in real time.

5. The system according to claim 1, characterized in that, The hardware configuration includes: (1) The HKY-06B+ heart sound sensor is connected to the RK3588 development board via a USB interface; (2) The RK3588 processor (NPU computing power 6 TOPS, memory 16GB) runs the quantization model; (3) The MIPI interface display (7-inch) outputs classification results and waveforms; (4) The power module provides ≤4W power consumption support, which is suitable for portable application scenarios.

6. The system according to claim 1, characterized in that, The workflow is as follows: (1) The sensor acquires the raw signal (duration 2.5s) and transmits it to RK3588 via USB; (2) The preprocessing module performs normalization and segmentation; (3) The NPU loads the INT8 quantized model in parallel and performs denoising (output PSNR>24.15dB) and classification (accuracy ≥96.1%). (4) The results are displayed via the MIPI interface, with an end-to-end delay of ≤3.97ms; (5) The GUI interface draws heart sound waveforms in real time and marks the pathological type (such as mitral regurgitation, aortic stenosis).

7. The system according to claim 2, characterized in that, The frequency domain attenuation function of the STFT mask filter is: Where α = 0.5 and β = 0.02, electromyographic interference and friction noise are suppressed.

8. The system according to claim 3, characterized in that, The variance score screening method is used to calculate the time series variance of MFCC in each dimension: reserve The top 13 largest features (k∈[1,13]) are used to eliminate dimensions with low discriminative power.

9. The system according to claim 1, characterized in that, It maintains a classification accuracy of ≥94.2% even in noisy environments (10dB fan noise) and consumes ≤4W of power, making it suitable for primary healthcare cardiac sound-assisted diagnosis scenarios.

Images