Integrated lightweight heart sound detection system and method

Abstract

The invention discloses an integrated lightweight heart sound detection system and method. According to the heart sound detection system, the heart sound sensor and the microprocessor are integrated, the heart sound detection deep learning model is simplified and optimized through a lightweight algorithm, and the lightweight heart sound detection deep learning model is deployed on the microprocessor through floating point quantification, so that real-time and accurate analysis of heart sound signals can be carried out. The heart sound detection system is low in power consumption, high in detection precision, low in cost, simple in structure, small in size, portable and easy to use.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence medical instruments, and in particular to an integrated lightweight heart sound detection system and method. Background Art

[0002] Cardiovascular disease is the leading cause of death in the Chinese population, and has a huge impact on the health and socioeconomic status of our people. In particular, coronary heart disease, as the most common form of cardiovascular disease, is considered a leading health threat in China and worldwide. The potential serious consequences of coronary heart disease, such as myocardial infarction or acute heart failure, show the importance of effective monitoring of cardiovascular health in the early stages of disease development.

[0003] Although the development of modern medical technology has provided many sophisticated detection methods for the diagnosis of heart disease, such as coronary angiography, CT examination, physiological stress test and serum myocardial enzyme detection, these devices usually require expensive hardware facilities and highly specialized operating environments. This means that these advanced detection methods are mainly concentrated in large general hospitals or specialized heart disease centers, and are difficult to popularize in primary medical institutions with limited resources or remote areas.

[0004] Existing detection devices, after collecting heart sound signals, cardiac acoustic signals and other signals, need to transmit the collected signals to other servers or devices through communication modules for processing. They do not have the portable characteristics of direct use and processing, and do not use onboard executable artificial intelligence solutions for heart sound detection. In addition, there are many commercial devices. For example, 3M Littmann and others have launched excellent electronic stethoscopes. Such products have always focused on direct auscultation and do not provide support for digital-to-analog conversion and data storage, so they cannot collect data and use computer-assisted calculations. In addition, they are very expensive, which is not conducive to popularization.

[0005] In order to solve the above problems, a low-cost, portable and easy-to-use heart sound detection system is urgently needed to enhance the monitoring capabilities of cardiovascular health in primary medical centers, community clinics and remote areas. Summary of the invention

[0006] In view of this, the present invention provides an integrated lightweight heart sound detection system and method, which can achieve efficient and accurate heart sound detection, and the equipment is simple, easy to carry and low cost.

[0007] The integrated lightweight heart sound detection system of the present invention comprises: a handheld housing, and a vibrating membrane stethoscope head, a printed circuit main board and a power module installed in the handheld housing;

[0008] The vibrating membrane stethoscope head is located at the bottom of the handheld housing and is used to collect heart sound vibration signals; the printed circuit main board is provided with a microphone amplifier circuit, an anti-aliasing filter circuit, an analog-to-digital conversion circuit, a microcontroller unit and a low-voltage difference linear voltage regulator circuit; the microphone amplifier circuit is used to convert the heart sound vibration signal collected by the vibrating membrane stethoscope head into an electrical signal and amplify it; the anti-aliasing filter circuit is used to filter out noise in the collected heart sound signal; the analog-to-digital conversion circuit is used to convert the filtered electrical signal into a digital signal and transmit it to the microcontroller unit; the low-voltage difference linear voltage regulator circuit is used to adjust the output voltage of the power module to supply power to the microphone amplifier circuit, the anti-aliasing filter circuit, the analog-to-digital conversion circuit and the microcontroller unit; the microcontroller unit is used to extract the characteristics of the heart sound signal, and complete the heart sound analysis based on the lightweight deep learning model deployed on the microcontroller unit;

[0009] Among them, the lightweight deep learning model is obtained by streamlining and optimizing the pre-trained heart sound analysis deep learning model through a lightweight algorithm, and then the pytorch model of the lightweight deep learning model is converted into a cross-framework intermediate model based on a cross-framework intermediate model converter, and the floating-point numbers of the cross-framework intermediate model are quantized, and then deployed to the microcontroller unit; the cross-framework intermediate model converter is used to convert the pytorch model into a cross-framework intermediate model represented by a computational graph that can be executed by the microprocessor unit.

[0010] Preferably, the microphone amplifier circuit uses MAX9814, MAX9814, INA217, LM386 or SSM2167.

[0011] Preferably, the anti-aliasing filter circuit adopts a bandpass filter based on an FIR filter, a bandpass filter based on a Butterworth filter, or a bandpass filter based on an IIR filter.

[0012] Preferably, the microcontroller unit adopts STM32, Raspberry Pi, Arduino, TI LaunchPad or RISC-V.

[0013] The better lightweight algorithms are knowledge distillation, model pruning, low-rank decomposition, parameter sharing or network architecture search.

[0014] Preferably, the cross-framework intermediate model adopts TFLite or ONNX model.

[0015] Preferably, the microcontroller unit is STM32; first use the ONNX model converter to convert the pytorch model of the lightweight deep learning model into an onnx model, then quantize the floating-point number of the onnx model into an 8-bit wide floating-point number, and finally use the AI.Cube converter on the STM32 to migrate the quantized model to the STM32 for deployment.

[0016] The present invention also provides a heart sound detection method using the above-mentioned integrated lightweight heart sound detection system, comprising:

[0017] The heart sound detection system is placed at the heart, and the vibrating membrane stethoscope head collects the heart sound vibration signal, which is converted into an electrical signal and amplified by a microphone amplifier circuit, filtered out noise by an anti-aliasing filter circuit, and converted into a digital signal by an analog-to-digital conversion circuit and transmitted to a microcontroller unit;

[0018] The microcontroller unit slices the heart sound signal and extracts the features of each segment of the heart sound signal; the lightweight deep learning module completes the detection of the heart sound according to the extracted features.

[0019] Preferably, the microcontroller unit normalizes the heart sound signal first and then performs slice detection.

[0020] Preferably, the features of the heart sound signal adopt Log Mel spectrogram features, MFCC features, Delta features or second-order spectrum features.

[0021] Beneficial effects:

[0022] The heart sound detection system of the present invention integrates a heart sound sensor and a microprocessor. Through a lightweight algorithm, the heart sound detection deep learning model is simplified and optimized, and through model conversion and floating-point quantization, the lightweight heart sound detection deep learning module is deployed on the microprocessor. The heart sound analysis and detection of a 2-second slice can be completed in only 1.5 seconds, meeting the real-time requirements. The heart sound detection system of the present invention has low power consumption, high detection accuracy, low cost, simple structure, small size, easy portability, and easy use. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a physical picture of the heart sound detection system of the present invention.

[0024] Figure 2 This is a flow chart of heart sound detection according to the present invention.

[0025] Figure 3 This is the knowledge distillation process.

[0026] Figure 4 The process of deploying the model on STM32. DETAILED DESCRIPTION

[0027] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0028] The present invention provides an integrated lightweight heart sound detection system, comprising a handheld housing, and a vibrating membrane stethoscope head, a printed circuit board and a power module installed in the handheld housing; the physical diagram thereof is shown in FIG. Figure 1 shown.

[0029] Among them, the vibrating membrane stethoscope head is located at the bottom of the handheld shell, and is responsible for collecting heart sound vibration signals; the printed circuit main board is provided with a microphone amplifier circuit, an anti-aliasing filter circuit, an analog-to-digital conversion circuit, a microcontroller unit and a low-voltage difference linear voltage regulator circuit; wherein, the microphone amplifier circuit is used to convert the heart sound vibration signal collected by the vibrating membrane stethoscope head into an electrical signal and amplify the heart sound signal to match the dynamic range of the analog-to-digital converter; the anti-aliasing filter circuit is used to filter out the noise in the collected heart sound signal to obtain a relatively clean heart sound signal; the analog-to-digital conversion circuit is used to convert the filtered electrical signal into a digital signal; the low-voltage difference linear voltage regulator circuit is used to adjust the output voltage of the battery to realize different voltage power supply for the microphone amplifier circuit, the anti-aliasing filter circuit, the analog-to-digital conversion circuit, and the microcontroller unit; a lightweight neural network model is deployed on the microcontroller unit to realize the analysis of heart sound signals. The heart sound detection process is as follows Figure 2 shown.

[0030] Among them, the microphone amplifier circuit can use MAX9814 (Analog Devices, Inc.), INA217 (Texas Instruments), LM386 (Texas Instruments), SSM2167 (Analog Devices, Inc.), etc., which have automatic gain control and low-noise microphone bias, low cost and high conversion quality.

[0031] The anti-aliasing filter circuit can use a bandpass filter based on a finite-length unit impulse response filter (FIR filter), a bandpass filter based on a Butterworth filter, a bandpass filter based on an IIR filter, etc. This embodiment uses a bandpass filter based on an FIR filter, selects 500 orders, and has a passband boundary frequency of 5 Hz to 400 Hz, which is less sensitive to quantization noise and rounding errors in finite word length operations (such as fixed-point operations).

[0032] The microcontroller unit may be STM32, Raspberry Pi, Arduino, TI LaunchPad, RISC-V, etc. Among them, the STM32 microcontroller has been widely used in industrial equipment due to its low power consumption, high integration and excellent cost performance. The STM32 microcontroller implements a full set of digital signal processing instructions and has a single-precision floating-point unit, supporting all data processing instructions and all data types used in the heart sound processing process. This embodiment uses the STM32 microcontroller.

[0033] There are still many problems to be solved when deploying neural network models on microcontrollers. First, many existing neural network models for processing heart sounds are very large, with many parameters and complex models, while the processing power and memory resources of microcontrollers are far lower than those of traditional computing platforms. It is impossible to deploy large-scale neural network models, nor can it meet the accuracy and real-time requirements of heart sound signal processing; secondly, the MCU chip on the microcontroller is controlled by software (code), and it is also necessary to ensure that the control execution command program allows the microprocessor to run stably.

[0034] In order to achieve efficient heart sound signal analysis under limited hardware conditions, the present invention first adopts a lightweight algorithm to deeply optimize the neural network model, streamline redundant parameters, reduce the occupancy of computing resources and storage resources for the microcontroller, and try to keep its performance unaffected, so as to achieve real-time analysis of heart sound signals; then, based on the cross-framework intermediate model converter, the pytorch model of the lightweight deep learning model is converted into a cross-framework intermediate model; the cross-framework intermediate model converter converts the pytorch model used on the computer into a cross-framework intermediate model represented by a computational graph that can be executed by the microprocessor unit; then, the floating-point numbers of the converted cross-framework intermediate model are quantized, and then the quantized model is migrated to the microprocessor for deployment, thereby realizing the integration of the neural network model and the microcontroller hardware.

[0035] Among them, the cross-framework intermediate model is generally a TFLite or ONNX model. This embodiment uses an STM32 microcontroller; the lightweight neural network model conversion deployment is specifically as follows: first use the ONNX model converter to convert the pytorch model of the lightweight deep learning model into an onnx model, then quantize the floating-point number of the onnx model into an 8-bit wide floating-point number, and finally use the AI.Cube converter on the STM32 to migrate the quantized model to the STM32 for deployment.

[0036] Among them, the lightweight algorithm can adopt knowledge distillation, model pruning, low-rank decomposition, parameter sharing, network architecture search and other methods to achieve lightweight neural network.

[0037] This embodiment uses knowledge distillation to "migrate" the knowledge in the pre-trained large-scale heart sound analysis neural network model to a smaller lightweight model, so that it can run efficiently on the resource-constrained STM32. Through knowledge distillation, the lightweight model can significantly improve the reasoning speed, reduce the computing cost, and achieve real-time reasoning without significantly reducing the accuracy.

[0038] Specifically: First, select the teacher model and the student model: the teacher model is a pre-trained neural network model for heart sound analysis with superior performance and large size, while the student model is a target model, which is usually simpler in structure and has fewer parameters. The core idea of ​​distillation is to let the student model learn the knowledge in the teacher model. Secondly, construct soft labels: the predicted probability distribution generated by the teacher model on the training data is called "soft label". Soft labels not only contain the confidence of the correct category, but also the teacher model's understanding of other categories. This information is very important for the training of the student model. Compared with hard labels (only 0 or 1), soft labels can better guide the student model to learn. Define the loss function: The loss function of knowledge distillation usually consists of two parts: distillation loss: the gap between the output probability distribution of the student model and the teacher model, usually measured by KL divergence (Kullback-LeiblerDivergence); standard cross entropy loss: the error between the student model and the true label of the training data. Next, the student model is trained: the parameters of the student model are optimized by combining the distillation loss and the standard cross entropy loss so that it can simulate the behavior of the teacher model while maintaining a low complexity. During the training process, hyperparameters such as the distillation temperature can be adjusted to further improve the learning effect of the student model.

[0039] In addition, the neural network model on the computer is in pytorch format, so its built-in floating-point unit needs to be quantized, that is, all its 32-bit floating-point numbers need to be quantized into 8-bit wide floating-point numbers, so that it can be migrated to STM32 for deployment. This embodiment first uses ONNX's model quantization converter to convert the pytorch model of the lightweight neural network into an onxx model, and then quantizes all 32-bit floating-point numbers in the model into 8-bit wide floating-point numbers. Finally, the STM32 AI.Cube converter is used to migrate the quantized model to STM32 for deployment, thereby realizing the integration of the neural network model and the microcontroller hardware. The quantization process is as follows Figure 3 shown.

[0040] During the heart sound detection, the heart sound detection system of the present invention is placed at the heart, and the vibrating membrane stethoscope head collects the heart sound vibration signal, which is converted into an electrical signal and amplified by a microphone amplifier circuit, and then filtered out noise by an anti-aliasing filter circuit, converted into a digital signal by an analog-to-digital conversion circuit, and transmitted to an STM32 microcontroller;

[0041] The STM32 microcontroller first normalizes the heart sound signal to reduce the influence of the signal amplitude difference caused by the acquisition equipment; this embodiment uses xi=xi / max(|x|) for normalization to transform the heart sound signal into the range of [-1,1]; of course, other normalization methods can also be used to transform the heart sound signal into the range of [-1,1] or [0,1]; then, the heart sound signal is sliced, and each detection needs to collect 10s of heart sound signals, and fixed-length segments are extracted using overlapping sliding windows, and interference signals before and after 1s are removed. Preprocessing and segmentation are performed every 0.5s to obtain 12 heart sound segments; then, the features of the heart sound signals of each segment are extracted, and the features of the heart sound signals can be Log Mel spectrogram features, MFCC features, Delta features, second-order spectrum features, etc. This embodiment adopts the Log Mel spectrogram feature, which has excellent performance in acoustics. This embodiment extracts 64 Mels to obtain audio information, and finally uses z-score normalization to process the feature: F = (Fu) / sigma; finally, the extracted features of the heart sound signal are input into a lightweight neural network to obtain the detection result.

[0042] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An integrated lightweight heart sound detection system, characterized in that: include: A handheld housing, and a diaphragm stethoscope head, a printed circuit board, and a power module installed in the handheld housing; The vibrating membrane stethoscope head is located at the bottom of the handheld housing and is used to collect heart sound vibration signals; the printed circuit main board is provided with a microphone amplifier circuit, an anti-aliasing filter circuit, an analog-to-digital conversion circuit, a microcontroller unit and a low-voltage difference linear voltage regulator circuit; the microphone amplifier circuit is used to convert the heart sound vibration signal collected by the vibrating membrane stethoscope head into an electrical signal and amplify it; the anti-aliasing filter circuit is used to filter out noise in the collected heart sound signal; the analog-to-digital conversion circuit is used to convert the filtered electrical signal into a digital signal and transmit it to the microcontroller unit; the low-voltage difference linear voltage regulator circuit is used to adjust the output voltage of the power module to supply power to the microphone amplifier circuit, the anti-aliasing filter circuit, the analog-to-digital conversion circuit and the microcontroller unit; the microcontroller unit is used to extract the characteristics of the heart sound signal, and complete the heart sound analysis based on the lightweight deep learning model deployed on the microcontroller unit; Among them, the lightweight deep learning model is obtained by streamlining and optimizing the pre-trained heart sound analysis deep learning model through a lightweight algorithm, and then the pytorch model of the lightweight deep learning model is converted into a cross-framework intermediate model based on a cross-framework intermediate model converter, and the floating-point numbers of the cross-framework intermediate model are quantized, and then deployed to the microcontroller unit; the cross-framework intermediate model converter is used to convert the pytorch model into a cross-framework intermediate model represented by a computational graph that can be executed by the microprocessor unit.

2. The system according to claim 1, characterized in that The microphone amplifier circuit uses MAX9814, MAX9814, INA217, LM386 or SSM2167.

3. The system according to claim 1, characterized in that The anti-aliasing filter circuit adopts a bandpass filter based on an FIR filter, a bandpass filter based on a Butterworth filter, or a bandpass filter based on an IIR filter.

4. The system according to claim 1, characterized in that The microcontroller unit adopts STM32, RaspberryPi, Arduino, TI LaunchPad or RISC-V.

5. The system according to claim 1, characterized in that The lightweight algorithms include knowledge distillation, model pruning, low-rank decomposition, parameter sharing or network architecture search.

6. The system according to claim 1, wherein: The cross-framework intermediate model adopts TFLite or ONNX model.

7. The system according to claim 6, characterized in that The microcontroller unit is STM32; first, the ONNX model converter is used to convert the pytorch model of the lightweight deep learning model into an onnx model, and then the floating-point number of the onnx model is quantized into an 8-bit wide floating-point number, and finally the AI.Cube converter on the STM32 is used to migrate the quantized model to the STM32 for deployment.

8. A heart sound detection method using the integrated lightweight heart sound detection system according to any one of claims 1 to 7, characterized in that: The heart sound detection system is placed at the heart, and the vibrating membrane stethoscope head collects the heart sound vibration signal, which is converted into an electrical signal and amplified by a microphone amplifier circuit, filtered out noise by an anti-aliasing filter circuit, and converted into a digital signal by an analog-to-digital conversion circuit and transmitted to a microcontroller unit; The microcontroller unit slices the heart sound signal and extracts the features of each segment of the heart sound signal; the lightweight deep learning module completes the detection of the heart sound according to the extracted features.

9. The method according to claim 8, characterized in that The microcontroller unit first normalizes the heart sound signal and then performs slice detection.

10. The method according to claim 8 or 9, characterized in that The features of the heart sound signal adopt Log Mel spectrogram features, MFCC features, Delta features or second-order spectrum features.

Images