Heart sound classification method and device and computer readable storage medium

Abstract

The invention provides a heart sound classification method and device and a computer readable storage medium. The method comprises the following steps: acquiring a heart sound signal by an auscultation part; the data processor performs segment processing on the heart sound signals through the sliding window to obtain segment signals; the data processor converts the segmented signals from a time domain to a frequency domain to generate a segmented spectrogram; the data processor inputs the segmented spectrogram into a trained deep learning model to obtain a classification result, the deep learning model takes a residual convolutional neural network as a basic network architecture and comprises an attention mechanism module, and the attention mechanism module is used for enhancing attention to key features in the segmented spectrogram; and the communicator uploads the classification result to a back-end server and / or a personal terminal. According to the heart sound classification method provided by the invention, the heart sound classification precision of the deep learning model is improved, and the generalization ability and adaptability to heart sound signals are enhanced.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medicine, and particularly relates to a heart sound classification method and device and a computer readable storage medium. BACKGROUND

[0002] In recent years, the field of structural heart disease (SHD) has made significant technological progress, focusing on minimally invasive intervention and instrument innovation. Structural heart disease refers to abnormalities in the structure of the heart, such as valvular heart disease and congenital heart disease. The traditional diagnostic method for structural heart disease mainly relies on auscultation and imaging examination, but has the problems of strong subjectivity and high cost. At present, heart sound signal analysis technology has gradually become a new method for assisting in the diagnosis of structural heart disease, but the existing technology has low accuracy in heart sound classification, and cannot accurately classify the characteristics of heart sounds.

[0003] Therefore, how to improve the accuracy of heart sound classification is a problem to be solved. SUMMARY

[0004] In view of the problems existing in the prior art, a heart sound classification method, device and computer readable storage medium are provided, which can improve the accuracy of heart sound classification.

[0005] The application provides the following solutions.

[0006] In a first aspect, the application provides a heart sound classification method, which is applied to a heart sound classification device, the heart sound classification device comprising an auscultation part, a data processor, a portable wearing part and a communicator, the portable wearing part being used to relatively fix the heart sound classification device to a human body, and the method comprising the following steps: obtaining a heart sound signal by the auscultation part; segmenting and processing the heart sound signal by the data processor through a sliding window to obtain a segmented signal; converting the segmented signal from a time domain to a frequency domain by the data processor to generate a segmented spectrum graph; inputting the segmented spectrum graph into a trained deep learning model by the data processor to obtain a classification result, the deep learning model being based on a residual convolutional neural network as a basic network architecture and the deep learning model comprising an attention mechanism module, the attention mechanism module being used to enhance the attention to key features in the segmented spectrum graph; uploading the classification result to a backend server and / or a personal terminal by the communicator.

[0007] In some possible embodiments, the attention mechanism module comprises a channel attention module and a spatial attention module, the channel attention module and the spatial attention module being in a sequential cascading relationship, and the channel attention module being prior to the spatial attention module. The channel attention module is used to enhance the attention to key channels in the segmented spectrogram, and the spatial attention module is used to enhance the attention to key positions in the segmented spectrogram.

[0008] In some possible embodiments, the data processor segments the heart sound signal through a sliding window to obtain a segmented signal, including: The data processor segments the heart sound signal through a sliding window to obtain a segmented signal including continuous 5-second heart sound signals.

[0009] In some possible embodiments, the output layer of the deep learning model adopts a full connection layer, the full connection layer maps the extracted features to predefined heart sound categories, and the classification result is optimized through a cross-entropy loss function.

[0010] In some possible embodiments, the heart sound classification device further includes an alarm, and the method further includes: When the classification result obtained by the data processor matches the first preset result, the data processor sends a reminder signal to the alarm.

[0011] In some possible embodiments, the method further includes: When the classification result obtained by the data processor matches the second preset result, an alarm signal is sent to the backend server through the communication device; After receiving the alarm signal, the backend server sends a help signal to all user terminals associated with the user account.

[0012] In some possible embodiments, the deep learning model takes a ResNet18 network as a basic network architecture.

[0013] In a second aspect, the present application provides a heart sound classification device, which includes a auscultation part, a data processor, a portable wearing part and a communication device, the portable wearing part is used to relatively fix the heart sound classification device with the human body: The auscultation part is used to obtain a heart sound signal. The data processor is used to segment the heart sound signal through a sliding window to obtain a segmented signal. The data processor is used to convert the segmented signal from a time domain to a frequency domain to generate a segmented spectrogram. The data processor is used to input the segmented spectrogram into a trained deep learning model to obtain a classification result, the deep learning model takes a residual convolutional neural network as a basic network architecture and the deep learning model includes an attention mechanism module, the attention mechanism module is used to enhance the attention to key features in the segmented spectrogram. The communication device is used to upload the classification result to a backend server and / or a personal terminal.

[0014] In some possible embodiments, the attention mechanism module comprises a channel attention module and a spatial attention module, the channel attention module and the spatial attention module are in a sequential cascade relationship, and the channel attention module is prior to the spatial attention module. The channel attention module is used to enhance the attention to key channels in the segmented spectrogram, and the spatial attention module is used to enhance the attention to key positions in the segmented spectrogram.

[0015] In a third aspect, the present application provides a computer-readable storage medium, which stores a program, when the program is executed by a multi-core processor, the multi-core processor executes the heart sound classification method described above.

[0016] The heart sound classification method provided by the present application can effectively capture the long-time dynamic characteristics of the heart sound signal by using a sliding window for segmentation processing and converting the heart sound signal from the time domain to the frequency domain to generate a heart sound spectrogram. The present application also sets up a residual convolutional neural network as the architecture of the deep learning model, and optimizes the multi-dimensional features of the segmented spectrogram through an attention mechanism module, which can improve the learning ability and convergence speed of the deep learning model and significantly improve the classification ability of the model for heart sound characteristics. In this way, the present application combines Fourier transform, sliding window segmentation technology and residual convolutional neural network based on deep learning, and optimizes feature expression through an attention mechanism. This architecture design not only improves the classification accuracy of the deep learning model for heart sounds, but also enhances the generalization ability and adaptability of the heart sound signal.

[0017] Other advantages of the present application will be described in more detail in conjunction with the following description and drawings.

[0018] It should be understood that the above description is only a summary of the technical solutions of the present application, so as to enable a clearer understanding of the technical means of the present application, and thus the content of the description can be implemented. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application are described below. BRIEF DESCRIPTION OF DRAWINGS

[0019] By reading the detailed description of the exemplary embodiments below, those skilled in the art will understand the advantages and benefits described herein and other advantages and benefits. The accompanying drawings are for the purpose of illustrating exemplary embodiments only and are not considered limiting to the present application. In the drawings: Figure 1 A schematic diagram of a heart sound classification method provided by an embodiment of the present application; Figure 2 A processing schematic diagram of a heart sound signal provided by an embodiment of the present application; Figure 3 A schematic diagram of the accuracy rate of a heart sound classification method provided by an embodiment of the present application during training; Figure 4 A schematic diagram of a heart sound classification device according to an embodiment of the application.

[0020] In the drawings, identical or similar reference signs indicate identical or similar parts. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present application will be described in detail with reference to the drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application can be embodied in various forms without being limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present application will be thorough and complete, and will fully convey the scope of the application to those skilled in the art.

[0022] In the description of the embodiments of the present application, it should be understood that terms such as "include" or "have" are intended to indicate that there can be additional features, numbers, steps, actions, components, parts, or combinations thereof, in addition to those described in the specification, and do not exclude the presence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0023] Unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" herein only describes the association relationship of the associated objects, and means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone.

[0024] The terms "first", "second", and the like are used only to facilitate the description of the same or similar technical features, and should not be understood to indicate or imply that the relative importance or quantity of the technical features. Therefore, the features defined by "first", "second", and the like can explicitly or implicitly include one or more such features. In the description of the embodiments of the present application, unless otherwise specified, the meaning of the term "plurality" is two or more than two.

[0025] In addition, it should be further noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0026] Reference Figure 1 , Figure 1 A schematic diagram of a heart sound classification method according to an embodiment of the application.

[0027] As Figure 1As shown, the heart sound classification method provided in the embodiment of the present application is applied to a heart sound classification device, and all the steps in the embodiment of the present application can be completed by the heart sound classification device. The heart sound classification device comprises a auscultation part, a data processor, a portable wearing part and a communicator. The portable wearing part is used to relatively fix the heart sound classification device to the human body. The heart sound classification method provided in the embodiment of the present application comprises: S101: The heart sound signal obtained by the auscultation part.

[0028] S102: The data processor performs segmented processing on the heart sound signal through a sliding window to obtain a segmented signal.[1] S103: The data processor converts the segmented signal from the time domain to the frequency domain to generate a segmented frequency spectrum.

[0029] S104: The data processor inputs the segmented frequency spectrum into a trained deep learning model to obtain a classification result. The deep learning model takes a residual convolutional neural network as a basic network architecture and comprises an attention mechanism module. The attention mechanism module is used to enhance the attention to key features in the segmented frequency spectrum.

[0030] S105: The communicator uploads the classification result to a backend server and / or a personal terminal.

[0031] It should be noted that the sound signal generated by the mechanical activities such as myocardial contraction, valve closing and blood flow can be heard or detected through the body surface to obtain the heart sound signal. The intensity of the heart sound signal is affected by many factors, such as the contraction force of the heart, the elasticity of the valve and the thickness of the chest wall. Moreover, the individual differences of the heart sound signal are large. The heart sound signals of people with different ages, genders and body types may be significantly different. The heart sound signal can be converted from the time domain to the frequency domain by the Fast Fourier Transform (FFT) in the embodiment of the present application. The heart sound signal can also be converted from the time domain to the frequency domain by the Short-Time Fourier Transform (STFT) or other means to achieve the same purpose. The embodiment of the present application does not limit this.

[0032] In some possible embodiments, the heart sound signal is segmented and processed by a sliding window to obtain a segmented signal, and the segmented signal comprising 5 seconds of continuous heart sound signal is obtained. Then the data processor converts the segmented signal from the time domain to the frequency domain to generate a segmented frequency spectrum. It should be noted that the original heart sound signal is converted from the time domain to the frequency domain, and the 5-second sliding window is used for segmented processing, which can effectively capture long-time dynamic characteristics. This method not only solves the problem of insufficient feature extraction in the traditional heart sound classification method, but also provides more abundant input information for the subsequent deep learning model.

[0033] It should be noted that the classification result in the embodiments of the present application can include a normal category and an abnormal category, or other classification categories, which are not limited in the embodiments of the present application. The heart sound classification method provided by the embodiments of the present application is suitable for various types of heart sound classification tasks (such as normal heart sound, arrhythmia, etc.), has a wide application prospect, and does not need to design complex features manually, reduces human intervention, and can improve the efficiency of heart sound classification.

[0034] The deep learning model in the embodiments of the present application can take ResNet18 network as a basic network architecture. ResNet18 effectively alleviates the gradient disappearance problem in deep network by introducing a residual learning module, and improves the learning ability and convergence speed of the model. The present application classifies heart sound features through the ResNet18 network, which not only ensures the high performance of the deep learning model, but also reduces the demand for computing resources.

[0035] The attention mechanism module (Convolutional Block Attention Module, CBAM) in the embodiments of the present application can include a channel attention module and a spatial attention module. In actual application, the channel attention module is used to enhance the attention to the key channels in the segmented spectrogram, and the spatial attention module is used to enhance the attention to the key positions in the segmented spectrogram. It should be noted that the two attention mechanism modules are in a cascaded relationship, and the present application can first strengthen the channels, and then select more important spatial positions after selecting more important channels. The embodiments of the present application optimize the spectrogram through the channel attention module and the spatial attention module, which can automatically identify and highlight the key frequency information and time position, thereby significantly improving the detection ability of the deep learning model for heart sound abnormalities.

[0036] The output layer of the deep learning model in the embodiments of the present application can adopt a full connection layer, which maps the extracted features to the pre-defined heart sound categories, and optimizes the classification result through a cross-entropy loss function. As a possible implementation, the present application can also use an Adam (Adaptive Moment Estimation) optimizer to update the parameters, which automatically adjusts the learning rate during the training process, thereby improving the training efficiency and convergence speed.

[0037] Figure 2 A heart sound signal processing schematic diagram provided by the embodiments of the present application is shown in FIG. 1. Figure 2As shown, after obtaining the heart sound signal, the embodiments of the present application standardize the data, which can specifically include constructing model data pipe segments, dividing samples, converting energy spectrum by short-time Fourier transform (STFT), and unifying dimensions, packing according to batch size (Batch Size), and then batch feeding into the deep learning model. The deep learning model includes the architecture of ResNet18 network and the CBAM module embedded in the architecture of ResNet18 network, and the deep learning model outputs the classification result [2]. As an example, "Healthy" represents normal, and "UnHealthy" represents abnormal. Figure 3 The accuracy of the heart sound classification method provided by the present application in training is shown as follows: Figure 3 As shown, the model training accuracy provided by the embodiments of the present application is above 96%.

[0038] In actual application, the deep learning model provided by the present application can be compressed and accelerated for the system of the heart sound classification device to which the heart sound classification method is applied, for example, using a lightweight architecture or a knowledge distillation technology to compress and accelerate the deep learning model provided by the present application.

[0039] In summary, the heart sound classification method provided by the present application can effectively capture the long-time dynamic characteristics of the heart sound signal by using a sliding window for segmentation processing and converting the heart sound signal from the time domain to the frequency domain to generate a heart sound spectrum graph. The present application also sets the residual convolutional neural network as the architecture of the deep learning model, and optimizes the multi-dimensional features of the segmented spectrum graph through the attention mechanism module, which can improve the learning ability and convergence speed of the deep learning model and significantly improve the classification ability of the model for heart sound features. In this way, the present application combines Fourier transform, sliding window segmentation technology and residual convolutional neural network based on deep learning, and optimizes feature expression through attention mechanism. This architecture design not only improves the classification accuracy of the deep learning model for heart sound, but also enhances the generalization ability and adaptability of the heart sound signal, and has strong robustness to complex background noise, which can effectively adapt to heart sound signals in different environments.

[0040] In the description of the present specification, the description referring to the terms "some possible embodiments", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application, and the above terms do not necessarily represent the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0041] With regard to the method flowchart of the embodiments of the present application, some operations are described as different steps executed in a certain order. Such flowcharts are illustrative rather than limiting. Some steps described herein can be grouped together and executed in a single operation, or some steps can be divided into multiple sub-steps, and some steps can be executed in an order different from that shown herein. Each step shown in the flowchart can be implemented in any way by any circuit structure and / or tangible mechanism (for example, by software running on a computer device, hardware (for example, processor or chip implemented logic function), etc., and / or any combination thereof) in any manner.

[0042] The person skilled in the art can understand that in the method described in the above specific embodiments, the writing order of each step does not mean a strict execution order, and the specific execution order of each step should be determined by its function and possible inherent logic.

[0043] According to the heart sound classification method provided in the above embodiments, the present application further provides a heart sound classification device.

[0044] As Figure 4 shown, the heart sound classification device provided by the present application comprises: a auscultation part, a data processor, a portable wearing part and a communicator, the portable wearing part is used to relatively fix the heart sound classification device with the human body.

[0045] The auscultation part 100 is used to obtain a heart sound signal; The data processor 200 is used to segment the heart sound signal by a sliding window to obtain a segmented signal; The data processor 200 is used to convert the segmented signal from time domain to frequency domain to generate a segmented spectrum graph; The data processor 200 is used to input the segmented spectrogram into the trained deep learning model to obtain the classification result. The deep learning model is based on the residual convolutional neural network architecture and includes an attention mechanism module. The attention mechanism module is used to enhance the attention to key features in the segmented spectrogram. The communicator 300 is used to upload the classification results to the backend server and / or personal terminal.

[0046] As one possible implementation, the attention mechanism module includes a channel attention module and a spatial attention module, which are cascaded in a specific order, with the channel attention module taking precedence over the spatial attention module. The channel attention module is used to enhance attention to key channels in the segmented spectrogram, while the spatial attention module is used to enhance attention to key locations in the segmented spectrogram.

[0047] As one possible implementation, the data processor is used to segment the heart sound signal through a sliding window to obtain a segmented signal including a continuous 5-second heart sound signal.

[0048] As one possible implementation, the output layer of the deep learning model uses a fully connected layer, which maps the extracted features to predefined heart sound categories and optimizes the classification results through a cross-entropy loss function.

[0049] As one possible implementation, the heart sound classification device further includes: an alarm, and the method further includes: When the classification result obtained by the data processor matches the first preset result, the data processor sends an alert signal to the alarm.

[0050] As one possible implementation, the communicator is used to send an alarm signal to the backend server when the classification result obtained by the data processor matches the second preset result; the backend server is used to send a distress signal to all user terminals associated with the user account after receiving the alarm signal.

[0051] As one possible implementation, the deep learning model is based on the ResNet18 network architecture.

[0052] It should be noted that the apparatus in the embodiments of this application can implement the various processes of the aforementioned method embodiments and achieve the same effects and functions, which will not be repeated here.

[0053] According to some embodiments of this application, a heart sound classification device according to an embodiment of this application is provided for performing... Figure 1The heart sound classification method shown, the device comprises: at least one processor; And, the memory connected with at least one processor in communication;Wherein, the memory has the instruction that can be executed by at least one processor, the instruction is executed by at least one processor, to make at least one processor can execute the method described in the above embodiment.

[0054] According to some embodiments of the application, a non-volatile computer storage medium of heart sound classification method is provided, and computer executable instructions are stored on the non-volatile computer storage medium, the computer executable instructions are set to execute the following when running by the processor: the method described in the above embodiment.

[0055] The computer readable medium includes permanent and non-permanent, removable and non-removable media, and can be realized by any method or technology to store information. Information can be computer readable instructions, data structure, program module or other data. Examples of computer readable storage medium include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage device or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. In addition, although the operations of the method of the application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the shown operations must be performed to achieve the desired results. In addition, some steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of sub-steps.

[0056] Although the spirit and principles of the application have been described with reference to several specific embodiments, it should be understood that the application is not limited to the disclosed specific embodiments, and the division of aspects does not mean that the features in these aspects cannot be combined. The application is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A method for classifying heart sounds, characterized in that, The method is applied to a heart sound classification device, which includes an auscultation unit, a data processor, a portable wearable unit, and a communicator. The portable wearable unit is used to fix the heart sound classification device relative to a human body. The method includes: The heart sound signals obtained by the auscultation unit; The data processor segments the heart sound signal using a sliding window to obtain segmented signals; The data processor converts the segmented signal from the time domain to the frequency domain, generating a segmented spectrum. The data processor inputs the segmented spectrogram into the trained deep learning model to obtain the classification result. The deep learning model is based on a residual convolutional neural network architecture and includes an attention mechanism module. The attention mechanism module is used to enhance the attention to key features in the segmented spectrogram. The communicator uploads the classification results to the backend server and / or personal terminal.

2. The method according to claim 1, characterized in that, The attention mechanism module includes a channel attention module and a spatial attention module. The channel attention module and the spatial attention module are cascaded in a specific order, with the channel attention module taking precedence over the spatial attention module. The channel attention module is used to enhance attention to key channels in the segmented spectrum, and the spatial attention module is used to enhance attention to key positions in the segmented spectrum.

3. The method according to claim 1, characterized in that, The data processor segments the heart sound signal using a sliding window to obtain segmented signals, including: The data processor segments the heart sound signal using a sliding window to obtain a segmented signal that includes a continuous 5-second heart sound signal.

4. The method according to claim 1, characterized in that, The output layer of the deep learning model is a fully connected layer, which maps the extracted features to predefined heart sound categories and optimizes the classification results through the cross-entropy loss function.

5. The method according to claim 1, characterized in that, The heart sound classification device further includes an alarm, and the method further includes: When the classification result obtained by the data processor matches the first preset result, the data processor sends an alert signal to the alarm.

6. The method according to claim 5, characterized in that, The method further includes: When the classification result obtained by the data processor matches the second preset result, an alarm signal is sent to the backend server through the communicator. After receiving the alarm signal, the backend server sends a distress signal to all user terminals associated with the user account.

7. The method according to any one of claims 1-6, characterized in that, The deep learning model is based on the ResNet18 network architecture.

8. A heart sound classification device, characterized in that, The heart sound classification device includes an auscultation unit, a data processor, a portable wearable unit, and a communicator. The portable wearable unit is used to fix the heart sound classification device relative to the human body. The auscultation unit is used to obtain heart sound signals; The data processor is used to segment the heart sound signal through a sliding window to obtain segmented signals; The data processor is used to convert the segmented signal from the time domain to the frequency domain and generate a segmented spectrum. The data processor is used to input the segmented spectrogram into a trained deep learning model to obtain a classification result. The deep learning model is based on a residual convolutional neural network architecture and includes an attention mechanism module. The attention mechanism module is used to enhance the attention to key features in the segmented spectrogram. The communicator is used to upload the classification results to a backend server and / or a personal terminal.

9. The heart sound classification device according to claim 1, characterized in that, The attention mechanism module includes a channel attention module and a spatial attention module. The channel attention module and the spatial attention module are cascaded in a specific order, with the channel attention module taking precedence over the spatial attention module. The channel attention module is used to enhance attention to key channels in the segmented spectrum, and the spatial attention module is used to enhance attention to key positions in the segmented spectrum.

10. A computer-readable storage medium storing a program that, when executed by a multi-core processor, causes the multi-core processor to perform the method as described in any one of claims 1-7.

Images