A heart sound classification system based on quaternion deep learning framework

Abstract

The present invention discloses a heart sound classification system based on a quaternion deep learning framework, comprising the following steps: (1) using a quaternion dynamic residual convolution module (QDRCM) to extract potential features of heart sounds; (2) using a quaternion local attention connection learning framework (QLACL) to expand attention to the quaternion space and maintain channel relationships; (3) using a global attention connection module (GCACM) to learn global channel relationships. In the present invention, the quaternion takes into account the channel relationship of the input data due to its characteristic that four elements constitute a quaternion entity, and better preserves the mutual relationship between channels during the learning process. Compared with a normal real-valued network, the number of parameters is reduced to 25%, and the weight distribution characteristics of the Hamiltonian product of the quaternion are used to capture the internal relationship between channels, and quaternion components are used to share weights and learn internal relationships, so as to achieve lightweight, fast and efficient diagnosis and classification of heart sounds.

Description

Technical Field

[0001] The present invention relates to the technical field of physiological signal recognition, and in particular to a heart sound recognition system based on a quaternion dynamic residual convolution module and a quaternion local attention connection learning framework. Background Art

[0002] In recent years, with the development of productivity and the improvement of living standards, various "rich diseases" have begun to plague human society. According to statistics from the World Health Organization, cardiovascular disease (CVD) has become the most important health killer facing modern humans. Cardiovascular disease generally refers to ischemic or hemorrhagic diseases of the heart and systemic tissues caused by hyperlipidemia, blood viscosity, atherosclerosis, hypertension, etc. Cardiovascular disease is a common disease that seriously threatens the health of humans, especially middle-aged and elderly people over 50 years old. It has the characteristics of high morbidity, high disability rate and high mortality rate. Even with the most advanced and complete treatment methods, the number of people who die from cardiovascular disease each year in the world is as high as 15 million, ranking first among all causes of death.

[0003] Phonocardiogram (PCG) is a graph obtained by recording the heart sounds, i.e., different vibration waves, generated by the heart's contraction and relaxation cycles and transmitted to the chest wall. The phonocardiogram records the heart sounds and heart murmurs for clinical analysis, which helps to diagnose the cause of heart disease and understand the mechanism of heart murmurs. Heart sounds are of unique significance for the determination of cardiac function and the diagnosis of certain cardiovascular diseases, and cannot be replaced by other tests.

[0004] Heart sound signals are one of the medical signals that contain important physiological and pathological information. They are a basic method for clinically evaluating the functional status of the heart. Before certain cardiovascular diseases develop into obvious pathological characteristics, they will be reflected in the corresponding components of heart sounds. For example, coronary artery stenosis will produce abnormal murmurs during the diastole of heart sounds. However, the diagnosis of heart sounds is easily biased due to the personal experience and subjective judgment of doctors. In China, high-quality medical resources are often concentrated in cities, and remote rural areas often lack doctors with rich clinical auscultation experience. In these areas, doctors' diagnosis of heart sounds is not accurate, which in turn affects the subsequent treatment of patients.

[0005] Most of the current heart sound recognition algorithms are based on real-valued networks, while less attention is paid to complex-valued networks. In addition, most complex-valued neural networks are used in image processing. For example, Xunyu Zhu et al. proposed QCNN for color image restoration, and Shan Gai et al. proposed RQV-CNN for color image denoising and classification.

[0006] Now, with the advancement of technology, the research on heart sounds is more in-depth, the equipment is more advanced, the collection of heart sounds is more convenient and accurate, and it provides more powerful support for scholars to study and explore intelligent clinical diagnosis. Therefore, the research on an objective, accurate, and intelligent computer-aided diagnosis of heart sounds is of great significance. At present, MFCC-CNN, PSD-CNN, etc. have large computational complexity and time consumption, so studying a fast and intelligent computer-aided diagnosis method for heart sounds is an urgent problem to be solved. Summary of the invention

[0007] In order to solve the deficiencies in the above-mentioned prior art, the present invention uses a deep learning algorithm to provide a lightweight and accurate heart sound recognition system, namely, a heart sound recognition system based on a quaternion dynamic residual convolution module and a quaternion local attention connection learning framework.

[0008] The object of the present invention is achieved by comprising the following steps:

[0009] A heart sound classification system based on quaternion deep learning framework, the main steps are as follows:

[0010] Step 1: Preprocess the heart sound signal to obtain the heart sound MFCC features and splice them into input data;

[0011] Step 2: Extract features of heart sound signals (QDRCM);

[0012] Step 3: Global channel characteristics (GCACM);

[0013] Step 4: Original feature assistance;

[0014] Step 5: Fully connected classification;

[0015] The step 1 uses a second-order Butterworth bandpass filter to filter the original data, remove interference, normalize and slice the filtered signal, extract MFCC features from the sliced ​​signal, and combine them into the final heart sound MFCC features to constitute the input signal of the network;

[0016] The input signal of the network formed in step 1 is passed into the built framework for learning; the framework mainly includes: quaternion dynamic residual convolution module (QDRCM), quaternion local attention connection learning module (QLACL), global channel attention connection module (GCACM), original feature assistance module (OFAM) and fully connected classification module;

[0017] The quaternion dynamic residual convolution module (QDRCM) introduces convolution into the complex value field. In the convolution space, quaternions can maintain the relationship between channels and greatly reduce the number of parameters. Dynamic convolution can dynamically predict the convolution kernel, increasing the network complexity without increasing the network depth; the quaternion local attention connection learning module (QLACL) is mainly connected to the attention layer in the quaternion dynamic residual module, connecting adjacent attention blocks, so that information can flow between attention blocks and propagate information adaptively; since the input data has rich channel information, the one-dimensional energy part of the data is extracted and sent to the original feature assistance module (OFAM) for dimensional transformation and feature extraction, and finally connected to the fully connected classification module; since the quaternion is a quaternion entity composed of 4 channels, only the relationship between the 4 channels can be paid attention to, and the global channel attention module (GCACM) replaces the convolution in the quaternion dynamic residual convolution module with ordinary convolution, which can extract the global channel relationship.

[0018] Each weight in the convolution layer of the Quaternion Dynamic Residual Convolution Module (QDRCM) is a quaternion entity. The weight distribution property of the Hamilton product is used to capture its internal relationship. By using the Hamilton product, the quaternion weight components are shared through multiple quaternion inputs to create and learn relationships in the elements. Quaternion dynamic convolution dynamically predicts the convolution kernel through the quaternion convolution layer. The convolution kernels are dynamically gathered together according to the attention of multiple parallel convolution kernels, increasing the complexity of the model without increasing the depth or width of the network. Using the residual module, the input is jumped to the output position after 3 quaternion convolutions and dynamic convolutions and the attention layer to avoid the disappearance or explosion of the gradient, and connect the previous and next features to avoid network degradation.

[0019] The quaternion local attention connection learning module (QLACL) is to expand the attention to the quaternion space and extract the interesting part of the network under the premise of ensuring the channel correlation. Then the attention layers in the quaternion dynamic residual convolution module are connected so that the attention can flow between adjacent attention blocks and propagate messages adaptively. This connection is added to the context conversion part of the attention layer to ensure that the current attention is learned from the current module features and the previous attention information together. This can ensure that during the network learning process, the loss of channel relationships is greatly reduced and useful relationships are retained.

[0020] The global channel attention connection module (GCACM) can only focus on the relationship between the four channels due to the characteristic that the quaternion is a quaternion entity composed of four channels. The use of real-valued convolution can focus on the global channel relationship. Here we change the quaternion convolution in the quaternion dynamic residual convolution module to a real-valued convolution, let it learn the global channel features, and then connect it to the fully connected module to realize the global channel feature connection and reduce the channel relationship loss in learning.

[0021] The original feature assistance module (OFAM) extracts the one-dimensional energy part of the data because the input data has rich channel information, and sends it to the original feature assistance module for dimensionality transformation and feature extraction, and finally connects to the fully connected classification module.

[0022] Positive and beneficial effects: The present invention discloses a heart sound recognition system based on a quaternion dynamic residual convolution module and a quaternion local attention connection learning framework, which extends the heart sound recognition method to the complex value field. Due to its characteristics, the quaternion can treat four channels as a quaternion entity, retain the channel relationship in the network, and reduce the network parameter amount to 25% while ensuring accuracy. The present invention introduces a quaternion dynamic residual convolution module and uses quaternion dynamic separable convolution to increase the network complexity and learn more useful features. The quaternion local attention connection learning module can reduce the loss of channel relationships and improve accuracy in the learning of the network. Through experimental testing, the network's recognition rate for heart sound signals can reach 97%, and the parameter amount is 25% of that of the real-valued neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is the overall flow chart of the present invention;

[0024] Figure 2 It is the quaternion local attention connection learning module in the present invention. DETAILED DESCRIPTION

[0025] The present invention is described in detail below in conjunction with the accompanying drawings, and mainly comprises the following steps:

[0026] Step 1: Preprocess the heart sound signal to obtain the heart sound MFCC features and splice them into input data.

[0027] Step 2: Feature extraction of heart sound signals (QDRCM)

[0028] like Figure 1The QDRCM shown in the figure is a quaternion dynamic residual convolution module, which consists of 3 quaternion convolution layers, 1 quaternion dynamic convolution and an attention layer; the convolution kernel sizes of the 3 quaternion convolutions are set to 1, 3, and 1 respectively, with a step size of 1, and padding is used to ensure that the length of the data remains unchanged during the convolution process.

[0029] Quaternions are composed of a real number plus three imaginary units i, j, and k.

[0030] like , where i, j, and k have the following relationship:

[0031]

[0032] If the quaternion is regarded as a matrix, it can be expressed as follows:

[0033]

[0034] Then the quaternion convolution can be expressed as the matrix multiplication form shown in the figure below:

[0035]

[0036] From this, we can see that in quaternion convolution, all weights are quaternions, and the weight distribution characteristics of Hamilton product are used to capture their internal relationships. By using Hamilton product, quaternion weight components are shared through multiple quaternion inputs, and the quaternion network treats each element as an entity of four related elements. Therefore, internal relationships are naturally learned through this process.

[0037] The output features obtained from the quaternion convolution are input into the quaternion dynamic convolution module, allowing the network to adaptively learn the convolution kernel based on the input information and obtain deeper feature information. We extend the dynamic convolution to the complex value field. This layer dynamically predicts the convolution kernel through the quaternion convolution layer. The convolution kernel is dynamically aggregated according to the attention of multiple parallel convolution kernels, increasing the complexity of the model without increasing the depth or width of the network, and retaining the channel characteristics. Its mathematical expression is:

[0038]

[0039] Therefore, compared with static convolution, quaternion dynamic convolution has stronger feature expression ability. Quaternion dynamic convolution is a generalization of the convolution layer. This module can obtain the output feature map of this layer based on the feature map of the previous layer, which can significantly improve the model's expression ability and performance.

[0040] Finally, the features are passed to the attention layer, the parameters of which are also quaternions. In this layer, the local attention connection learning module is used, such as Figure 2 As shown in the figure, we extend attention to the complex field, use MaxPool and AvgPool to extract the context of features, use the quaternion convolution layer to convert the context of attention, and then connect it with the attention of the previous layer and send it to the convolution layer to learn the attention of this layer. That is, the attention of this layer is learned by connecting the attention of the upper layer and the features of this layer. Such attention can be expressed as:

[0041]

[0042] This can ensure that during the network learning process, the loss of features and the destruction of channel relationships are effectively reduced, and good performance is maintained when the depth of the network increases.

[0043] Step 3: Global Channel Characterization (GCACM)

[0044] like Figure 1 The GCACM shown in is a global channel attention connection module. Since the quaternion is composed of four channels to form a quaternion entity, only local channel relationships can be paid attention to. Therefore, the present invention designs a global channel attention connection module, which replaces the quaternion convolution in the quaternion dynamic residual convolution module with a real-valued convolution, focusing on the global channel characteristics of the data, wherein the convolution is used to learn potential features, and the attention layer can focus on the useful parts of the network to improve the model effect. The residual structure can effectively avoid network degradation and gradient explosion or disappearance.

[0045] The input data is directly connected to the global channel attention connection module, and after obtaining the output features, it is connected to the fully connected classification module of the network.

[0046] Step 4: Primitive feature assistance

[0047] Since the input signal has rich potential features, we built Figure 1 The original feature assistance module shown in the figure directly connects the one-dimensional energy signal of the input signal to the original feature assistance module. The module is mainly composed of a fully connected layer and is used to learn the original features of the data. After obtaining the output features, it is connected to the fully connected classification module of the network to supplement the original feature information for the output features after the quaternion dynamic residual convolution, which can effectively improve the network performance.

[0048] Step 5: Fully connected classification

[0049] The output features after step 2, step 3, and step 4 are connected and connected to the fully connected classification module. The module mainly consists of 4 fully connected layers, the activation function is ReLU, and the DropOut structure is used to randomly inactivate neurons to prevent overfitting, and finally obtain a 2-classification output result.

[0050] The present invention discloses a heart sound recognition system based on a quaternion dynamic residual convolution module and a quaternion local attention connection learning framework, which extends the heart sound recognition method to the complex value field. Due to its characteristics, quaternion can treat four channels as a quaternion entity, retain the relationship between channels in the network, and reduce the network parameters to 25% while ensuring accuracy. The present invention introduces a quaternion dynamic residual convolution module and uses quaternion dynamic separable convolution to increase network complexity and learn more useful features. The quaternion local attention connection learning module can reduce the loss of channel relationships and improve accuracy in network learning. Through experimental testing, the network's recognition rate for heart sound signals can reach 97%, and the parameter amount is 25% of that of a real-valued neural network.

Claims

1. A heart sound classification system based on quaternion deep learning framework, characterized in that: The system includes a quaternion dynamic residual convolution module QDRCM, a quaternion local attention connection learning module QLACL, a global channel attention connection module GCACM, an original feature assistance module OFAM and a fully connected classification module; the main steps of the system for heart sound classification are as follows: Step 1: preprocess the heart sound signal to obtain the heart sound MFCC features, and splice them into input data; Step 2: extract features from the heart sound signal using the quaternion dynamic residual convolution module QDRCM; Step 3: global channel attention connection module GCACM; Step 4: original feature assistance; Step 5: Fully connected classification; In step 1, the original data is filtered using a second-order Butterworth bandpass filter to remove interference, the filtered signal is normalized and sliced, and the sliced ​​signal is subjected to MFCC feature extraction of heart sounds, and the resulting sliced ​​signal is combined into the final MFCC feature of heart sounds, which constitutes the input signal of the network; The input signal of the network constructed in step 1 is passed into the constructed framework for learning; The framework mainly includes: quaternion dynamic residual convolution module QDRCM, quaternion local attention connection learning module QLACL, global channel attention connection module GCACM, original feature assistance module OFAM and fully connected classification module; The quaternion dynamic residual convolution module QDRCM introduces convolution into the complex value field. In the convolution space, the quaternion maintains the relationship between channels and greatly reduces the number of parameters. The dynamic convolution dynamically predicts the convolution kernel, which increases the network complexity without increasing the network depth. The quaternion local attention connection learning module QLACL is mainly connected to the attention layer in the quaternion dynamic residual module, which connects adjacent attention blocks so that information can flow between attention blocks and propagate information adaptively. Since the input data has rich channel information, the one-dimensional energy part of the data is extracted and sent to the original feature assistance module OFAM for dimensional transformation and feature extraction, and finally connected to the fully connected classification module. Since the quaternion is a quaternion entity composed of 4 channels, only the relationship between the 4 channels can be paid attention to, and the global channel attention module GCACM replaces the convolution in the quaternion dynamic residual convolution module with ordinary convolution to extract the global channel relationship. Each weight in the convolution layer of the quaternion dynamic residual convolution module QDRCM is a quaternion entity; the weight distribution characteristics of the Hamilton product are used to capture its internal relationship; by using the Hamilton product, the quaternion weight components are shared by multiple quaternion inputs to create and learn relationships in the elements; the quaternion dynamic convolution dynamically predicts the convolution kernel through the quaternion convolution layer, and the convolution kernels are dynamically gathered together according to the attention of multiple parallel convolution kernels, increasing the complexity of the model without increasing the depth or width of the network; using the residual module, after the input passes through 3 quaternion convolutions and dynamic convolutions, the attention layer jumps to the output position to avoid the disappearance or explosion of the gradient, and connects the previous and next features to avoid network degradation; The quaternion local attention connection learning module QLACL is to expand the attention to the quaternion space, extract the network's interesting part under the premise of ensuring channel correlation; then the attention layers in the quaternion dynamic residual convolution module are connected so that the attention flows between adjacent attention blocks and propagates messages adaptively; the connection is added to the context conversion part in the attention layer to ensure that the current attention is learned from the current module features and the previous attention information together; it ensures that in the process of network learning, the loss of channel relations is greatly reduced and useful mutual relations are retained;The global channel attention connection module GCACM can only focus on the relationship between the four channels due to the characteristic that the quaternion is a quaternion entity composed of four channels, and uses real-valued convolution to focus on the global channel relationship, and changes the quaternion convolution in the quaternion dynamic residual convolution module to real-valued convolution, learns the global channel features, and then connects to the fully connected module to achieve global channel feature connection and reduce the channel relationship loss in learning; The original feature assistance module OFAM, because the input data has rich channel information, extracts the one-dimensional energy part of the data, sends it to the original feature assistance module, performs dimensional transformation and feature extraction, and finally connects to the fully connected classification module. ;

Images