Cardiac electrophysiological signal classification method and system based on electrocardiogram and magnetocardiogram multi-scale fusion

Abstract

The heart electrophysiological signal classification method comprises the steps that electrocardiosignals and magnetocardiogram signals are obtained, filtering and denoising preprocessing is conducted on original ECG signals and original MCG signals respectively, heart beats are divided through R-wave positioning, a two-dimensional spectrogram is obtained after Fourier transform, and the two-dimensional spectrogram is used for classifying the electrocardiosignals and the magnetocardiogram. After feature extraction, respectively establishing a multi-domain cardiac function signal data set; a convolutional recurrent neural network is constructed, synchronous processing of two different forms of heart electrophysiological signals is realized through a double-input model, shallow signal features extracted by the convolutional neural network are input into a time sequence feature extraction module formed by the recurrent neural network, local saliency features and time sequence features of ECG and MCG signals are fused, and a time sequence feature extraction module is constructed. And finally, ECG and MCG signal features for classification are formed. And inputting the fused features into a nonlinear classification layer, performing model training by using Softmax in combination with a cross entropy loss function, and outputting a classification result.

Description

Technical Field

[0001] The present invention belongs to the field of signal processing and the technical field of joint analysis of electrocardiogram (ECG) signals and magnetic heart signals, and specifically relates to a cardiac electrophysiological signal classification method and system based on multi-scale fusion of electrocardiogram (ECG) signals and magnetic heart signals. Background Art

[0002] ECG and MCG are homologous heterogeneous signals generated by the transmembrane movement of ions in myocardial cells. Compared with the attenuation of ECG tissue conduction and the polarization voltage generated by skin contact, MCG signals are derived from the measurement of the magnetic field around the heart. They are not affected by tissues, blood, etc. during the conduction process, and can enrich the representation of cardiac electrophysiological information, provide complementary information to the electrocardiogram, and have higher sensitivity and specificity. Different physiological signal reactions have different physiological mechanisms. Therefore, through the joint analysis of different physiological signals, multi-angle information can be provided for the identification of different types of cardiac electrophysiological signals, which is more accurate than the judgment method of a single signal.

[0003] Compared with the traditional machine learning classification method of manually extracting features, neural networks can automatically mine deep features in the data from the original time series and improve the performance of the model by sharing the underlying features. Most of the existing models are based on the recognition of a single ECG signal. Considering the difference between ECG signals and magnetic heart signals, a method for cardiac electrophysiological signal recognition based on a dual-input convolutional recurrent neural network model is proposed. A reasonable structure is designed to extract the significant features and time series features of ECG and MCG signals to improve the performance of the multi-input classification model. Research on the application of deep learning technology in the classification task of integrating two physiological signals usually refers to the network structure in the ECG multi-classification research task. ECG and MCG signals are both time series signals. Generally, the local features of the heartbeat waveform are recognized by a deep convolutional neural network (CNN), and the time series features are extracted by a recurrent neural network (RNN). These two types of network structures are usually combined using CNN and LSTM. LSTM is a RNN variant structure proposed to solve the problems of gradient vanishing and gradient exploding. GRU is simpler in structure and has fewer parameters than LSTM, but in practice there is no significant performance gap between it and LSTM, and it may even perform better in some tasks. Therefore, it is also a more popular RNN variant structure. Summary of the invention

[0004] In order to solve the problems existing in the prior art, the present invention provides a cardiac electrophysiological signal classification method and system based on multi-scale fusion of electrocardiogram and magnetic heart, aiming to achieve more accurate cardiac electrophysiological signal classification by combining multiple signals and multi-domain features and integrating the advantages of different signals.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] The present invention provides a cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetic heart, comprising the following steps:

[0007] Acquire the original ECG signal and the original MCG signal, pre-process the original ECG signal and the original MCG signal respectively, divide the heartbeat according to the R wave positioning, extract the heartbeat signal, and obtain the corresponding frequency domain information after Fourier transformation of the extracted heartbeat signal, wherein the frequency domain information includes: one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features;

[0008] The obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features are input into the convolutional recurrent neural network for shallow feature extraction. Taking ECG and MCG signals as input, a convolutional recurrent neural network with two parallel branches is designed to realize the joint analysis of ECG and MCG signals.

[0009] By integrating the characteristics of a single heartbeat cycle, i.e., heartbeat, in the time domain and frequency domain, the cardiac electrophysiological signals are comprehensively classified and detected.

[0010] Preferably, obtaining the original ECG signal, preprocessing the original ECG signal, dividing the heartbeat according to the R wave positioning, extracting the heartbeat signal, and obtaining the corresponding frequency domain information after Fourier transforming the extracted heartbeat signal includes:

[0011] Wavelet threshold denoising is used to preprocess the original ECG signal to remove the original noise of the data, filter out baseline drift and power frequency interference, and use multi-order Daubechies wavelets to perform wavelet decomposition on the input signal to obtain coefficients at different levels, where the coefficients include approximate coefficients and detail coefficients. The denoising threshold λ is calculated based on the detail coefficients, and the wavelet coefficients of the two highest-frequency high-frequency components are set to zero, and the coefficients less than the threshold are set to zero to remove noise. The processed coefficients are converted back to the time domain signal using inverse wavelet transform to obtain the entire denoised ECG signal and the preprocessed time series S1.

[0012] With a heartbeat of fixed length as input, the data points with a length of Fs before and after the R wave in the time series data S1 are intercepted as a heartbeat cycle signal, thereby dividing the heartbeat;

[0013] Perform FFT on each heartbeat cycle signal to obtain the spectrum of the heartbeat signal, perform STFT on each heartbeat cycle signal to obtain the time-frequency spectrum of the heartbeat, and then construct a one-dimensional ECG signal, a two-dimensional ECG spectrum, and a two-dimensional ECG time-frequency spectrum dataset;

[0014] The calculation method of the threshold λ is:

[0015]

[0016] Where D is the processed wavelet coefficient, C is the signal length, and median means taking the median;

[0017] Among them, the data points with a length of Fs before and after the R wave in the time series data S1 are intercepted as a heartbeat cycle signal including:

[0018] The Pan-Tompkins detection algorithm is used to locate the R wave, and then the R wave position is used as a reference point to move forward. sampling points, take the Sampling points are taken, and a total of Fs sampling points are intercepted as the length of a complete heartbeat cycle, where Fs is the sampling rate.

[0019] Preferably, obtaining the original magnetic heart signal, preprocessing the original magnetic heart signal, dividing the heartbeat according to the R wave positioning, extracting the heartbeat signal, and obtaining the corresponding frequency domain information after Fourier transforming the extracted heartbeat signal includes:

[0020] The original magnetic cardiogram signal was denoised and baseline drift was removed using a 1-40 Hz bandpass filter and a 300-window median filter.

[0021] The R wave is enhanced using the Pan-Tompkins detection algorithm, and the position of the R wave is detected by calculating the amplitude, slope, and waveform characteristics of the signal;

[0022] After determining the window size of the heartbeat, each heartbeat cycle signal is extracted, and the corresponding frequency domain information is obtained through Fourier transform. Then, a one-dimensional MCG signal, a two-dimensional MCG spectrum graph, and a two-dimensional MCG time-spectrum graph dataset are constructed.

[0023] Preferably, the convolutional recurrent neural network adopts a three-module combination structure, wherein the first module is a dual-input convolution module, comprising two parallel convolution branches, each branch having the same structure, comprising a convolution layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer; the second module is a temporal feature extraction module composed of a gated recurrent unit GRU, including a reset gate and an update gate; the third module is a decision module, comprising a flat layer, a fully connected layer, and a dense layer, and finally outputs a multi-classification probability distribution through a Softmax function to obtain a classification result.

[0024] Preferably, the dual-input convolution module adopts one-dimensional and two-dimensional CNN models as the network structure respectively, and two cardiac electrophysiological signals are simultaneously used as the input of the model; the ECG, MCG signal heart beat or spectrum diagram, and time-spectrum diagram are respectively input into three independent network branches. For the one-dimensional CNN model, a batch normalization layer is used after the first convolution layer to standardize the data of each batch; the maximum pooling layer placed after batch normalization is used to reduce the feature dimension; the second and third convolution layers respectively increase the number of convolution kernels to extract more features, and other parameters are the same as the first convolution layer; for the two-dimensional CNN model, a two-dimensional convolution kernel is used for feature extraction.

[0025] Preferably, the time series feature extraction module adopts a gated recurrent unit GRU as the network structure; the input is the shallow signal feature extracted by the dual-input convolution module, and the output is the extracted time series feature.

[0026] Preferably, in the decision module, a convolution layer is first added to merge the results of different inputs from the previous layer, and then the merged results are nonlinearly transformed using the Softmax activation function; thereafter, the model uses a convolution kernel with the same output size as the previous step as a fully connected layer, convolves with the output of the Softmax activation function, and uses a dropout layer and early stopping strategy in the fully connected layer to prevent overfitting.

[0027] The present invention also provides a cardiac electrophysiological signal classification system based on multi-scale fusion of electrocardiogram and magnetic heart, the system is used to implement the method described, comprising: a preprocessing module, a fusion module and a classification module;

[0028] The preprocessing module is used to obtain the original ECG signal and the original magnetic heart signal, preprocess the original ECG signal and the original magnetic heart signal respectively, divide the heartbeat according to the R wave positioning, extract the heartbeat signal, and obtain the corresponding frequency domain information after Fourier transformation of the extracted heartbeat signal, wherein the frequency domain information includes: one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features;

[0029] The fusion module is used to input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into the convolutional recurrent neural network for shallow feature extraction, and the convolutional recurrent neural network with two parallel branches is designed to realize the joint analysis of the ECG signal and the MCG signal with ECG and MCG signals as input respectively;

[0030] The classification module is used to comprehensively classify and detect cardiac electrophysiological signals by fusing the characteristics of a single heartbeat cycle, ie, heartbeat, in the time domain and the frequency domain.

[0031] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention can help better understand and process complex signals for non-steady-state cardiac electrophysiological signals by fusing the time domain and frequency domain information of the ECG signal and the magnetic heart signal, and fusing the time and frequency domain information for analysis; and automatically identifies the features of the input signal using a deep neural network model, thereby improving the accuracy of the classification model. The joint analysis of multiple signals and multi-domain features can provide higher accuracy for performing classification tasks.

[0032] (2) The present invention implements a dual-input model through two parallel convolutional neural networks and expands the network width through a multi-branch network structure. Two parallel branches are designed to extract the features of one-dimensional ECG signals and MCG signals, two-dimensional ECG spectrum and MCG spectrum, and two-dimensional ECG time-frequency spectrum and MCG time-frequency spectrum and effectively fuse them. One of them inputs one-dimensional data, and its function is to extract the time domain features in ECG and MCG signals; the other two input two-dimensional images, and their function is to extract the frequency domain information in ECG and MCG signals; the output results of the three branches are spliced, and two forms of information fusion representation are obtained after channel dimensionality reduction. By fusing ECG and MCG signals and combining the attention mechanism, the utilization capacity of ECG and MCG signals in space, time and frequency domains is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0034] Figure 1 It is a flow chart of a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and magnetic heart;

[0035] Figure 2 This is a time domain diagram of the preprocessed signal in a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and MCG, where (A) is the ECG signal and (B) is the MCG signal;

[0036] Figure 3 This is a preprocessed signal spectrum diagram in a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and MCG, where (A) is an ECG signal and (B) is an MCG signal;

[0037] Figure 4 It is a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and MCG. The heartbeat cycle time-frequency spectrum diagram, where (A) is the ECG signal and (B) is the MCG signal;

[0038] Figure 5It is a schematic diagram of the overall structure of a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and magnetic heart;

[0039] Figure 6 This is a schematic diagram of the structure of the residual block in the cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and MCG, where (A) is a one-dimensional CNN residual block and (B) is a two-dimensional CNN residual block;

[0040] Figure 7 It is a schematic diagram of the structure of a gated recurrent unit in a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and magnetic heart;

[0041] Figure 8 It is a structural schematic diagram of the SE module in a cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and magnetic heart. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] Embodiment 1

[0045] The present invention provides a cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetic heart, comprising the following steps:

[0046] (1) The original ECG signal is preprocessed, and the R wave is located using the Pan-Tompkins detection algorithm. A single heartbeat cycle is extracted through the R wave position of each heartbeat cycle. The extracted heartbeat signal is subjected to Fourier transform to obtain a spectrum diagram and a time-frequency spectrum diagram, and a one-dimensional ECG time domain signal, a two-dimensional ECG spectrum diagram, and a two-dimensional ECG time-frequency spectrum diagram dataset are established respectively;

[0047] (2) Collect the original magnetic heart signals for preprocessing, use the Pan-Tompkins detection algorithm to locate the R wave, divide the heartbeats, and obtain the frequency domain information after the above-mentioned Fourier transform, and establish the one-dimensional MCG time domain signal, two-dimensional MCG spectrum, and two-dimensional MCG time-frequency spectrum data sets respectively;

[0048] (3) Considering the difference in time scale between ECG signal and MCG signal, the present invention uses a dual-input convolution module to extract the implicit features of different signals, which is achieved by using different convolution modules for different inputs. A first network branch is constructed for the above-mentioned one-dimensional time series data, and the branch model includes two parallel one-dimensional convolution modules as feature extraction modules, and a GRU network as a time series feature learning module; a second network branch is constructed for the two-dimensional spectrum graph, and the branch model includes two parallel two-dimensional convolution modules and a fully connected layer; a third network branch is constructed for the two-dimensional time-frequency spectrum graph, and the branch model includes two parallel two-dimensional convolution modules and a fully connected layer. Finally, the three branch networks are connected, and Softmax is used in combination with the cross entropy loss function, and the gradient descent method is used to train the model. After the training is completed, the test set is used for testing, and the classification result is output.

[0049] Furthermore, the step (1) comprises:

[0050] (11) Wavelet threshold denoising is used to preprocess the original ECG signal to remove the original noise of the data, filter out baseline drift and power frequency interference, use multi-order Daubechies wavelet to decompose the input signal, obtain coefficients at different levels (including approximate coefficients and detail coefficients), calculate the denoising threshold λ according to the detail coefficient, set the wavelet coefficients of the two highest frequency high-frequency components to zero, and set the coefficients less than the threshold to zero to remove noise, use inverse wavelet transform to convert the processed coefficients back to the time domain signal, obtain the entire ECG signal after denoising, and obtain the preprocessed time series S1;

[0051] (12) The model requires a heartbeat of fixed length as input, so it is necessary to intercept the data points with a length of Fs before and after the R wave in the time series data S1 as a heartbeat cycle signal to divide the heartbeat;

[0052] (13) Perform FFT on each heartbeat cycle signal to obtain the spectrum of the heartbeat signal, perform STFT to obtain the time-frequency spectrum of the heartbeat, and then construct a one-dimensional ECG signal, a two-dimensional ECG spectrum, and a two-dimensional ECG time-frequency spectrum data set to facilitate subsequent model training and analysis.

[0053] Furthermore, in step (11), the threshold λ is calculated as follows:

[0054]

[0055] Among them, D is the processed wavelet coefficient, C is the signal length, and median means taking the median.

[0056] Furthermore, the step (12) specifically includes the steps of:

[0057] According to the empirical value, based on the R peak forward Points, backward The length of a heartbeat cycle is exactly one point.

[0058] Furthermore, in step (13), the calculation method of FFT is as follows:

[0059] Call the function to perform FFT transformation on the input signal, convert the time domain signal into a frequency domain signal, which contains the amplitude information of each frequency; calculate the frequency range. For real number signals, the spectrum is symmetrical, so only the positive frequency part is taken. By drawing the spectrum graph, the intensity of different frequency components in the cardiac electrophysiological signal can be observed.

[0060] The STFT is calculated as follows:

[0061] STFT usually relies on FFT algorithm, which divides the whole signal into several segments, and then performs Fourier transform on each segment to extract the frequency information of each segment; in STFT, the choice of window will affect the trade-off between time resolution and frequency resolution, and the length of each window is 64 sampling points, that is, each Fourier transform uses 64 continuous sampling points to calculate the spectrum of the segment; the overlap between windows is 32 sampling points, which can provide higher time resolution and reduce the distortion of the signal on the time axis; through logarithmic transformation, zero values ​​are avoided when visualizing the time spectrum diagram. The short-term nature of STFT makes it suitable for analyzing non-steady-state signals such as abnormal cardiac electrophysiological signals.

[0062] Furthermore, the step (2) comprises:

[0063] (21) The original magnetic cardiogram signal was denoised and baseline drift was removed using a 1-40 Hz bandpass filter and a 300-window median filter;

[0064] (22) The Pan-Tompkins detection algorithm mainly includes preprocessing to achieve R wave enhancement, such as differential operation after bandpass filtering, and then detecting the position of the R wave by calculating the amplitude, slope, and waveform characteristics of the signal;

[0065] (23) After determining the window size of the heartbeat, each heartbeat cycle signal is extracted, and its frequency domain information is obtained through Fourier transform. Then, a one-dimensional MCG signal, a two-dimensional MCG spectrum graph, and a two-dimensional MCG time-frequency spectrum graph dataset are constructed.

[0066] Furthermore, the step (3) comprises:

[0067] (31) Establish three network branches with dual-input convolution modules, where the dual-input convolution module consists of two parallel convolution branches; for the one-dimensional CNN model, the first convolution layer uses 16 convolution kernels of size 5 and a sliding step of 1; the batch normalization layer is used to standardize the data of each batch to speed up the training; the maximum pooling layer is used to reduce the feature dimension, the pooling window size is 2, and the sliding step is 2; the second and third convolution layers increase the number of convolution kernels to 32 and 64 respectively to extract more features, and the other parameters are the same as the first convolution layer; for the two-dimensional CNN model, the first convolution layer uses 32 convolution kernels of size 3×3 and a sliding step of 1; the second and third convolution layers use 64 convolution kernels of size 3×3 and a sliding step of 2;

[0068] (32) Input the one-dimensional ECG signal and MCG signal data sets obtained after preprocessing into the first and second branch convolution layers respectively; input the two-dimensional ECG spectrum map and MCG spectrum map into the first and second branch convolution layers of the two-dimensional convolution module; input the two-dimensional ECG time-frequency spectrum map and MCG time-frequency spectrum map into the first and second branch convolution layers of another two-dimensional convolution module.

[0069] (33) Establish a GRU temporal feature extraction module and introduce an attention mechanism based on the SE module;

[0070] (34) At the end of the network, the output results of the one-dimensional and two-dimensional network branches are merged through the fully connected layer, and the output values ​​are converted into prediction probabilities using the Softmax layer, and the classification results are finally output.

[0071] Among them, each network branch convolution module includes two parallel one-dimensional convolution layers with a convolution kernel size of 5, a BatchNorm layer and a ReLu activation function; the GRU network includes 36 hidden units.

[0072] Specifically, the feature vector is reduced in dimension through a maximum pooling layer, and the step size of the maximum pooling layer is preferably 2; the GRU network can extract long-term dependency features in processing time series signals, the maximum pooling layer is connected to the GRU network, and normalized through a BatchNorm layer.

[0073] Furthermore, the one-dimensional ECG and MCG signals are respectively input into the one-dimensional convolutional layer, the time domain features are preliminarily learned, and the feature maps are output. Then, the feature maps of the two convolutional layers are connected as the input of the one-dimensional CNN model; the two-dimensional ECG and MCG frequency domain and time-frequency domain signals are respectively input into the two-dimensional convolutional layer, the frequency domain and time-frequency domain features are preliminarily learned, and the two-dimensional feature maps are output. Then, the feature maps of the two convolutional layers are connected as the input of the two-dimensional CNN model, so as to simultaneously reflect the mixed features of ECG and MCG characteristics.

[0074] In order to enhance the feature extraction capability of the model, the SE module is applied to the GRU network that can capture future and past information. The SE module structure is relatively simple, which can maintain the original network structure and performance, and can also increase the performance of the network model by introducing a small number of parameters.

[0075] Furthermore, the output vectors of the convolution module and GRU are directly concatenated to obtain fused features.

[0076] Furthermore, the above fusion features are input into the fully connected layer for integration, and finally a Softmax activation function is used to calculate the output classification result.

[0077] Furthermore, by setting parameters such as epoch, batchsize, loss function, optimizer, learning rate, etc., the training set data is continuously iterated and calculated. At the end of the training, the model with the best performance in the training process is selected and the model file is saved.

[0078] Embodiment 2

[0079] Reference Figure 1 The present invention provides a cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetic heart, comprising the following steps:

[0080] S1 obtains the original one-dimensional ECG signal and magnetic heart signal for preprocessing and divides the heart beats;

[0081] S2 performs Z-Score normalization on the heartbeat segmented in step S1, and constructs a one-dimensional ECG and MCG signal data set using the processed heartbeat;

[0082] S3 performs Fourier transform on the heartbeat signal in step S2 to obtain a two-dimensional ECG, MCG spectrum, and a two-dimensional ECG, MCG time-frequency spectrum data set;

[0083] S4 designs a classification model based on the time scale of the electrocardiogram and magnetic heart signals in the technical solution. Group convolution is used for the input in each layer; a fusion classification model containing a dual-channel CNN feature extractor, a GRU-based feature learner, and a feature classifier is constructed; the first layer of convolution blocks uses 16 convolution kernels for each signal, the length of the convolution kernel is set to 3, the sliding stride is 1, and no zero padding is used for the input during convolution. For two-dimensional features, a two-dimensional convolution kernel with a shape of 3×3 is used for convolution to obtain a convolution feature map with a shape of 3×3. The size of the pooling layer is set to 2, and the sliding step is 2. This setting makes the maximum pooling downsampling rate 2. Therefore, in order to integrate the information extracted from different signal channels, the model first uses a stride of 1, 2 convolution kernels with a length of 3 and a width of 2×30 to merge the results of different inputs from the previous layer in the decision layer, and then uses the Softmax activation function to perform nonlinear transformation on the merged results. After that, the model uses two convolution kernels with the same kernel size as the output size of the previous step as the FC layer, and convolves with the output of the Softmax activation function. The model adopts the Dropout strategy in the FC layer, discarding 30% of the elements in the FC, that is, using Dropout with a probability of 0.3 to reduce the generalization error, and finally obtains the result vector;

[0084] S5 selects the preprocessed one-dimensional ECG and MCG signals as the input of the model. The loss function is defined according to the training objective of the dual-input convolutional recurrent neural network. When training the model, the cross entropy loss function is used to calculate the loss, and the Adam optimization algorithm is used to optimize the network parameters. The learning rate is set to 0.001, the batch size is 64, and the maximum number of epochs is set to 50. At the end of the training, the model with the best performance during the training process is selected and the model file is saved.

[0085] Furthermore, the training set and validation set are divided according to the five-fold cross-validation method. During the model training process, the early stopping strategy is used to monitor the validation loss of the model to prevent overfitting. If the validation loss does not improve within 5 consecutive cycles, the training will stop early. After the training is completed, the weight of the model is restored when the validation loss is the lowest to prevent performance degradation caused by overfitting.

[0086] S6 implements the classification task of the data to be tested.

[0087] Furthermore, the model is trained using the training set and gradient descent method to obtain the final classification model of the electrocardiogram and magnetic heart signals.

[0088] When selecting network hyperparameters according to the characteristics of cardiac electrophysiological signals in a dual-input convolutional recurrent network, the model is trained using ECG signals and MCG signals respectively, and appropriate model hyperparameters are selected through a large number of experiments and experience.

[0089] According to step S1, the electrocardiogram data is used to synchronize the magnetic heart signal data. Figure 2 ,ECG and MCG signals have similar waveforms in the time domain, so they have the advantage of collaborative analysis in cardiac electrophysiological signals.

[0090] The Pan-Tompkins algorithm is an adaptive QRS complex detection algorithm used to identify heartbeat signals. The algorithm process includes high-pass filtering, low-pass filtering, gradient filtering, squaring, moving average filtering, root mean square transformation, threshold detection and other steps. The preprocessing step filters the MCG signal, and the transfer function of the filter is as follows:

[0091]

[0092] After filtering, a derivative function with a transfer function is used to enhance the slope information of the QRS complex. The transfer function for a 2-point derivative filter with a gain of 5.0 and a processing delay of 1 sample is shown below:

[0093] H(z)=0.1(-z -2 -2z -1 +2z 1 +z 2 )

[0094] Threshold screening was used to extract the filtered peaks, and if no R value appeared within 1.5 times the average RR interval, a backtracking procedure was performed to lower the judgment criteria, and the peak was considered as an R wave if the criteria were met.

[0095] According to step S2, the electrocardiogram signals in the data set are ECG and MCG signals of labeled types.

[0096] Z-Score standardization is a method that divides the difference between the mean scores by the standard deviation. Its function is to scale the overall sample data according to a certain ratio so that it is normalized to a certain range, but it does not change the overall data distribution. In the process of collecting ECG and MCG signals, the signal amplitudes of different people are different. Therefore, Z-Score standardization is used to reduce the amplitude difference between ECG data and MCG data, while improving the model convergence speed. The calculation formula is as follows:

[0097]

[0098] Among them, xz represents the final output value obtained after the data is Z-Score standardized, μ represents the mean value of the overall sample data, and σ represents the standard deviation of the overall sample data.

[0099] Reference Figure 3The frequency range of the obtained ECG spectrum is 0-360Hz (the sampling frequency of the original ECG signal), and the frequency range of the MCG spectrum is 0-50Hz (the frequency of the MCG signal after filtering out the power frequency interference).

[0100] Reference Figure 4 After STFT, the intensity of the frequency components of the cardiac electrophysiological signal in each time period can be represented, and the logarithmic transformation can avoid the appearance of zero values ​​when the spectrum is visualized (due to the undefined zero of the logarithmic function), making the spectrum easier to process and observe.

[0101] Step S4 is a preferred implementation of this embodiment, constructing a fusion classification model, referring to Figure 5 . In the dual-input neural network, the convolutional neural network and the recurrent neural network are used to process the one-dimensional and two-dimensional data of the three inputs respectively, and then the outputs of the three models are merged together and input into the Softmax layer to obtain the final classification prediction probability. The fusion classification model includes a dual-input CNN feature extractor, a GRU-based feature learner and a feature classifier connected in sequence. The convolution module is divided into two branches, the first branch is used to identify the ECG signal, and the second branch is used to process the magnetic heart signal, and then the multi-domain features of the two signals are integrated for classification.

[0102] The convolution module mainly includes convolution layer, pooling layer, fully connected layer, etc. The convolution layer is mainly composed of convolution kernel, convolution layer parameters, batch normalization (BN) and activation function. The convolution kernel can be set to different sizes to obtain local receptive fields of different sizes. The convolution layer parameters include convolution kernel size, step size, etc. In the neural network, the role of the activation function is to introduce nonlinear factors. The pooling layer is usually placed immediately after the convolution layer. It is actually a downsampling that can continuously reduce the size of the data and reduce the number of parameters. The maximum pooling extracts important information of the input features by selecting the maximum value. The fully connected layer is mainly used to convert the feature map into the vector required for classification, realize end-to-end learning, and perform the final classification.

[0103] As a preferred implementation of this embodiment, the parameters of the dual-input CNN feature extractor are shown in the following table:

[0104] Table 1 Parameter configuration of convolutional layers in one-dimensional and two-dimensional CNN models

[0105]

[0106] Essentially, a convolutional layer performs a matrix multiplication between the layer's input and the selected filters by shifting the filters over the input data matrix. In addition to convolutional layers, the CNN architecture also includes pooling layers and fully connected layers, which are placed after each convolutional layer to reduce the dimensionality of the data. Fully connected layers are placed at the end of the network and are responsible for classifying the input data based on the data extracted from the sequence of convolutional and pooling layers.

[0107] Specifically, the specific formula for the convolution linear operation between two time series is as follows:

[0108]

[0109] Among them, x(t) and w(t) represent two time series.

[0110] In order to prevent gradient vanishing, unify the data distribution within a batch, and speed up model convergence, a batch normalization operation is added after convolution. The batch normalized data is passed through the ReLU activation function and then input into the GRU network. Figure 6 ,Except for the different input shape dimensions, the 1D CNN and 2D CNN residual blocks have similar ,structures, repeating the units consisting of Conv layer, batch normalization layer and ReLU activation function, which improves the ,ability of the network to learn deeper features and fully ,extract data features.

[0111] The temporal feature extraction module is a GRU network, and its input is the shallow features output by the convolution layer in the convolution extraction module in step S4. Figure 7 , the GRU unit consists of an update gate and a reset gate. First, the gated signal is reset by the reset gate and then concatenated with the input. The concatenated data is scaled to the range of -1 to 1 by the tanh activation function to obtain data containing the current input. In the update memory stage, forgetting and memorizing operations are performed simultaneously.

[0112] like Figure 8 As shown in the figure, we also introduced an attention mechanism based on the SE module. The attention mechanism enables the model to automatically learn important features in the signal and focus more on signal segments that are critical to the classification task. ECG signals and MCG signals are composed of a large number of time series data points, each of which represents the electrical activity of the heart. The attention mechanism helps the model focus on the decisive parts of ECG signals and MCG signals, so that it can better learn and utilize the key information in the signal and optimize the performance of the classification task.

[0113] Since the model proposed in the present invention is a dual-input network model including two parallel branches, in which there are three network branches to identify multi-domain features, the three network branches need to be trained separately during the training process, and the trained models are used as the basic model. In the process of training the three network branches separately, a consistent training strategy is adopted.

[0114] According to step S5, the processed training set data is sent to the network training. When the features of the feature extractor are sent to the learner, the gradient descent method is selected to update the weights, where the learning rate is fixed to 0.001 and the batch size is 64. The Softmax layer is used to calculate the loss using the cross entropy loss function, and the result vector output by the FC layer is converted into a decision probability. The cross entropy loss function calculation formula is as follows:

[0115]

[0116] Where N represents the total number of samples, K represents the number of label values, i.e. the number of categories, y represents the actual data label, and p represents the probability that a sample is predicted to be a certain category.

[0117] Softmax is a generalization of the binary Sigmoid algorithm for multi-classification problems. The Softmax activation function maps the input value to a continuous output between 0 and 1.

[0118] Dropout is a regularization strategy that can effectively prevent model overfitting. During the training process, Dropout will randomly shut down some neurons, which helps reduce the network's over-reliance on certain features, thereby improving the model's generalization ability.

[0119] Adjust the model's hyperparameters as needed, and then conduct a large number of experiments to find the best parameter combination to fine-tune the above settings. The specific steps include:

[0120] Use the grid search method to set the values ​​of multiple convolution kernel parameters, and evaluate the model performance of each configuration through cross-experiments; by observing the training and validation loss curves under different numbers of convolution kernels, determine whether the model is overfitting or underfitting, and further adjust the hyperparameters.

[0121] When the loss rate stops decreasing, save the trained model.

[0122] According to step S6, the test set is input for testing to achieve automatic classification of cardiac electrophysiological signals.

[0123] In order to evaluate the performance of the model, this embodiment uses a confusion matrix to count the category distribution. Accuracy (Acc), recall (R), precision (P) and F1Score are used to quantitatively evaluate the performance of the model. The calculation method is as follows:

[0124]

[0125]

[0126]

[0127]

[0128] Among them, TP is true positive, which means the number of samples correctly classified in the positive class; TN is true negative, which means the number of samples correctly classified in the negative class; FP is false positive, which means the number of samples incorrectly classified in the positive class; FN is false negative, which means the number of samples incorrectly classified in the negative class. Accuracy refers to the proportion of correctly classified positive samples and negative samples to the total number of samples. The higher the accuracy, the better the model performance. Recall is the ability of the model to successfully identify positive samples. The higher the recall, the lower the missed diagnosis rate. Precision refers to the proportion of the number of samples predicted correctly to the number of samples predicted as positive examples. The higher the precision, the lower the misdiagnosis rate. F1Score represents the comprehensive performance of the model. The larger the F1Score value, the better the comprehensive performance of the model.

[0129] Embodiment 3

[0130] The present invention also provides a cardiac electrophysiological signal classification system based on multi-scale fusion of electrocardiogram and magnetic heart, the system is used to implement the method described, comprising: a preprocessing module, a fusion module and a classification module;

[0131] The preprocessing module is used to obtain the original ECG signal and the original magnetic heart signal, preprocess the original ECG signal and the original magnetic heart signal respectively, divide the heartbeat according to the R wave positioning, extract the heartbeat signal, and obtain the corresponding frequency domain information after Fourier transformation of the extracted heartbeat signal, wherein the frequency domain information includes: one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features;

[0132] The fusion module is used to input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into the convolutional recurrent neural network for shallow feature extraction. Taking ECG and MCG signals as input, a convolutional recurrent neural network with two parallel branches is designed to realize the joint analysis of ECG and MCG signals.

[0133] The classification module is used to comprehensively classify and detect cardiac electrophysiological signals by fusing the characteristics of a single heartbeat cycle, i.e., heartbeat, in the time domain and frequency domain.

[0134] In this embodiment, the preprocessing module filters and denoises the original ECG and MCG signals, respectively, and then performs single heart beat interception, and then obtains the spectrum diagram and time-spectrum diagram through Fourier transform, and establishes ECG and MCG one-dimensional time domain signal data set, two-dimensional spectrum data set, and two-dimensional time-spectrum data set, respectively.

[0135] In this embodiment, the long signal interception method is to use the Pan-Tompkins detection algorithm to locate the R wave, and then use the R wave position as a reference point to extract the long signal. sampling points, take the sampling points, and a total of Fs sampling points are intercepted as a complete heart beat (where Fs is the sampling rate).

[0136] In this embodiment, the frequency spectra of ECG and MCG signals are obtained by fast Fourier transform (FFT), and their time-frequency spectra are obtained by short-time Fourier transform (STFT);

[0137] In this embodiment, the network adopts a three-module combination structure, in which the first module is a dual-input convolution module, which includes two parallel convolution branches, each branch has the same structure, including a convolution layer, a batch normalization layer, a ReLU activation function and a maximum pooling layer; the second module is a time series feature extraction module composed of a gated recurrent unit (GRU), including a reset gate and an update gate; the third module is a decision module, which includes a flat (Flatten) layer, a fully connected (FC) layer, and a dense (Dense) layer, and finally outputs a multi-classification probability distribution through a Softmax function to obtain a classification result.

[0138] In this embodiment, the dual-input convolution module adopts one-dimensional and two-dimensional CNN models as the network structure respectively, and two kinds of cardiac electrophysiological signals are used as the input of the model at the same time; the ECG and MCG signal heart beats or spectrum diagrams and time-spectrum diagrams are respectively input into three independent network branches. For the one-dimensional CNN model, a batch normalization layer is used after the first convolution layer to standardize the data of each batch and speed up the training speed; the maximum pooling layer placed after batch normalization is used to reduce the feature dimension; the second and third convolution layers increase the number of convolution kernels to extract more features, and the other parameters are the same as the first convolution layer; for the two-dimensional CNN model, a two-dimensional convolution kernel is used for feature extraction; when constructing the model, the above structure can be repeatedly superimposed, and through the multi-level convolution structure, the model can better understand and process complex input data.

[0139] In this embodiment, the time series feature extraction module adopts GRU as the network structure; its input is the shallow signal features extracted by the above convolution module, and its output is the extracted time series features.

[0140] In this embodiment, in the decision module, a convolution layer is first added to merge the results of different inputs from the previous layer, and then the merged results are nonlinearly transformed using the Softmax activation function. After that, the model uses a convolution kernel with the same output size as the previous step as the FC layer, convolves with the output of the Softmax activation function, and uses a dropout layer (Dropout) and early stopping strategy in the FC layer to prevent overfitting.

[0141] In this embodiment, before each signal is input into the network, the signal is normalized using Z-Score; and the data set is divided using a five-fold cross validation method.

[0142] In this embodiment, in order to improve the feature extraction capability of the model, the SE module is introduced. Through the dual-input CNN-GRU hybrid neural network structure based on the combined attention mechanism, the time-frequency domain features in the ECG and MCG signals can be fully integrated.

[0143] In this embodiment, network hyperparameters are selected in a dual-input convolutional recurrent network according to the characteristics of cardiac electrophysiological signals, and the parameters of the convolution kernel or the number of hidden units of the GRU network are changed through multiple experiments to improve the training speed, classification accuracy, stability, and generalization ability of the model.

[0144] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A cardiac electrophysiological signal classification method based on multi-scale fusion of ECG and magnetocardiography, characterized in that: The following steps are involved: Acquire the original ECG signal and the original MCG signal, pre-process the original ECG signal and the original MCG signal respectively, divide the heartbeat according to the R wave positioning, extract the heartbeat signal, and obtain the corresponding frequency domain information after Fourier transformation of the extracted heartbeat signal, wherein the frequency domain information includes: one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features; The obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features are input into the convolutional recurrent neural network for shallow feature extraction. Taking ECG and MCG signals as input, a convolutional recurrent neural network with two parallel branches is designed to realize the joint analysis of ECG and MCG signals. By integrating the characteristics of a single heartbeat cycle, i.e., heartbeat, in the time domain and frequency domain, the cardiac electrophysiological signals are comprehensively classified and detected.

2. The method according to claim 1, characterized in that: Obtain the original ECG signal, pre-process the original ECG signal, divide the heartbeat according to the R wave positioning, extract the heartbeat signal, and obtain the corresponding frequency domain information after Fourier transformation of the extracted heartbeat signal, including: Wavelet threshold denoising is used to preprocess the original ECG signal to remove the original noise of the data, filter out baseline drift and power frequency interference, and use multi-order Daubechies wavelets to perform wavelet decomposition on the input signal to obtain coefficients at different levels, where the coefficients include approximate coefficients and detail coefficients. The denoising threshold λ is calculated based on the detail coefficients, and the wavelet coefficients of the two highest-frequency high-frequency components are set to zero, and the coefficients less than the threshold are set to zero to remove noise. The processed coefficients are converted back to the time domain signal using inverse wavelet transform to obtain the entire denoised ECG signal and the preprocessed time series S1. With a heartbeat of fixed length as input, the data points with a length of Fs before and after the R wave in the time series data S1 are intercepted as a heartbeat cycle signal, thereby dividing the heartbeat; Perform FFT on each heartbeat cycle signal to obtain the spectrum of the heartbeat signal, perform STFT on each heartbeat cycle signal to obtain the time-frequency spectrum of the heartbeat, and then construct a one-dimensional ECG signal, a two-dimensional ECG spectrum, and a two-dimensional ECG time-frequency spectrum dataset; The calculation method of the threshold λ is: Where D is the processed wavelet coefficient, C is the signal length, and median means taking the median; Among them, the data points with a length of Fs before and after the R wave in the time series data S1 are intercepted as a heartbeat cycle signal including: The Pan-Tompkins detection algorithm is used to locate the R wave, and then the R wave position is used as a reference point to move forward. sampling points, take the Sampling points are taken, and a total of Fs sampling points are intercepted as the length of a complete heartbeat cycle, where Fs is the sampling rate.

3. The method according to claim 1, characterized in that: The original magnetic heart signal is obtained, the original magnetic heart signal is preprocessed, and the heartbeat is divided according to the R wave positioning, the heartbeat signal is extracted, and the corresponding frequency domain information is obtained after the extracted heartbeat signal is subjected to Fourier transformation, including: The original magnetic cardiogram signal was denoised and baseline drift was removed using a 1-40 Hz bandpass filter and a 300-window median filter. The R wave is enhanced using the Pan-Tompkins detection algorithm, and the position of the R wave is detected by calculating the amplitude, slope, and waveform characteristics of the signal. After determining the window size of the heartbeat, each heartbeat cycle signal is extracted, and the corresponding frequency domain information is obtained through Fourier transform. Then, a one-dimensional MCG signal, a two-dimensional MCG spectrum graph, and a two-dimensional MCG time-spectrum graph dataset are constructed.

4. The method according to claim 1, characterized in that The convolutional recurrent neural network adopts a three-module combination structure, in which the first module is a dual-input convolution module, which contains two parallel convolution branches. Each branch has the same structure, including a convolution layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer; the second module is a temporal feature extraction module composed of a gated recurrent unit GRU, including a reset gate and an update gate; the third module is a decision module, which contains a flat layer, a fully connected layer, and a dense layer. Finally, the Softmax function is used to output a multi-classification probability distribution to obtain the classification result.

5. The method according to claim 4, characterized in that The dual-input convolution module uses a one-dimensional and a two-dimensional CNN model as the network structure, and two cardiac electrophysiological signals are simultaneously used as the input of the model; the ECG and MCG signal heart beats or spectrograms and time-spectrum spectrograms are respectively input into three independent network branches. For the one-dimensional CNN model, a batch normalization layer is used after the first convolution layer to standardize the data of each batch; the maximum pooling layer placed after the batch normalization is used to reduce the feature dimension; The second and third convolutional layers increase the number of convolution kernels to extract more features, and the other parameters are the same as the first convolutional layer. For the two-dimensional CNN model, two-dimensional convolution kernels are used for feature extraction.

6. The method according to claim 4, characterized in that The temporal feature extraction module adopts the gated recurrent unit GRU as the network structure; the input is the shallow signal feature extracted by the dual-input convolution module, and the output is the extracted temporal feature.

7. The method according to claim 4, characterized in that In the decision module, a convolutional layer is first added to merge the results of different inputs from the previous layer, and then the merged results are nonlinearly transformed using the Softmax activation function; Afterwards, the model uses a convolution kernel with the same output size as the previous step as the fully connected layer, convolves it with the output of the Softmax activation function, and uses a dropout layer and early stopping strategy in the fully connected layer to prevent overfitting.

8. A cardiac electrophysiological signal classification system based on multi-scale fusion of ECG and MCG, the system being used to implement the method of any one of claims 1 to 7, characterized in that: include: Preprocessing module, fusion module and classification module; The preprocessing module is used to obtain the original ECG signal and the original magnetic heart signal, preprocess the original ECG signal and the original magnetic heart signal respectively, divide the heartbeat according to the R wave positioning, extract the heartbeat signal, and obtain the corresponding frequency domain information after Fourier transformation of the extracted heartbeat signal, wherein the frequency domain information includes: one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features; The fusion module is used to input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into the convolutional recurrent neural network for shallow feature extraction, and the convolutional recurrent neural network with two parallel branches is designed to realize the joint analysis of the ECG signal and the MCG signal with ECG and MCG signals as input respectively; The classification module is used to comprehensively classify and detect cardiac electrophysiological signals by fusing the characteristics of a single heartbeat cycle, ie, heartbeat, in the time domain and the frequency domain.

Images