A cardiac electrophysiological signal classification method and system based on electrocardio and magnetocardiography multi-scale fusion

Abstract

The application discloses a kind of based on electrocardiogram and cardiac magnetic multi-scale fusion cardiac electrophysiological signal classification method and system, comprising: obtaining electrocardiogram signal and cardiac magnetic signal, and the original ECG, MCG signal is respectively filtered and denoised pretreatment, and through R wave positioning division heartbeat, after Fourier transform, obtain two-dimensional spectrogram, after feature extraction, respectively establish multi-domain cardiac function signal dataset;Convolutional recurrent neural network is constructed, and the synchronous processing of two different forms of cardiac electrophysiological signals is realized by double input model, shallow signal feature extracted by convolutional neural network is input into the time sequence feature extraction module formed by recurrent neural network, fusion ECG, MCG signal local saliency feature and time sequence feature, form the final ECG, MCG signal feature used for classification.Combined with cross-entropy loss function, the feature after fusion is input into nonlinear classification layer, using Softmax carries out model training, and outputs classification result.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of signal processing and the technical field of joint analysis of electrocardiogram signals and magnetocardiogram signals, and particularly relates to a cardiac electrophysiological signal classification method and system based on electrocardiogram and magnetocardiogram multi-scale fusion. BACKGROUND

[0002] The electrocardiogram signal and the magnetocardiogram signal are homologous signals generated by the ion transmembrane movement of myocardial cells. Compared with the polarization voltage generated by the tissue conduction attenuation and skin contact of the electrocardiogram signal, the magnetocardiogram signal is derived from the measurement of the magnetic field around the heart and is not affected by tissues, blood and the like in the conduction process, can enrich the representation of cardiac electrophysiological information, provide complementary information to the electrocardiogram, and has higher sensitivity and specificity. The physiological mechanisms of different physiological signals are different, so joint analysis of different physiological signals can provide multi-angle information for identifying different types of cardiac electrophysiological signals, and is more accurate than the single signal judgment method.

[0003] Compared with the traditional manual feature extraction machine learning classification method, the neural network can automatically mine the deep features in the data from the original time sequence, and improve the performance of the model by sharing the bottom features. Most of the existing models are based on the recognition of a single electrocardiogram signal. Considering the difference between the electrocardiogram signal and the magnetocardiogram signal, a cardiac electrophysiological signal recognition method based on a double-input convolutional recurrent neural network model is proposed, and a reasonable structure is designed to extract the significant features and time sequence features of the ECG signal and the MCG signal, so as to improve the performance of the multi-input classification model. The research on applying deep learning technology in the classification task of fusing two kinds of physiological signals usually refers to the network structure in the research task of electrocardiogram multi-classification. The ECG and MCG signals are both time sequence signals, and the local features of the heartbeat waveform are generally recognized by a deep convolutional neural network (CNN), and the time sequence features are extracted by a recurrent neural network (RNN). These two types of network structures are usually combined using CNN and LSTM. LSTM is a variant structure of RNN proposed to solve the problems of gradient disappearance and gradient explosion, and GRU is simpler in structure and has fewer parameters than LSTM, but in practice, the performance of GRU is not significantly different from that of LSTM, and even can be better in some tasks, so it is also a popular variant structure of RNN at present. SUMMARY

[0004] In order to solve the problems existing in the prior art, the present application provides a cardiac electrophysiological signal classification method and system based on electrocardiogram and magnetocardiogram multi-scale fusion, which aims to combine the advantages of different signals by jointly using multiple signals and multi-domain features, and to realize more accurate cardiac electrophysiological signal classification.

[0005] To achieve the above object, the present application provides the following scheme:

[0006] The application provides a cardiac electrophysiological signal classification method based on electrocardio and magnetocardiogram multi-scale fusion, comprising the following steps:

[0007] Obtain original electrocardio signals and original magnetocardiogram signals, respectively pretreat the original electrocardio signals and the original magnetocardiogram signals, and divide heartbeats according to R-wave positioning, extract heartbeat signals, and obtain corresponding frequency domain information after Fourier transform of the extracted heartbeat signals, wherein the frequency domain information comprises one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features;

[0008] Input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into a convolution recurrent neural network for shallow feature extraction, design a convolution recurrent neural network with two parallel branches to realize joint analysis of electrocardio signals and magnetocardiogram signals by taking ECG and MCG signals as inputs respectively;

[0009] The cardiac electrophysiological signal is comprehensively classified and detected by fusing the features of a single heartbeat cycle, i.e., a heartbeat, in the time domain and the frequency domain.

[0010] Preferably, the original electrocardio signals are obtained, pretreated, and divided into heartbeats according to R-wave positioning, the heartbeat signals are extracted, and the corresponding frequency domain information is obtained after Fourier transform of the extracted heartbeat signals, which comprises:

[0011] Wavelet threshold denoising is used to pretreat the original electrocardio signals to remove the original noise of the data, filter out baseline drift and power frequency interference, and use multi-order Daubechies wavelet to decompose the input signal to obtain coefficients at different levels, wherein the coefficients include approximation coefficients and detail coefficients, a denoising threshold λ is calculated according to the detail coefficients, the wavelet coefficients of the two highest frequency components are set to zero, the coefficients less than the threshold are set to zero to remove noise, and the processed coefficients are converted back to time domain signals using wavelet inverse transform to obtain the denoised whole segment electrocardio signals, and the pretreated time sequence S1 is obtained.

[0012] A heartbeat with a fixed length is taken as input, and data points with a length of Fs before and after the R wave in the time sequence data S1 are intercepted as a heartbeat cycle signal, so that the heartbeats are divided.

[0013] Perform FFT on each heartbeat cycle signal to obtain the frequency spectrum of the heartbeat signal, perform STFT on each heartbeat cycle signal to obtain the time-frequency spectrum of the heartbeat, and then construct one-dimensional ECG signal, two-dimensional ECG frequency spectrum and two-dimensional ECG time-frequency spectrum data sets.

[0014] The threshold λ is calculated as follows:

[0015]

[0016] Wherein, D is the wavelet coefficient after processing, C is the signal length, median represents the median;

[0017] Wherein, the data points with a length of Fs before and after the R wave in the intercepted time series data S1 are a heartbeat cycle signal, including:

[0018] The R wave is located using the Pan-Tompkins detection algorithm, and then the R wave position is taken as a reference point, Fs sampling points are taken forward Fs sampling points are taken backward A total of Fs sampling points are intercepted as the length of a complete heartbeat cycle, wherein Fs is the sampling rate.

[0019] Preferably, the original MCG signal is obtained, the original MCG signal is preprocessed, and the heartbeats are divided according to the R wave positioning, the heartbeat signal is extracted, and the corresponding frequency domain information is obtained after Fourier transform of the extracted heartbeat signal, including:

[0020] The original MCG signal is denoised by using a band-pass filter with a frequency of 1-40Hz and a 300-window median filter, and the baseline drift is removed;

[0021] The R wave is enhanced using the Pan-Tompkins detection algorithm, and then the position of the R wave is detected by calculating the amplitude, slope and waveform characteristics of the signal;

[0022] After determining the window size of the heartbeats, each heartbeat cycle signal is extracted, the corresponding frequency domain information is obtained by Fourier transform, and then a one-dimensional MCG signal, a two-dimensional MCG spectrum diagram and a two-dimensional MCG time-frequency spectrum diagram data set are constructed.

[0023] Preferably, the convolutional recurrent neural network adopts a three-module combined structure, wherein the first module is a double-input convolutional module, which includes two parallel convolutional branches, each branch has the same structure, including a convolutional layer, a batch normalization layer, a ReLU activation function and a max-pooling layer; the second module is a time sequence feature extraction module composed of a gated recurrent unit GRU, including a reset gate and an update gate; the third module is a decision module, including a flattening layer, a fully connected layer, a dense layer, and finally outputting a multi-class probability distribution through a Softmax function to obtain a classification result.

[0024] Preferably, the double-input convolution module respectively adopts a one-dimensional and a two-dimensional CNN model as a network structure, and two kinds of cardiac electrophysiological signals are simultaneously used as inputs of the model; the ECG and MCG signal heartbeat or spectrogram and time-frequency spectrogram are respectively input into three independent network branches, for the one-dimensional CNN model, a batch normalization layer is used after the first convolution layer to standardize the data of each batch; a max-pooling layer placed after the batch normalization is used to reduce the feature dimension; the number of convolution kernels of the second and third convolution layers is increased respectively to extract more features, and other parameters are the same as those of the first convolution layer; for the two-dimensional CNN model, a two-dimensional convolution kernel is used for feature extraction.

[0025] Preferably, the time sequence feature extraction module adopts a gated recurrent unit (GRU) as a network structure; the input is the shallow signal features extracted by the double-input convolution module, and the output is the extracted time sequence features.

[0026] Preferably, in the decision module, a convolution layer is first added to combine the results of different inputs from the previous layer, and then a Softmax activation function is used for nonlinear transformation on the combined results; thereafter, the model uses a convolution kernel with the same output size as the previous step as a fully connected layer, convolves with the output of the Softmax activation function, and uses a dropout layer and an early stopping strategy in the fully connected layer to prevent overfitting.

[0027] The application also provides a cardiac electrophysiological signal classification system based on electrocardiogram and magnetocardiogram multi-scale fusion, which is used to realize the method and comprises a preprocessing module, a fusion module and a classification module.

[0028] The preprocessing module is used to acquire original electrocardiogram signals and original magnetocardiogram signals, pre-process the original electrocardiogram signals and original magnetocardiogram signals respectively, divide heartbeats according to R-wave positioning, extract heartbeat signals, and obtain corresponding frequency domain information after Fourier transform of the extracted heartbeat signals, wherein the frequency domain information comprises one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features.

[0029] The fusion module is used to input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into a convolution recurrent neural network for shallow feature extraction, design a convolution recurrent neural network with two parallel branches to realize joint analysis of electrocardiogram signals and magnetocardiogram signals by taking the ECG and MCG signals as inputs respectively.

[0030] The classification module is used to comprehensively classify and detect cardiac electrophysiological signals by fusing the features of a single heartbeat cycle, i.e., a heartbeat, in the time domain and the frequency domain.

[0031] Compared with the prior art, the beneficial effects of the present application are: (1) The present application fuses the time domain and frequency domain information of electrocardiogram signals and magnetocardiogram signals, and analyzes the time-frequency domain information, which can help better understand and process complex signals of non-steady-state cardiac electrophysiological signals; and the features of the input signals are automatically identified by using a deep neural network model, thereby improving the accuracy of the classification model. The joint analysis of multiple signals and multiple domain features can provide higher accuracy for performing a classification task.

[0032] (2) The present application realizes a double-input model through two parallel convolutional neural networks, and expands the network width through a multi-branch network structure. Two parallel branches are designed to extract the features of one-dimensional ECG signals and MCG signals, two-dimensional ECG frequency spectrum and MCG frequency spectrum, and two-dimensional ECG time-frequency spectrum and MCG time-frequency spectrum, and to effectively fuse them. One of the branches inputs one-dimensional data, which is used to extract the time domain features in ECG and MCG signals; the other two branches input two-dimensional images, which are used to extract the frequency domain information in ECG and MCG signals; the output results of the three branches are spliced, and after channel dimension reduction, two forms of information fusion representation are obtained. Through the fusion of ECG and MCG signals and the combination of the attention mechanism, the utilization ability of ECG and MCG signals in space, time and frequency domain is improved. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the present application, the following briefly introduces the drawings needed in the embodiments. Obviously, the drawings described in the following are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0034] Figure 1 It is a flow chart of a cardiac electrophysiological signal classification method based on electrocardiogram and magnetocardiogram multi-scale fusion;

[0035] Figure 2 It is a time domain graph of a preprocessed signal in a cardiac electrophysiological signal classification method based on electrocardiogram and magnetocardiogram multi-scale fusion, wherein (A) is an ECG signal and (B) is an MCG signal;

[0036] Figure 3 It is a frequency spectrum graph of a preprocessed signal in a cardiac electrophysiological signal classification method based on electrocardiogram and magnetocardiogram multi-scale fusion, wherein (A) is an ECG signal and (B) is an MCG signal;

[0037] Figure 4 It is a cardiac cycle time-frequency spectrum graph in a cardiac electrophysiological signal classification method based on electrocardiogram and magnetocardiogram multi-scale fusion, wherein (A) is an ECG signal and (B) is an MCG signal;

[0038] Figure 5It is a kind of whole structure schematic diagram of cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetocardiogram;

[0039] Figure 6 It is a kind of structure schematic diagram of residual block in cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetocardiogram, wherein (A) is a one-dimensional CNN residual block, (B) is a two-dimensional CNN residual block;

[0040] Figure 7 It is a kind of structure schematic diagram of gated recurrent unit in cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetocardiogram;

[0041] Figure 8 It is a kind of structure schematic diagram of SE module in cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetocardiogram. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0043] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0044] Embodiment one

[0045] The present application provides a kind of cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetocardiogram, comprising the following steps:

[0046] (1) the original electrocardiogram signal is preprocessed, and Pan-Tompkins detection algorithm is used to locate R wave, and the single heartbeat cycle is extracted by the R wave position of each heartbeat cycle, and the extracted heartbeat signal is obtained after Fourier transform spectrum diagram, time-frequency spectrum diagram, one-dimensional ECG time domain signal, two-dimensional ECG spectrum diagram, two-dimensional ECG time-frequency spectrum diagram data set are established respectively;

[0047] (2) the original magnetocardiogram signal is preprocessed, and Pan-Tompkins detection algorithm is used to locate R wave, and after dividing heartbeat, frequency domain information is also obtained after the above Fourier transform, one-dimensional MCG time domain signal, two-dimensional MCG spectrum diagram, two-dimensional MCG time-frequency spectrum diagram data set are established respectively;

[0048] (3) Considering the difference in time scale between ECG signals and MCG signals, the present application uses a double-input convolution module to extract the features implied by different signals, which is realized by using different convolution modules for different inputs. A first network branch is constructed for the above-mentioned one-dimensional time series data, the branch model comprising two parallel one-dimensional convolution modules as feature extraction modules, and a GRU network as a time sequence feature learning module; a second network branch is constructed for a two-dimensional frequency spectrum, the branch model comprising two parallel two-dimensional convolution modules and a fully connected layer; a third network branch is constructed for a two-dimensional time-frequency spectrum, the branch model comprising two parallel two-dimensional convolution modules and a fully connected layer, and finally the three branch networks are connected, a Softmax is used in combination with a cross-entropy loss function, and a gradient descent method is used to train the model, after the training is completed, the test set is used for testing, and the classification result is output.

[0049] Further, the step (1) comprises:

[0050] (11) The original ECG signal is preprocessed by using wavelet threshold denoising to remove the original noise of the data, filter out the baseline drift and power frequency interference, use multi-order Daubechies wavelet to decompose the input signal to obtain different levels of coefficients (including approximate coefficients and detail coefficients), calculate the denoising threshold λ according to the detail coefficients, set the wavelet coefficients of the two highest frequency components to zero, set the coefficients less than the threshold to zero to remove the noise, use wavelet inverse transform to convert the processed coefficients back to time domain signals to obtain the denoised whole ECG signal, and obtain the preprocessed time series S1;

[0051] (12) In the model, a fixed length heartbeat is needed as input, so the data points with a length of Fs before and after the R wave in the time series data S1 are intercepted as a heartbeat cycle signal, so as to divide the heartbeats.

[0052] (13) Perform FFT on each heartbeat cycle signal to obtain the frequency spectrum of the heartbeat signal, perform STFT to obtain the time-frequency spectrum of the heartbeat, and then construct one-dimensional ECG signal, two-dimensional ECG frequency spectrum and two-dimensional ECG time-frequency spectrum data sets for subsequent model training and analysis.

[0053] Further, in the step (11), the threshold λ is calculated as follows:

[0054]

[0055] Where D is the processed wavelet coefficient, C is the signal length, and median represents the median.

[0056] Further, the step (12) specifically comprises the steps of:

[0057] According to the empirical value, take points forward and points backward, which is exactly the length of a heart beat cycle.

[0058] Further, the step (13) is calculated as follows:

[0059] The function is called to perform FFT transform on the input signal, converting the time domain signal into the frequency domain signal, containing the amplitude information of each frequency; calculate the frequency range, and the spectrum of the real signal is symmetrical, so only take the positive frequency part. By drawing the spectrum diagram, the intensity of different frequency components in the cardiac electrophysiological signal can be observed.

[0060] The calculation method of STFT is as follows:

[0061] STFT usually relies on FFT algorithm, which divides the entire signal into several segments, and then performs Fourier transform on each segment to extract the frequency information of each segment; in STFT, the selection of window will affect the trade-off between time resolution and frequency resolution, the length of each window is 64 sampling points, that is, 64 consecutive sampling points are used for Fourier transform to calculate the frequency spectrum of this segment; the overlapping part between windows is 32 sampling points, which can provide higher time resolution and reduce the distortion of signal on the time axis; through logarithmic transformation, it can avoid zero value when visualizing the time-frequency spectrum. The short time of STFT makes it suitable for analyzing non-stationary signals such as abnormal cardiac electrophysiological signals.

[0062] Further, the step (2) includes:

[0063] (21) The original MCG signal is denoised by using band-pass filter with frequency of 1-40 Hz and 300 window median filter, and the baseline drift is removed;

[0064] (22) Pan-Tompkins detection algorithm mainly includes R wave enhancement after pre-processing, such as band-pass filter and difference operation, then the position of R wave is detected by calculating the amplitude, slope and waveform characteristics of the signal;

[0065] (23) After determining the window size of heart beat, extract the signal of each heart beat cycle, obtain its frequency domain information by Fourier transform, and then construct one-dimensional MCG signal, two-dimensional MCG spectrum and two-dimensional MCG time-frequency spectrum data set.

[0066] Further, the step (3) includes:

[0067] (31) three network branches with double-input convolution modules are established, wherein the double-input convolution modules are composed of two parallel convolution branches; for a one-dimensional CNN model, 16 convolution kernels with a scale of 5 are used in the first layer of convolution layer, and the sliding step is 1; a batch normalization layer is used to standardize the data of each batch and accelerate the training speed; a maximum pooling layer is used to reduce the feature dimension, and the pooling window size is 2 and the sliding step is 2; the number of convolution kernels is increased to 32 and 64 in the second and third layers of convolution layer respectively to extract more features, and other parameters are the same as those of the first convolution layer; for a two-dimensional CNN model, 32 convolution kernels with a size of 3*3 are used in the first layer of convolution layer, and the sliding step is 1; 64 convolution kernels with a size of 3*3 are used in the second and third layers of convolution layer, and the sliding step is 2;

[0068] (32) the preprocessed one-dimensional ECG signal and MCG signal data sets are respectively input into the first and second branch convolution layers; the two-dimensional ECG spectrum and MCG spectrum are input into the first and second branch convolution layers of the two-dimensional convolution module; the two-dimensional ECG time-frequency spectrum and MCG time-frequency spectrum are input into the first and second branch convolution layers of another two-dimensional convolution module

[0069] (33) a GRU time sequence feature extraction module is established, and an attention mechanism based on an SE module is introduced.

[0070] (34) at the end of the network, the results output by the one-dimensional and two-dimensional network branches are merged through a fully connected layer, the output values are converted into prediction probabilities by using a Softmax layer, and finally the classification results are output.

[0071] Each network branch convolution module includes two parallel one-dimensional convolution layers with a kernel size of 5, a BatchNorm layer and a ReLu activation function; the GRU network includes 36 hidden units.

[0072] Specifically, the feature vectors are processed by a maximum pooling layer, and the step of the maximum pooling layer is preferably 2; the GRU network can extract long-term dependence features when processing time sequence signals, the maximum pooling layer is connected to the GRU network, and a BatchNorm layer is used for normalization processing.

[0073] Further, the one-dimensional ECG and MCG signals are respectively input into one-dimensional convolution layers to preliminarily learn the time domain features, and output feature maps, then the feature maps of the two convolution layers are connected as the input of the one-dimensional CNN model; the two-dimensional ECG, MCG frequency domain and time-frequency domain signals are respectively input into two-dimensional convolution layers to preliminarily learn the frequency domain and time-frequency domain features, and output two-dimensional feature maps, then the feature maps of the two convolution layers are respectively connected as the input of the two-dimensional CNN model, so that the mixed features reflecting the characteristics of ECG and MCG can be obtained.

[0074] In order to enhance the feature extraction capability of the model, the SE module is applied to the GRU network capable of capturing future past information, the SE module structure is relatively simple, can maintain the original network structure and performance, and can also increase the performance of the network model by introducing a small number of parameters.

[0075] Further, the output vectors of the convolution module and the GRU are directly spliced to obtain the fusion features.

[0076] Further, the fusion features are input into the full connection layer for integration, and finally a Softmax activation function is used to calculate the output classification result.

[0077] Further, by setting the epoch, batchsize, loss function, optimizer, learning rate and other parameters, the training set data is continuously iterated and calculated, and when the training is completed, the model with the best performance in the training process is selected and the model file is saved.

[0078] Embodiment two

[0079] Reference Figure 1 The application provides a cardiac electrophysiological signal classification method based on electrocardio and magneto-cardiogram multi-scale fusion, which comprises the following steps:

[0080] S1 obtains original one-dimensional electrocardio signals and magneto-cardiogram signals, performs pretreatment and divides heartbeats;

[0081] S2 performs Z-Score standardization on the segmented heartbeats in step S1, and constructs one-dimensional ECG and MCG signal data sets by using the processed heartbeats;

[0082] S3 performs Fourier transform on the heartbeat signals in step S2 to obtain two-dimensional ECG and MCG frequency spectrum and two-dimensional ECG and MCG time-frequency spectrum data sets;

[0083] S4, the classification model is designed according to the time scale of the technical scheme center electrical signal and the magnetocardiogram signal as a reference. In each layer, grouping convolution is adopted for input; a fusion classification model including a feature extractor containing a double-channel CNN, a feature learner based on GRU and a feature classifier is constructed; a first layer convolution block uses 16 convolution kernels for each signal, the length of the convolution kernel is set to 3, the sliding step is 1, and zero padding is not adopted during convolution. For two-dimensional features, a two-dimensional convolution kernel with a shape of 3*3 is used for convolution to obtain a convolution feature map with a shape of 3*3. The size of the pooling layer is set to 2, and the sliding step is 2, which makes the maximum pooling down-sampling rate 2. Therefore, in order to integrate the information extracted from different signal channels, the model first uses a convolution kernel with a step of 1, a length of 2 and a width of 2*30 to combine the results of different inputs from the previous layer in the decision layer, and then uses a Softmax activation function to perform nonlinear transformation on the combined results. Then, the model uses two convolution kernels with the same size as the output size of the previous step as the FC layer, and convolves the output of the Softmax activation function, and the model uses a Dropout strategy in the FC layer, which discards 30% of the elements in the FC, that is, uses a Dropout with a probability of 0.3 to reduce the generalization error, and finally obtains a result vector;

[0084] S5, the one-dimensional ECG and MCG signals after preprocessing are selected as the input of the model. A loss function is defined according to the training target of the double-input convolution recurrent neural network, and during model training, the cross-entropy loss function is used to calculate the loss, and the Adam optimization algorithm is used to optimize the network parameters. The learning rate is set to 0.001, the batch size is 64, and the maximum number of epochs is set to 50. When the training is completed, the model with the best performance in the training process is selected, and the model file is saved.

[0085] Further, the training set and the validation set are divided according to the five-fold cross-validation method. During the model training process, an early stopping strategy is adopted to monitor the validation loss of the model to prevent overfitting. If the validation loss does not improve for 5 consecutive periods, the training will be stopped in advance. After the training is completed, the weights of the model at the time when the validation loss is the lowest are restored to prevent performance degradation caused by overfitting.

[0086] S6, the classification task of the to-be-tested data is implemented.

[0087] Further, the model is trained using the training set and the gradient descent method to obtain the final electrocardiosignal and magnetocardiogram classification model.

[0088] In the double-input convolution recurrent network, the model is trained using the ECG signal and the MCG signal respectively, and appropriate model hyperparameters are selected through a large number of experiments and experience.

[0089] According to step S1, the magnetocardiogram signal data is synchronized with the electrocardiogram data. Referring to Figure 2 , the ECG and MCG signals have similar waveforms in the time domain, and therefore have the advantage of synergistic analysis in cardiac electrophysiological signals.

[0090] The Pan-Tompkins algorithm is a QRS complex detection algorithm for identifying heart signals, which has adaptability. The flow of the algorithm includes steps such as high-pass filtering, low-pass filtering, gradient filtering, squaring, moving average filtering, square root transformation, threshold detection, etc. The pre-processing step filters the MCG signal, and the transfer function of the filter is as follows:

[0091]

[0092] After filtering, a differential function with a transfer function is used to enhance the slope information of the QRS wave. The transfer function of a 2-point derivative filter with a gain of 5.0 and a processing delay of 1 sample is as follows:

[0093] H(z) = 0.1(-z -2 -2z -1 +2z 1 +z 2 )

[0094] Threshold screening is used to extract the filtered peaks, and if there is no R value within 1.5 times the average RR interval, a backtracking procedure is performed to lower the judgment standard, and if the standard is met, the peak is considered as an R wave.

[0095] According to step S2, the electrocardiogram signal of the data set is an annotated type of ECG and MCG signal.

[0096] Z-Score standardization is a method of removing the difference from the score average by the standard deviation, which can scale the overall sample data according to a certain proportion, so that it is standardized to a certain interval range, but does not change the overall data distribution. In the process of collecting electrocardiogram signals and magnetocardiogram signals, the signal amplitudes of different people are different, so Z-Score standardization is used to reduce the amplitude difference between electrocardiogram data and magnetocardiogram data, and to improve the model convergence speed. The calculation formula is as follows:

[0097]

[0098] where xz represents the final output value obtained after Z-Score standardization of the data, μ represents the average value of the overall sample data, and σ represents the standard deviation of the overall sample data.

[0099] Referring to Figure 3, the frequency range of the obtained ECG spectrum is 0-360Hz (the sampling frequency of the original ECG signal), and the frequency range of the MCG spectrum is 0-50Hz (the frequency of the MCG signal after filtering out power frequency interference).

[0100] With reference to Figure 4 , after STFT, the intensity of the frequency component of the cardiac electrophysiological signal in each time period can be represented, and through logarithmic transformation, the zero value in the spectrum visualization is avoided (since the logarithmic function is not defined for zero), so that the spectrum is easier to process and observe.

[0101] Step S4, as a preferred embodiment of the present embodiment, a fusion classification model is constructed, with reference to Figure 5 In the dual-input neural network, the one-dimensional and two-dimensional data of the three inputs are processed by using a convolutional neural network and a recurrent neural network respectively, and then the outputs of the three models are combined together and input into a Softmax layer to obtain the final classification prediction probability. The fusion classification model includes a dual-input CNN feature extractor, a GRU-based feature learner and a feature classifier connected in sequence. The convolutional module is divided into two branches, the first branch is used to identify the electrocardiogram signal, and the second branch is used to process the magnetocardiogram signal, and then the multi-domain features of the two signals are integrated for classification.

[0102] The convolutional module mainly includes convolutional layers, pooling layers, fully connected layers and the like. The convolutional layer is mainly composed of a convolution kernel, parameters of the convolutional layer, batch normalization (BN) and an activation function. The convolution kernel is set to different sizes to obtain local receptive fields of different sizes, and the convolutional layer parameters include convolution kernel size, step size and the like. In the neural network, the activation function is used to introduce a nonlinear factor, and the pooling layer is usually arranged immediately after the convolutional layer. In fact, it is a kind of down-sampling, which can continuously reduce the size of the data and reduce the parameter amount. The maximum pooling extracts important information of the input features by selecting the maximum value. The fully connected layer is mainly used to convert the feature mapping into a vector required for classification, realizes end-to-end learning, and performs the final classification.

[0103] As a preferred embodiment of the present embodiment, the parameters of the dual-input CNN feature extractor are shown in the following table:

[0104] Table 1 Parameter configuration of the convolutional layer in the one-dimensional and two-dimensional CNN models

[0105]

[0106] In essence, a convolutional layer performs a matrix multiplication between the input of the layer and a selected filter by shifting the filter over the input data matrix. In addition to the convolutional layers, the CNN architecture also includes pooling layers, which are placed after each convolutional layer to reduce the dimensionality of the data, and fully connected layers, which are placed at the end of the network and are responsible for classifying the input data based on the data extracted from the sequence of convolutional and pooling layers.

[0107] Specifically, the convolution linear operation between two time series is specifically formulated as follows:

[0108]

[0109] where x(t) and w(t) represent two time series.

[0110] To prevent gradient vanishing, unify the distribution of data within the batch, and accelerate model convergence, a batch normalization operation is added after the convolution. The data that has undergone batch normalization is input into the GRU network after passing through the ReLU activation function. Referring to Figure 6 The one-dimensional CNN and the two-dimensional CNN residual block have similar structures except for the different input shape dimensions. The unit composed of the Conv layer, the batch normalization layer, and the ReLU activation function is repeated to improve the network's ability to learn deeper features and fully extract data features.

[0111] The time series feature extraction module is a GRU network, and the input of the GRU network is the shallow features output by the convolutional layer in the convolutional extraction module in step S4. Referring to Figure 7 The GRU unit is composed of an update gate and a reset gate. First, the gating signal is reset through the reset gate, and then it is spliced with the input. The spliced data is scaled to the range of -1 to 1 by the tanh activation function to obtain data containing the current input. In the update memory phase, the forgetting operation and the memory operation are performed simultaneously.

[0112] As shown in Figure 8 we introduce an attention mechanism based on the SE module. The attention mechanism enables the model to automatically learn important features in the signal and focus more attention on signal segments that are crucial to the classification task. The electrocardiogram and magnetocardiogram are composed of a large number of time series data points, each representing the electrical activity of the heart. The attention mechanism helps the model focus on the parts of the electrocardiogram and magnetocardiogram that play a decisive role, so that it can better learn and utilize key information in the signal and optimize the performance of the classification task.

[0113] Since the model proposed in the present application is a double-input network model containing two parallel branches, and there are three network branches to identify multi-domain features, it is necessary to train the three network branches separately during the training process, and the trained model is used as the base model. During the training of the three network branches respectively, a consistent training strategy is adopted.

[0114] According to step S5, the processed training set data is sent to network training. The features of the feature extractor are sent to the learner process to update the weights using the gradient descent method, where the learning rate is fixed at 0.001 and the batch size is 64. The Softmax layer is used to calculate the loss using the cross-entropy loss function, which converts the result vector output by the FC layer into a decision probability. The cross-entropy loss function calculation formula is as follows:

[0115]

[0116] Where N represents the total number of samples, K represents the number of label values, i.e., the number of categories, y represents the true data label, and p represents the probability of a sample being predicted as a certain category.

[0117] Softmax is an extension of binary Sigmoid to multi-classification problems. The Softmax activation function maps the input value to a continuous output between 0 and 1.

[0118] Dropout is a regularization strategy that can effectively prevent model overfitting. During training, Dropout randomly turns off some neurons, which helps to reduce the network's over-reliance on certain features, thereby improving the model's generalization ability.

[0119] Adjust the hyperparameters of the model as needed, and then conduct a large number of experiments to find the optimal parameter combination to fine-tune the above settings. The specific steps include the following:

[0120] Use the grid search method, i.e., set multiple values of the convolution kernel parameters, evaluate the performance of each configuration of the model through cross-experiments; by observing the training and validation loss curves under different numbers of convolution kernels, determine whether the model is overfitting or underfitting, and further adjust the hyperparameters.

[0121] When the loss rate no longer decreases, save the trained model.

[0122] According to step S6, input the test set for testing to realize automatic classification of cardiac electrophysiological signals.

[0123] To evaluate the performance of the model, the present embodiment uses a confusion matrix to count the class distribution. The accuracy (Acc), recall (R), precision (P), and F1Score are selected to quantitatively evaluate the performance of the model. The calculation method is as follows:

[0124]

[0125]

[0126]

[0127]

[0128] wherein TP is true positive, representing the number of correctly classified samples in the positive class; TN is true negative, representing the number of correctly classified samples in the negative class; FP is false positive, representing the number of incorrectly classified samples in the positive class; FN is false negative, representing the number of incorrectly classified samples in the negative class. The accuracy represents the proportion of correctly classified positive samples and negative samples to the total number of samples, and the higher the accuracy, the better the model performance. The recall rate is the ability of the model to successfully identify positive samples, and the higher the recall rate, the lower the missed diagnosis rate. The precision rate is the proportion of correctly predicted samples in the predicted positive samples, and the higher the precision rate, the lower the misdiagnosis rate. The F1Score represents the comprehensive performance of the model, and the larger the F1Score value, the better the comprehensive performance of the model.

[0129] Example Three

[0130] The application also provides a cardiac electrophysiological signal classification system based on electrocardio and magnetocardiogram multi-scale fusion, which is used to realize the method and comprises a preprocessing module, a fusion module and a classification module.

[0131] The preprocessing module is used to obtain original electrocardio signals and original magnetocardiogram signals, to preprocess the original electrocardio signals and the original magnetocardiogram signals respectively, to divide heartbeats according to R-wave positioning, to extract heartbeat signals, and to obtain corresponding frequency domain information after Fourier transform of the extracted heartbeat signals, wherein the frequency domain information comprises one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features.

[0132] The fusion module is used to input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into a convolution recurrent neural network for shallow feature extraction, to design a convolution recurrent neural network with two parallel branches to realize joint analysis of electrocardio signals and magnetocardiogram signals by taking the ECG and MCG signals as inputs respectively.

[0133] The classification module is used to comprehensively classify and detect cardiac electrophysiological signals by fusing the features of single heartbeat cycle, i.e. heartbeats, in the time domain and the frequency domain.

[0134] In the present embodiment, the preprocessing module filters and denoises the original ECG and MCG signals respectively, then performs single heartbeat interception, and then obtains a frequency spectrum and a time-frequency spectrum through Fourier transform, and establishes ECG and MCG one-dimensional time domain signal data sets, two-dimensional frequency spectrum data sets and two-dimensional time-frequency spectrum data sets respectively.

[0135] In the embodiment, the long signal interception mode is to locate the R wave using the Pan-Tompkins detection algorithm, and then take sampling points forward and sampling points backward, totaling Fs sampling points as a complete heart beat (where Fs is the sampling rate).

[0136] In the embodiment, the frequency spectrum of the ECG and MCG signals is obtained by fast Fourier transform (FFT), and the time-frequency spectrum of the two is obtained by short-time Fourier transform (STFT).

[0137] In the embodiment, the network adopts a three-module combined structure, wherein the first module is a double-input convolution module, which includes two parallel convolution branches, each branch having the same structure, including a convolution layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer; the second module is a time sequence feature extraction module composed of a gated recurrent unit (GRU), including a reset gate and an update gate; and the third module is a decision module, including a flatten layer, a fully connected (FC) layer, a dense layer, and finally outputting a multi-class probability distribution through a Softmax function to obtain a classification result.

[0138] In the embodiment, the double-input convolution module adopts a one-dimensional and two-dimensional CNN model as the network structure, and the two cardiac electrophysiological signals are simultaneously used as the input of the model; the ECG and MCG signal heart beats or frequency spectrum, time-frequency spectrum are respectively input into three independent network branches; for the one-dimensional CNN model, a batch normalization layer is used after the first layer of convolution layer to standardize the data of each batch and accelerate the training speed; a maximum pooling layer is placed after the batch normalization to reduce the feature dimension; the number of convolution kernels of the second and third layers of convolution layer is increased to extract more features, and other parameters are the same as those of the first convolution layer; for the two-dimensional CNN model, a two-dimensional convolution kernel is used for feature extraction; when constructing the model, the above structure can be repeatedly stacked, and through the multi-level convolution structure, the model can better understand and process complex input data.

[0139] In the embodiment, the time sequence feature extraction module adopts a GRU as the network structure; the input is the shallow signal features extracted by the convolution module, and the output is the extracted time sequence features.

[0140] In the embodiment, a convolutional layer is first added in the decision module to combine the results of different inputs from the previous layer, and then the combined results are subjected to nonlinear transformation using a Softmax activation function. Subsequently, the model uses a convolution kernel with the same output size as the previous step as the FC layer, convolves with the output of the Softmax activation function, and uses a dropout layer (Dropout) and early stopping strategy in the FC layer to prevent overfitting.

[0141] In the embodiment, Z-Score is used to normalize the signals before each signal is input into the network; and the data set is divided using a five-fold cross-validation method.

[0142] In the embodiment, in order to improve the feature extraction capability of the model, an SE module is introduced, and through the double-input CNN-GRU hybrid neural network structure based on the combination of attention mechanism, the time-frequency domain features in the ECG and MCG signals can be fully fused.

[0143] In the embodiment, the network hyperparameters are selected according to the characteristics of the cardiac electrophysiological signals in the double-input convolutional recurrent network, and the parameters of the convolution kernel or the number of hidden units of the GRU network are changed through multiple experiments, so as to realize the improvement of the training speed, classification accuracy, stability and generalization ability of the model.

[0144] The above-described embodiments only describe the preferred modes of the present application, and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements to the technical solutions of the present application made by those skilled in the art shall fall within the protection scope determined by the claims of the present application.

Claims

1. A cardiac electrophysiological signal classification method based on multi-scale fusion of electrocardiogram and magnetocardiogram, characterized in that, The method comprises the following steps: Obtaining original electrocardio signals and original magnetocardiography signals, preprocessing the original electrocardio signals and the original magnetocardiography signals respectively, and dividing heartbeats according to R-wave positioning to extract heartbeat signals, and obtaining corresponding frequency domain information after Fourier transform of the extracted heartbeat signals, wherein the frequency domain information comprises one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features; The obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features are input into a convolutional recurrent neural network for shallow feature extraction, and a convolutional recurrent neural network with two parallel branches is designed to realize joint analysis of electrocardio signals and magnetocardiography signals by taking ECG and MCG signals as inputs; The characteristics of single heartbeat cycle, i.e., heartbeats, in the time domain and the frequency domain are fused to comprehensively classify and detect cardiac electrophysiological signals; The convolutional recurrent neural network adopts a three-module combined structure, wherein the first module is a double-input convolution module, which comprises two parallel convolution branches, each branch having the same structure, comprising a convolution layer, a batch normalization layer, a ReLU activation function and a maximum pooling layer; the second module is a time sequence feature extraction module composed of a gated recurrent unit (GRU), which comprises a reset gate and an update gate; and the third module is a decision module, which comprises a flattening layer, a full connection layer, a dense layer, and finally outputs a multi-class probability distribution through a Softmax function to obtain a classification result; The double-input convolution module adopts one-dimensional and two-dimensional CNN models as network structures, and the two cardiac electrophysiological signals are simultaneously taken as inputs of the model; the ECG and MCG signal heartbeats or the spectrum graph and the time-frequency spectrum graph are respectively input into three independent network branches; for the one-dimensional CNN model, a batch normalization layer is used after the first layer of convolution layer to standardize the data of each batch; a maximum pooling layer is placed after the batch normalization to reduce the feature dimension; the number of convolution kernels of the second and third convolution layers is increased to extract more features, and other parameters are the same as those of the first convolution layer; for the two-dimensional CNN model, a two-dimensional convolution kernel is used for feature extraction; The time sequence feature extraction module adopts a gated recurrent unit (GRU) as the network structure; the input is the shallow signal features extracted by the double-input convolution module, and the output is the extracted time sequence features; In the decision module, a convolution layer is first added to combine the results of different inputs from the previous layer, and then a Softmax activation function is used for nonlinear transformation of the combined results; thereafter, the model uses a convolution kernel with the same output size as the previous step as a full connection layer, convolves with the output of the Softmax activation function, and uses a dropout layer and an early stopping strategy in the full connection layer to prevent overfitting.

2. The method of claim 1, wherein, Obtaining original electrocardio signals, preprocessing the original electrocardio signals, and dividing heartbeats according to R-wave positioning to extract heartbeat signals, and obtaining corresponding frequency domain information after Fourier transform of the extracted heartbeat signals, which comprises: The original electrocardio signal is preprocessed by using wavelet threshold denoising to remove the original noise of the data, filter the baseline drift and power frequency interference, the input signal is decomposed by using multi-order Daubechies wavelet to obtain coefficients at different levels, wherein the coefficients include approximation coefficients and detail coefficients, the denoising threshold is calculated according to the detail coefficients The wavelet coefficients of the two high-frequency components with the highest frequency are set to zero, the coefficients less than the threshold are set to zero to remove the noise, the processed coefficients are converted back to time domain signals by using wavelet inverse transform, the whole electrocardio signal after denoising is obtained, and the time sequence S1 after preprocessing is obtained. Taking the fixed length of heart beat as input, the data points with the length of before and after R wave in time series data S1 are taken as a heart beat cycle signal, so as to divide heart beat; Performing FFT on each heartbeat cycle signal to obtain a spectrum graph of the heartbeat signal, performing STFT on each heartbeat cycle signal to obtain a time-frequency spectrum graph of the heartbeat, and then constructing a one-dimensional ECG signal, a two-dimensional ECG spectrum graph and a two-dimensional ECG time-frequency spectrum graph dataset; Wherein, threshold value The calculation method is: ； wherein D are the processed wavelet coefficients, C is the signal length, median denotes taking the median; Wherein, the data points with the length of before and after the R wave in the intercepted time series data S1 are a heartbeat cycle signal. The R wave is located using the Pan-Tompkins detection algorithm, and the R wave position is taken as a reference point. A total of sampling points are taken forward and sampling points are taken backward, for a total of sampling points as the length of a complete heartbeat cycle, wherein is the sampling rate.

3. The method of claim 1, wherein, The original magnetocardiogram signal is obtained, the original magnetocardiogram signal is preprocessed, and heartbeats are divided according to R-wave positioning, heartbeat signals are extracted, and the extracted heartbeat signals are subjected to Fourier transform to obtain corresponding frequency domain information including: The original magnetocardiogram signal is denoised by using a band-pass filter with a frequency of 1-40 Hz and a 300-window median filter, and baseline drift is removed; The Pan-Tompkins detection algorithm is used to realize R-wave enhancement, and then the positions of R-waves are detected by calculating the amplitude, slope and waveform characteristics of the signal; After determining the window size of the heartbeats, the heartbeat signal of each heartbeat cycle is extracted, the corresponding frequency domain information is obtained by Fourier transform, and then a one-dimensional MCG signal, a two-dimensional MCG spectrum diagram and a two-dimensional MCG time-frequency spectrum diagram dataset are constructed.

4. A cardiac electrophysiological signal classification system based on multi-scale fusion of electrocardiogram and magnetocardiogram, the system is used to implement the method of any one of claims 1-3, characterized in that, It comprises: a preprocessing module, a fusion module and a classification module; The preprocessing module is used to obtain original electrocardiogram signals and original magnetocardiogram signals, pre-process the original electrocardiogram signals and the original magnetocardiogram signals respectively, divide heartbeats according to R-wave positioning, extract heartbeat signals, and obtain corresponding frequency domain information by Fourier transform on the extracted heartbeat signals, wherein the frequency domain information includes one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features; The fusion module is used to input the obtained one-dimensional time domain features, two-dimensional frequency domain features and time-frequency domain features into a convolution recurrent neural network for shallow feature extraction, design a convolution recurrent neural network with two parallel branches to realize joint analysis of electrocardiogram signals and magnetocardiogram signals by taking ECG and MCG signals as inputs respectively; The classification module is used to comprehensively detect the cardiac electrophysiological signals by fusing the features of single heartbeat cycle, i.e., heartbeats, in the time domain and the frequency domain.

Images