Myocardial infarction positioning method and device based on multi-domain depth feature fusion and storage medium
Through the multi-domain deep feature fusion method, the S transform and discrete Fourier transform are used to generate multi-domain images, and combined with convolutional neural network, the inaccuracy problem caused by the dependence of single time domain features in the existing technology is solved, and the myocardium is improved Accuracy of infarction positioning and model generalization ability.
Patent Information
- Application Number
- CN202411245687.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-12-27
AI Technical Summary
The existing deep learning-based myocardial infarction positioning technology relies too much on a single temporal domain feature, resulting in inaccurate detection and poor generalization capabilities. The machine learning-based methods have subjectivity and uncertainty caused by cumbersome feature engineering and manual intervention.
Using the multi-domain deep feature fusion method, time-frequency images and two-dimensional spectral images are generated through S-transform and discrete Fourier transform, and input them into the multi-domain feature fusion convolutional neural network (MFF-CNN), combining one-dimensional Training with two-dimensional convolutional neural networks to realize automatic extraction and fusion of multi-domain features.
It improves the accuracy and generalization performance of myocardial infarction positioning, reduces manual intervention, and enhances the feature characterization ability and the robustness of the model.
Smart Images
Figure CN119184705A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of medical image processing, and in particular to a method, device and storage medium for locating myocardial infarction by fusion of multi-domain deep features. Background Art
[0002] Myocardial infarction, also known as myocardial infarction (MI), is one of the major cardiovascular diseases, and its influence cannot be ignored. The timely diagnosis of MI is of immeasurable importance for initiating emergency treatment measures, limiting the scope of myocardial damage, and improving the survival rate of patients.
[0003] When facing the threat of myocardial infarction, every minute and every second is crucial. Therefore, being able to quickly and accurately identify and locate the infarct area has extremely high medical value in guiding clinicians' decision-making and formulating treatment plans. This can not only buy precious treatment time for patients, but also improve treatment effects, reduce the occurrence of complications, and thus improve the quality of life of patients and prolong their lives.
[0004] Although traditional electrocardiogram (ECG) analysis has long been widely used in clinical diagnosis of MI, and its effectiveness and practicality have been widely recognized, existing deep learning-based MI localization technology often relies too much on a single time domain feature for analysis. This method has obvious deficiencies in capturing the rich frequency domain and time-frequency domain information contained in ECG data, which limits the accuracy and reliability of detection and leads to poor generalization of the model. In addition, the application of machine learning-based diagnostic methods in the field of myocardial infarction localization also faces challenges. These methods often require cumbersome and complex feature engineering, a process that not only requires rich expertise and experience, but also may increase the subjectivity and uncertainty of the results due to manual intervention. Therefore, developing a deep learning model that can comprehensively utilize multiple information in ECG data and reduce the impact of manual intervention is of great significance for improving the accuracy and reliability of MI localization. Summary of the invention
[0005] The purpose of the present invention is to propose a method for locating myocardial infarction by fusing multi-domain deep features, so as to solve the technical problem that the prior art overly relies on a single time domain feature for analysis, resulting in inaccurate myocardial infarction positioning.
[0006] The present invention provides a method for locating myocardial infarction by fusion of multi-domain deep features, comprising the following steps:
[0007] S1: obtaining original 12-lead ECG data and preprocessing it to obtain preprocessed ECG data;
[0008] S2: Use S transform to perform time-frequency analysis on the preprocessed ECG data to obtain the corresponding time-frequency image;
[0009] S3: The one-dimensional amplitude spectrum of the preprocessed ECG data is calculated by discrete Fourier transform, and converted into a two-dimensional spectrum image by Gram angle field;
[0010] S4: The time-frequency image and two-dimensional spectrum image obtained by S2 and S3 together with the original ECG data are input into the multi-domain feature fusion convolutional neural network MFF-CNN which is a combination of one-dimensional and two-dimensional convolutional neural network models designed for their respective data features for training;
[0011] S5: Use the MFF-CNN network for feature classification to localize myocardial infarction.
[0012] A storage medium stores instructions and data for implementing a method for locating myocardial infarction by integrating multi-domain deep features.
[0013] A myocardial infarction locating device with multi-domain deep feature fusion includes: a processor and the storage medium; the processor loads and executes instructions and data in the storage medium to implement a myocardial infarction locating method with multi-domain deep feature fusion.
[0014] The beneficial effects provided by the present invention are:
[0015] 1. The present invention generates spectrum images and time-frequency images and combines them with time-domain electrocardiograms to construct multi-domain information inputs. These inputs provide complementary characteristics, which help to enhance the representation ability and generalization performance of features, thereby improving the positioning accuracy of MI.
[0016] 2. This paper proposes a novel multi-domain feature fusion convolutional neural network (MFF-CNN) for automatically extracting multi-domain deep features from input ECG signals, spectrum images, and time-frequency images. The network integrates the complementary characteristics between domains, realizes reasonable and effective multi-domain feature learning, and improves the generalization ability of the model for MI positioning. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a schematic diagram of the process of the present invention;
[0018] Figure 2 This is a schematic diagram of the model of the myocardial infarction localization method proposed in the present invention.
[0019] Figure 3 This is a comparison diagram of the ECG signal before and after denoising according to the present invention.
[0020] Figure 4 An example of a two-dimensional time-frequency image generated by the present invention.
[0021] Figure 5 An example of a two-dimensional spectrum image generated for the present invention.
[0022] Figure 6 Schematic diagram of the MFF-CNN network structure used in the present invention.
[0023] Figure 7 It is the prediction result of the MFF-CNN network model for the entire data set in the present invention.
[0024] Figure 8 It is a working schematic diagram of the hardware device of the present invention. DETAILED DESCRIPTION
[0025] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0026] Before formally describing the present invention, the scheme of the present invention is first generally described for easy understanding.
[0027] Please refer to Figure 1-Figure 2 , Figure 1 It is a schematic flow diagram of the method of the present invention; Figure 2 This is a schematic diagram of the model of the myocardial infarction localization method proposed in the present invention.
[0028] The present invention provides a method for locating myocardial infarction by fusion of multi-domain deep features, comprising the following steps:
[0029] S1: obtaining original 12-lead ECG data and preprocessing it to obtain preprocessed ECG data;
[0030] It should be noted that the preprocessing in step S1 includes: resampling, denoising, R peak positioning and heart beat segmentation of ECG data.
[0031] As an embodiment, step S1 in the present invention is specifically as follows:
[0032] S11: Read 12-lead ECG data, the data including the original ECG data file and the annotated annotation file;
[0033] Please refer to Figure 3 , Figure 3 It is a comparative schematic diagram of the ECG signal before and after denoising according to the present invention.
[0034] S12: Resample the ECG data to the same sampling rate, use median filtering to remove baseline drift, and then use the improved threshold wavelet denoising method to denoise the ECG signal. The expression of this method is:
[0035]
[0036] Where b (b = 1, ..., 9) is the order of wavelet decomposition, and c is the number of sampling points in the signal. b is the set threshold, which is calculated as: ||d b || is the bi-norm of the wavelet coefficients, σ b is the estimated noise level, calculated as: b =(median(|d b |)) / 0.6745, where 0.6745 is the value of the 75% area (p=0.25) of a Gaussian distribution with a mean of 0 and a variance of 1.
[0037] Then, dB6 wavelet is selected as the wavelet basis, and the signal is decomposed by 5-level wavelet. The wavelet coefficients less than TEb after decomposition are set to zero, thereby removing other noise signals. Then, the Pan-Tompkins algorithm is used to locate the QRS complex and R peak, and the data 0.25s before the R peak and 0.4s after the R peak are taken as a heart beat.
[0038] S2: Use S transform to perform time-frequency analysis on the preprocessed ECG data to obtain the corresponding time-frequency image;
[0039] Please refer to Figure 4 , Figure 4 An example of a two-dimensional time-frequency image generated by the present invention.
[0040] It should be noted that step S2 specifically includes: performing localization processing on the pre-processed ECG data using a Gaussian function to obtain a localized signal; performing Fourier transform on the localized signal, and finally synthesizing it to obtain a time-frequency image.
[0041] As an embodiment, step S2 in the present invention is specifically as follows:
[0042] S21: For the input signal x(t), a Gaussian window function is used to localize the signal. The calculation method of the Gaussian window function is:
[0043]
[0044] Where t is the time length, τ is the time point at the center of the window, and f is the sampling frequency. The width of the Gaussian window is inversely proportional to the frequency f, which means that the window is narrower at higher frequencies and wider at lower frequencies. This design allows the S transform to maintain a constant relative bandwidth at all frequencies, thereby achieving better time-frequency resolution.
[0045] S22: According to the size of the Gaussian window function in S21, a local Fourier transform is performed on the signal x(t). The overall calculation method of the S transform is:
[0046]
[0047] By integrating the time range of the entire signal, we can get the time-frequency image corresponding to the x(t) signal. By observing the time-frequency graph, we can analyze the time-varying characteristics of the signal and identify the local frequency components and their changes.
[0048] S3: Calculate the one-dimensional amplitude spectrum of the preprocessed ECG data using discrete Fourier transform, and convert it into a two-dimensional spectrum image through Gramian Angular Field;
[0049] Please refer to Figure 5 , Figure 5 An example of a two-dimensional spectrum image generated for the present invention.
[0050] It should be noted that the specific process of step S3 is as follows:
[0051] S31: Calculate the one-dimensional amplitude spectrum sequence x={x1, x2, ..., x3, ..., x4, ..., x5, ..., x6, ..., x7, ..., x8, ..., x9, ..., x10, ..., x11, ..., x12, ..., x13, ..., x14, ..., x15, ..., x16, ..., x17, ..., x18, ..., x19, ..., x21, ..., x n};
[0052] S32: Use the following equation to rescale the one-dimensional amplitude spectrum sequence x in the interval [-1,1]:
[0053]
[0054] S33: The adjusted sequence Represented as polar coordinates, encoding each value as the cosine of the angle and radius r i , the formula is as follows:
[0055]
[0056] In the formula, t i and N represent the timestamp and a constant factor respectively;
[0057] S34: Calculate the corresponding Gram matrix by calculating the trigonometric difference of the polar coordinate values between each two points in S33. The calculation formula is:
[0058]
[0059] Where I is a unit row vector and G is a Gram matrix. Through the above transformation, the one-dimensional time series is converted into a two-dimensional matrix while maintaining the time dependency, thereby obtaining a two-dimensional spectrum image.
[0060] Please refer to Figure 6 , Figure 6Schematic diagram of the MFF-CNN network structure used in the present invention.
[0061] S4: The time-frequency image and two-dimensional spectrum image obtained by S2 and S3 together with the original ECG data are input into the multi-domain feature fusion convolutional neural network MFF-CNN which is a combination of one-dimensional and two-dimensional convolutional neural network models designed for their respective data features for training;
[0062] It should be noted that step S4 is specifically as follows:
[0063] S41, dividing the obtained two-dimensional spectrum image and time-frequency image into a training set, a verification set and a test set;
[0064] As an embodiment, the obtained one-dimensional 12-lead ECG heartbeats and the corresponding two-dimensional spectrum images and time-frequency images are divided into a training set, a validation set and a test set in a ratio of 7:2:1.
[0065] S42, for two-dimensional spectrum images, the original ResNet18 network is used; for time-frequency images, the improved one-dimensional SE-ResNet18 network is used for training;
[0066] It should be noted that, in the present invention, ResNet18 and an improved one-dimensional SE-ResNet18 network are used for data of different dimensions.
[0067] For two-dimensional image data, the original ResNet18 network is used; for one-dimensional data, the initial convolution kernel size of the one-dimensional ResNet18 network is modified to 15, the subsequent convolution kernel size is modified to 7, and SE blocks and a dropout layer (dropout = 0.2) are added between each residual block to obtain the improved one-dimensional SE-ResNet18 network.
[0068] It should be noted that the importance weight vector of each channel is output through the SE block. In this way, global information can be extracted from the multi-channel feature map, and the importance of each channel in the overall feature representation can be clarified. This adaptive channel weight allocation mechanism helps to improve model performance and robustness, better adapt to complex scenarios, and the added dropout layer can also reduce the overfitting of the model.
[0069] S43. Initialize the training hyperparameters and perform training based on the loss function to obtain the trained multi-domain feature fusion convolutional neural network MFF-CNN.
[0070] As an example, the hyperparameters in the present invention are set as follows: a stochastic gradient descent method with a learning rate of 0.001 and a momentum of 0.9 is used as an optimizer. During training, the data is divided into batches of 64 and trained for 60 cycles.
[0071] As an embodiment, during the training process of the present invention, due to the existence of the sample imbalance problem, a weighted cross-entropy loss function is used to train different models, and its expression is:
[0072]
[0073] Where C is the number of categories, here it is 12; y ic Represents the true value, which is a one-hot encoded vector in the formula. If sample i belongs to the corresponding category c, then y ic is 1, otherwise it is 0; is the probability that the model predicts that sample i belongs to the corresponding category c.
[0074] Finally, the present invention effectively integrates the feature vectors learned by each network through the fully connected layer, and performs feature classification through the final softmax layer to obtain the positioning prediction result of myocardial infarction by the MFF-CNN network.
[0075] S5: Use the MFF-CNN network for feature classification to localize myocardial infarction.
[0076] It should be noted that step S5 is specifically as follows:
[0077] S51: Load the MFF-CNN network model and import the model parameters saved in S44;
[0078] S52: The test data was input into the MFF-CNN network model to obtain the localization prediction results of myocardial infarction, and by comparing with the correct label, the accuracy (Acc), sensitivity (Sen), precision (Pre), specificity (Spe) and F1 score (F1-Score) of the model for myocardial infarction localization were calculated.
[0079] Please refer to Figure 7 , Figure 7 It is the prediction result of the MFF-CNN network model for the entire data set in the present invention.
[0080] See also Figure 8 , Figure 8It is a working diagram of the hardware device of an embodiment of the present invention, and the hardware device specifically includes: a myocardial infarction positioning device 401 with multi-domain deep feature fusion, a processor 402 and a storage medium 403.
[0081] A myocardial infarction positioning device 401 with multi-domain deep feature fusion: The myocardial infarction positioning device 401 with multi-domain deep feature fusion implements the myocardial infarction positioning method with multi-domain deep feature fusion.
[0082] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the method for locating myocardial infarction by integrating multi-domain deep features.
[0083] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the myocardial infarction positioning method using multi-domain deep feature fusion.
[0084] The beneficial effects of the present invention are:
[0085] 1. The present invention generates spectrum images and time-frequency images and combines them with time-domain electrocardiograms to construct multi-domain information inputs. These inputs provide complementary characteristics, which help to enhance the representation ability and generalization performance of features, thereby improving the positioning accuracy of MI.
[0086] 2. This paper proposes a novel multi-domain feature fusion convolutional neural network (MFF-CNN) for automatically extracting multi-domain deep features from input ECG signals, spectrum images, and time-frequency images. The network integrates the complementary characteristics between domains, realizes reasonable and effective multi-domain feature learning, and improves the generalization ability of the model for MI positioning.
[0087] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for locating myocardial infarction by fusion of multi-domain deep features, characterized by: The method comprises the following steps: S1: obtaining original 12-lead ECG data and preprocessing it to obtain preprocessed ECG data; S2: Use S transform to perform time-frequency analysis on the preprocessed ECG data to obtain the corresponding time-frequency image; S3: The one-dimensional amplitude spectrum of the preprocessed ECG data is calculated by discrete Fourier transform, and converted into a two-dimensional spectrum image by Gram angle field; S4: The time-frequency image and two-dimensional spectrum image obtained by S2 and S3 together with the original ECG data are input into the multi-domain feature fusion convolutional neural network MFF-CNN which is a combination of one-dimensional and two-dimensional convolutional neural network models designed for their respective data features for training; S5: Use the MFF-CNN network for feature classification to localize myocardial infarction.
2. The method for locating myocardial infarction by fusion of multi-domain deep features as claimed in claim 1, characterized in that: The preprocessing in step S1 includes resampling, denoising, R-peak location and heart beat segmentation of ECG data.
3. The method for locating myocardial infarction by fusion of multi-domain deep features as claimed in claim 1, characterized in that: Step S2 specifically includes: using a Gaussian function to perform localization processing on the pre-processed ECG data to obtain a localized signal; performing Fourier transform on the localized signal, and finally synthesizing it to obtain a time-frequency image.
4. The method for locating myocardial infarction by fusion of multi-domain deep features as claimed in claim 1, characterized in that: The specific process of step S3 is as follows: S31: Calculate the one-dimensional amplitude spectrum sequence x={x1, x2, ..., x3, ..., x4, ..., x5, ..., x6, ..., x7, ..., x8, ..., x9, ..., x10, ..., x11, ..., x12, ..., x13, ..., x14, ..., x15, ..., x16, ..., x17, ..., x18, ..., x19, ..., x21, ..., x n }; S32: Use the following equation to rescale the one-dimensional amplitude spectrum sequence x in the interval [-1,1]: S33: The adjusted sequence Represented as polar coordinates, encoding each value as the cosine of the angle and radius r i , the formula is as follows: Where, t i and N represent the timestamp and a constant factor respectively; S34: Calculate the corresponding Gram matrix by calculating the trigonometric difference of the polar coordinate values between each two points in S33. The calculation formula is: where I is a unit row vector, G is a Gram matrix, express Through the above transformation, the one-dimensional time series is converted into a two-dimensional matrix, thereby obtaining a two-dimensional spectrum image.
5. The method for locating myocardial infarction by fusion of multi-domain deep features as claimed in claim 1, characterized in that: Step S4 is specifically as follows: S41, dividing the obtained two-dimensional spectrum image and time-frequency image into a training set, a verification set and a test set; S42, for two-dimensional spectrum images, the original ResNet18 network is used; for time-frequency images, the improved one-dimensional SE-ResNet18 network is used for training; S43. Initialize the training hyperparameters and perform training based on the loss function to obtain the trained multi-domain feature fusion convolutional neural network MFF-CNN.
6. The method for locating myocardial infarction by fusion of multi-domain deep features as claimed in claim 5, characterized in that: The improved one-dimensional SE-ResNet18 network is specifically as follows: the initial convolution kernel size of the one-dimensional ResNet18 network is modified to 15, the subsequent convolution kernel size is modified to 7, and an SE block and a dropout layer are added between each residual block, and the importance weight vector of each channel is output through the SE block.
7. The method for locating myocardial infarction by fusion of multi-domain deep features as claimed in claim 5, characterized in that: The loss function in step S43 is as follows: Where C is the number of categories; w i represents the weight of category i, which is set according to the sample ratio; y ic Represents the true value, which is a one-hot encoded vector in the formula. If sample i belongs to the corresponding category c, then y ic is 1, otherwise it is 0; is the probability that the model predicts that sample i belongs to the corresponding category c.
8. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a method for locating myocardial infarction by fusion of multi-domain deep features as described in any one of claims 1 to 7.
9. A myocardial infarction location device with multi-domain deep feature fusion, characterized by: include: Processor and storage medium; the processor loads and executes instructions and data in the storage medium to implement the myocardial infarction localization method with multi-domain deep feature fusion as described in any one of claims 1 to 7.