A Scarred Cardiac Cardiac Imaging and Deep Learning-Based Identification and Classification Method

Abstract

This invention discloses an automatic identification and classification method for scarred myocardium based on cine magnetic resonance imaging (CMRI) and deep learning. After preprocessing cardiac cMRI images, the method extracts the relative motion and imaging features of the left ventricular myocardium. A multimodal feature fusion network is then used for training and prediction to identify and classify five different types of scarred myocardium. The extraction of relative motion and imaging features enables an accurate and detailed description of the state of scarred myocardium. The multimodal feature fusion network model captures the compensatory effects of long-distance scarred myocardium movement, the non-rigid deformation motion patterns of the myocardium, and the spatiotemporal relationship between myocardial motion and morphology. This invention achieves high efficiency and accuracy in identifying and classifying scarred myocardium, saving doctors time and effort in diagnosing cardiovascular diseases leading to scarred myocardium, reducing patient suffering, and facilitating a more stable and simpler examination and diagnostic method.

Description

Technical Field

[0001] This invention belongs to the field of medical imaging technology, specifically relating to an automatic identification and classification method for scarred myocardium based on cinema magnetic resonance imaging and deep learning. Background Technology

[0002] The identification and classification of scarred myocardium is of significant clinical importance for the diagnosis and treatment of cardiovascular diseases. The appearance of myocardial scarring often signifies disease aggravation and deterioration, indicating severe and irreversible damage to the heart. If the expansion of myocardial scarring is not detected and controlled in time, it can lead to life-threatening outcomes such as heart failure, cardiac arrest, or sudden death. Therefore, the identification of scarred myocardium plays a crucial role in assessing a patient's risk level and prognosis. Furthermore, many common cardiovascular diseases can lead to myocardial scarring. For example, myocarditis and hypertrophic cardiomyopathy can cause myocardial fibrosis, while myocardial infarction can lead to myocardial ischemia and necrosis. Although the underlying causes differ, they all present with similar myocardial scarring and clinical symptoms, increasing the difficulty and risk of cardiovascular disease diagnosis for physicians. Inaccurate etiology leading to incorrect treatment can significantly exacerbate myocardial damage. For example, if a myocardial infarction is misdiagnosed as acute myocarditis, it will delay the opening of blood vessels, causing greater irreversible damage and even endangering life. If digitalis drugs used to treat myocarditis are misused in patients with hypertrophic cardiomyopathy, it will aggravate the myocardial diastolic dysfunction in hypertrophic cardiomyopathy. Therefore, classifying scarred myocardium can fundamentally analyze the pathology and etiology of injured myocardium and assist doctors in clinical diagnosis.

[0003] Under current technological conditions, there are many challenges to be addressed in the clinical assessment of scarred myocardium. On the one hand, the diagnostic procedures are extremely cumbersome and time-consuming. A systematic and accurate assessment of scarred myocardium often requires multiple examination methods, such as electrocardiogram (ECG), echocardiogram (UCG), computed tomography (CT), and magnetic resonance imaging (MRI). On the other hand, some necessary examination methods and diagnostic techniques can cause harm to the human body. For example, myocardial biopsy—the gold standard for diagnosing myocarditis and cardiomyopathy—is invasive and causes great pain to patients, while also carrying the risk of complications such as cardiac perforation and cardiac tamponade. Delayed gadolinium contrast technology is also of great significance in the detection and visualization of scar tissue, but intravenous injection of gadolinium contrast agents is limited in patients with acute and chronic renal failure. Moreover, recent studies have shown that linear gadolinium chelates may deposit in the brain and bones, which brings uncertainty to both patients and treating physicians. Therefore, the field of scarred myocardium monitoring urgently needs a simple, rapid, and harmless detection technology.

[0004] Cine magnetic resonance imaging (Cine-MRI), as a non-invasive and non-radioactive imaging technique, has become an important tool for the examination and assessment of clinical cardiovascular diseases due to its multi-parameter and blind-spot-free characteristics. Therefore, it is of great significance to evaluate and detect scarred myocardium using Cine-MRI. However, the formation of scarred myocardium is very complex, exhibiting complex motion properties and morphological characteristics. The effects of viral infection and certain ischemic myocardial scars on myocardial motion are not significantly different, making differentiation on Cine-MRI images highly dependent on the physician's experience. Furthermore, if the size of the myocardial scar exceeds a certain threshold, it will disrupt the balance of myocardial motion during the compensatory period, resulting in complex and variable motion patterns, making the classification of scarred myocardium even more difficult. Therefore, assessing the condition of scarred myocardium on cardiac Cine-MRI images using traditional manual methods is challenging.

[0005] Using deep learning techniques to assess scarred myocardium in cardiac film images is very interesting; however, existing deep learning methods have certain limitations. The core of deep learning methods lies in feature extraction models and feature learning networks. Popular feature extraction methods include obtaining motion features such as myocardial strain coefficients or absolute motion parameters through optical flow calculations or image registration, as well as imaging features such as statistics, texture, and ejection fraction. In the MICCAI 2017 Cardiac Automated Diagnosis Challenge, several researchers proposed their methods for the ACDC dataset, which includes five subclasses: normal subjects, dilated cardiomyopathy, hypertrophic cardiomyopathy, abnormal right ventricle, and myocardial infarction. They all performed feature extraction based on cardiac segmentation, but there were some differences in the selected cardiac features. Mahendra Khened and Jelmer M. Wolterink chose to use the ejection fractions of the left and right ventricles, end-systolic and end-diastolic volumes, and patient height and weight information for network training; Fabian Isensee added some simple shape and texture information; Irem Cetin, in addition to shape and image intensity information, also used advanced texture radiometric features such as gray-level co-occurrence matrix and gray-level run-length matrix; subsequently, Henry... At the 43rd IEEE International Conference on Medical and Biological Engineering in 2021, a myocardial segmentation-based optical flow algorithm was proposed to extract the cardiac motion flow model from movie magnetic resonance images, and the feasibility of the method was demonstrated on the same dataset. In addition, many feature extraction methods have been proposed in the field of chronic myocardial infarction diagnosis: Bettina Baessler published an article in *Radiology*, using the open-source MaZda software to extract 286-dimensional myocardial texture features, including brightness, uniformity, density, and roughness of the overall texture, and then performed corresponding feature selection and dimensionality reduction to restore the feature set that contributes most to classification accuracy; Nan Zhang published an article in *Radiology*, selecting local motion features extracted from recurrent neural networks and global motion features derived using a dense optical flow algorithm for comprehensive training; M. Polacin's experimental study in *Scientific Reports* registered movie sequence images and extracted overall and segmental three-dimensional myocardial deformation indices from end-diastolic and end-systolic images as training datasets. In addition, Jennifer Mancio proposed in an article published in the European Heart Journal - Cardiovascular Imaging that combined morphological and functional characteristics of the myocardium to distinguish whether a patient with myocardial hypertrophy has fibrosis or not, which was mainly based on calculations of the endocardial and epicardial contours.

[0006] Existing models extract cardiac features, which can be broadly categorized into two types based on their properties: One type is cardiac motion features, primarily including image registration and optical flow methods. These methods involve extensive time-series-based detailed calculations of movie images to obtain absolute cardiac motion properties. For example, optical flow characterizes the displacement vector field of pixels in a continuous time-series image, necessitating pixel-level calculations frame-by-frame, requiring significant computational power and placing certain demands on image quality. The other type is radiographic features, mainly calculated based on the patterns between pixels in a single frame of magnetic resonance imaging (MRI) or the morphological features of the heart, such as texture features and radiographic features. On the one hand, while some advanced texture and radiographic features obtained through multiple high-dimensional operations possess considerable discriminative data, they also face limitations due to computational complexity. On the other hand, some computationally simple global scalar features only describe the overall properties, thus losing much detailed information.

[0007] In feature training networks, fully connected networks possess a global receptive field for movie images, convolutional neural networks are primarily used to acquire local information from magnetic resonance images, and recurrent neural networks have significant advantages in learning temporal features of movie images. However, these two traditional deep learning networks struggle with tasks requiring simultaneous acquisition of local and global information, as well as multimodal feature fusion learning tasks. The recently proposed Transformer architecture can better capture long-range dependencies and global information, thus exhibiting better performance in the aforementioned situations. However, its massive computational cost, loss of positional information, and weak ability to acquire local information indicate that it still falls short of perfectly meeting the multimodal feature learning task of scarred myocardium in movie magnetic resonance images.

[0008] In existing research, Cetin used support vector machines, while the rest implemented their methods by training random forest classifiers. In addition, Henry... In 2021, a novel approach using Conditional Generative Adversarial Networks (CGAN) was proposed, and its feasibility was demonstrated on the ACDC dataset. Currently, most networks applied to scar myocardial subtyping still fall under the category of machine learning, lacking interpretability in feature analysis and exhibiting relatively random model training mechanisms. The use of CGAN is a significant innovation, but the network model is complex and time-consuming, and still has certain shortcomings in terms of simplicity and stability. Summary of the Invention

[0009] In view of the above, the present invention provides an automatic identification and classification method for scarred myocardium based on cine magnetic resonance imaging and deep learning, which can achieve high efficiency and high accuracy in identifying and classifying scarred myocardium, saving doctors the difficulty and time in diagnosing related cardiovascular diseases that cause scarred myocardium, reducing patient suffering, and facilitating a more stable and simple examination and diagnosis method.

[0010] An automatic identification and classification method for scarred myocardium based on cinema magnetic resonance imaging and deep learning includes the following steps:

[0011] (1) Preprocess the cardiac cine magnetic resonance images of five types of scarred myocardium: hypertrophic cardiomyopathy with positive fibrosis, hypertrophic cardiomyopathy with negative fibrosis, myocardial infarction, myocarditis, and normal myocardium, to obtain left ventricular images with myocardial intima and endocardium annotations.

[0012] (2) The left ventricular image is subjected to feature extraction to obtain relative motion feature data and imaging feature images of the myocardium;

[0013] (3) Perform data augmentation processing on the imaging features of the myocardium;

[0014] (4) Obtain a large number of samples according to steps (1) to (3). Each set of samples includes the relative motion feature data of myocardium, the enhanced myocardial imaging feature image and the scar myocardium category label of the corresponding cardiac cine magnetic resonance image. Divide all samples into training set, validation set and test set.

[0015] (5) Construct a multimodal feature fusion network model and train it using training set samples to obtain an automatic identification and classification model for scarred myocardium; the multimodal feature fusion network model captures and extracts the compensatory effect of distant scarred myocardium, the non-rigid deformation motion law of myocardium, and the spatiotemporal relationship between myocardial motion and morphology based on the relative motion feature data and imaging feature images of myocardium to achieve classification and identification, and obtains the confidence level of the input sample corresponding to the above five types of scarred myocardium;

[0016] (6) Input the test set sample into the automatic identification and classification model of scar myocardium, and the confidence level of the sample corresponding to the above five types of scar myocardium can be output. Based on the confidence level, it can be determined whether there is scar myocardium and the type of scar myocardium in the cardiac cine magnetic resonance image corresponding to the test set sample.

[0017] Furthermore, the cardiac cine magnetic resonance images are image sequences obtained by performing cine magnetic resonance imaging on different layers of the heart in a short-axis view.

[0018] Furthermore, in step (1), the cardiac cine magnetic resonance image is preprocessed, including detecting and cropping the complete left ventricular region image and segmenting the left ventricular myocardium and epicardium.

[0019] Furthermore, the extraction process of myocardial imaging feature image in step (2) is as follows: first, the myocardial label is obtained by subtracting the myocardial epicondyle label information from the myocardial epicondyle label information in the left ventricular image, and then the result of the Hadamard product operation between the myocardial label and the left ventricular image is used as the myocardial imaging feature image.

[0020] Furthermore, the extraction process of myocardial relative motion feature data in step (2) is as follows:

[0021] First, a rectangular coordinate system is established for the left ventricular image to obtain the coordinates of each point on the inner and outer boundaries of the myocardium. The geometric centroid of the endocardium is then calculated as the centroid point C(x) of the myocardium. c y c );

[0022] Then, starting from the x-axis as the initial angle, rays are drawn outward from the centroid C as the endpoint, with a 3° interval between adjacent rays, resulting in a total of 120 rays. The coordinates of the intersection points of each ray with the myocardium and the endocardium are recorded.

[0023] For any given ray, calculate the deformation data of the intima relative to the center of mass, the deformation data of the epicondyle relative to the center of mass, the deformation data of the myocardium relative to the center of mass, the tangential deformation data of the intima, and the tangential deformation data of the epicondyle, where:

[0024] The deformation data of the inner membrane relative to the center of mass includes the x-axis component ΔX. en (i) y-axis direction component ΔY en (i) and distance D en (i);

[0025] ΔX en (i)=X enp (i)-x c

[0026] ΔY en (i)=Y enp (i)-y c

[0027]

[0028] The deformation data of the outer membrane relative to the center of mass includes the x-axis component ΔX. ep (i) y-axis direction component ΔY ep (i) and distance D ep (i);

[0029] ΔX ep (i)=X epp (i)-x c

[0030] ΔY ep (i)=Y epp (i)-y c

[0031]

[0032] The myocardial relative centroid deformation data includes the x-axis component ΔX. pn (i) y-axis direction component ΔY pn (i) and distance D pn (i);

[0033] ΔX pn (i)=X epp (i)-X enp (i)

[0034] ΔY pn (i)=Y epp (i)-Y enp (i)

[0035]

[0036] The tangential deformation data of the intima includes the x-axis component ΔX. tgn (i) y-axis direction component ΔY tgn (i) and distance D tgn (i);

[0037] ΔX tgn (i)=X enp (i+1)-X enp (i)

[0038] ΔY tgn (i)=Y enp (i+1)-Y enp (i)

[0039]

[0040] The outer membrane tangential motion deformation data includes the x-axis component ΔX. tgp (i) y-axis direction component ΔY tgp (i) and distance D tgp (i);

[0041] ΔX tgp (i)=X epp (i+1)-X epp (i)

[0042] ΔY tgp (i)=Y epp (i+1)-Y epp (i)

[0043]

[0044] Where: X enp (i) and Y enp (i) are the x-axis and y-axis coordinates of the intersection point of ray i and the myocardial endothelium, respectively. epp (i) and Y epp (i) are the x-axis coordinates and y-axis coordinates of the intersection point of ray i and the epicardium, respectively, where i is the index number of the ray;

[0045] Based on the above, the motion deformation data of each ray contains 15 feature values. Combining the motion deformation data of all rays yields a relative motion feature matrix with a dimension of 120×15, which serves as the relative motion feature data of the myocardium.

[0046] Furthermore, the specific implementation of step (3) is as follows: first, the imaging feature image of the myocardium is divided into blocks, and then the image blocks are randomly rotated, transposed and recombined into a new imaging feature image.

[0047] Furthermore, the multimodal feature fusion network model first segments the myocardial imaging feature images in the sample into multiple image patches, and then passes these image patches through a linear layer to obtain a set of image embeddings. Then, the image embeddings and the relative motion feature data of the myocardium in the sample are processed sequentially through a multilayer residual perceptron and an average pooling layer to obtain their respective feature vectors, which are then concatenated. The concatenated multimodal feature matrix is ​​then passed sequentially through a multilayer residual perceptron, an average pooling layer, and a linear classifier to output a confidence vector, which contains the confidence of the input sample for the five types of scarred myocardium.

[0048] Furthermore, the multilayer residual perceptron is composed of 6 cascaded ResMLP modules. Each ResMLP module consists of a linear sublayer applied across image blocks and a feedforward sublayer applied across channels. The linear sublayer and the feedforward sublayer have two affine element transformations at their beginning and end, respectively, and are connected using residuals.

[0049] Furthermore, the specific process of training the multimodal feature fusion network model in step (5) is as follows:

[0050] 5.1 Initialize model parameters, including the bias vector and weight matrix of each layer, learning rate, and optimizer;

[0051] 5.2 Input the myocardial relative motion feature data and myocardial imaging feature images from the training set samples into the model. The model outputs the corresponding classification and recognition result, i.e., the confidence vector, through forward propagation. Calculate the cross-entropy loss function between the confidence vector and the category label in the sample.

[0052] 5.3 Based on the loss function, the optimizer iteratively updates the model parameters using gradient descent until the loss function converges, and the training is complete;

[0053] 5.4 After training is completed, the model is validated using the validation set samples. The model that performs best on the validation set is used as the final automatic identification and classification model for scarred myocardium.

[0054] This invention extracts the relative motion and imaging features of the left ventricular myocardium after preprocessing cardiac cine magnetic resonance images, and uses a multimodal feature fusion network for training and prediction to identify and classify five different types of scarred myocardium. The extraction of relative motion and imaging features enables an accurate and detailed description of the state of scarred myocardium; the data augmentation module alleviates the problem of data imbalance among different scarred myocardium categories; and the multimodal feature fusion network model captures and extracts the compensatory effects of long-distance scarred myocardium movement, the non-rigid deformation motion patterns of the myocardium, and the spatiotemporal relationship between myocardial motion and morphology. This invention achieves high-efficiency and high-accuracy identification and classification of scarred myocardium, saving doctors the difficulty and time in diagnosing cardiovascular diseases leading to scarred myocardium, reducing patient suffering, and facilitating a more stable and simpler examination and diagnostic method. Attached Figure Description

[0055] Figure 1 This is a schematic flowchart of the scar myocardial tissue identification and classification method of the present invention.

[0056] Figure 2 This is a schematic diagram of the preprocessing workflow for cardiac cine magnetic resonance images.

[0057] Figure 3 This is a flowchart illustrating the process of a relative motion feature extraction model.

[0058] Figure 4 This is a flowchart illustrating the process of an image feature extraction model.

[0059] Figure 5 This is a flowchart illustrating the data augmentation model.

[0060] Figure 6 This is a schematic diagram of the structure of a multimodal feature fusion network model.

[0061] Figure 7 ROC plot of scar myocardial typing experiment using the method of the present invention. Detailed Implementation

[0062] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] The method for identifying and classifying scarred myocardium in this invention is as follows: Figure 1 As shown, the original cinema magnetic resonance images are first preprocessed. The preprocessing operations include detecting and cropping the complete left ventricular region and segmenting the left ventricular myocardium and epicardium.

[0064] The preprocessing operations are implemented using existing deep learning networks, including the YOLO-v5 target detection algorithm for left ventricular detection and the U-Net fully convolutional neural network for myocardial segmentation. The preprocessing steps are as follows: Figure 2 As shown, cardiac cine magnetic resonance images are input into the YOLO-v5 network. First, the location of the left ventricle is detected, and the region of interest containing the complete left ventricle is automatically cropped. The resulting image size is then uniformly adjusted to 128×128 before being used as input to the U-Net network. The U-Net network finally outputs the endocardial and epicardial label images of the left ventricular myocardium, completing the preprocessing operation.

[0065] Then, a relative motion-imagery multimodal feature extraction model is established, which includes two parts: a relative motion feature calculation model and an imagery feature extraction model.

[0066] The principle of the relative motion characteristic calculation model is as follows: Figure 3 As shown, the core of calculating relative motion features lies in establishing a center point angle quantification model for the left ventricular myocardium. First, a rectangular coordinate system needs to be established on the preprocessed left ventricular image, and the coordinates of each point on the inner and outer boundaries of the myocardium need to be obtained, denoted as (X...). en Y en ), (X ep Y ep Then, the geometric centroid of the endocardium is calculated as the centroid point C(x) of the myocardium. c y c ), and set as the center point of the quantization model, as shown in the following formula:

[0067]

[0068] Where: N is the intima coordinate vector X en Or Y en The length.

[0069] Starting from the x-axis as the initial angle, draw rays outward from the center point C, with a 3° interval between adjacent rays. Obtain the coordinate matrix P of the intersection points of each ray with the inner and outer membranes. en (X enp Y enp ), P ep (X epp Y epp The matrix has a length of 120. Then, based on the intersection point coordinate matrix P... en P ep The motion deformation data in different directions are calculated using a single ray as the unit, mainly including radial motion deformation and tangential motion deformation.

[0070] Radial motion deformation calculation includes the motion of the intima relative to the center of mass, the motion of the epicondyle relative to the center of mass, and the motion of the myocardium relative to the center of mass. Taking the i-th ray as an example, the motion of the intima relative to the center of mass is divided into x-axis direction vector ΔX. en (i) y-axis direction vector ΔY en (i) and distance D en (i), the calculation formula is as follows:

[0071] ΔX en (i)=X enp (i)-x c

[0072] ΔY en (i)=Y enp (i)-y c

[0073]

[0074] Similarly, the motion of the outer membrane relative to the center of mass also includes the x-axis direction vector ΔX. ep (i) y-axis direction vector ΔY ep (i) and distance D ep (i), the calculation formula is as follows:

[0075] ΔX ep (i)=X epp (i)-x c

[0076] ΔY ep (i)=Y epp (i)-y c

[0077]

[0078] The relative centroidal deformation of the myocardium is characterized by the relative relationship between the intersection of the same ray and the endocardium and endocardium. Similar to the former, it also includes the following three parts:

[0079] ΔX pn (i)=X epp (i)-X enp (i)

[0080] ΔY pn (i)=Y epp (i)-Y enp (i)

[0081]

[0082] Tangential motion deformation is characterized by calculating the directional and distance relationships between adjacent rays and the intersection points of the inner and outer membranes. It is divided into inner membrane tangential motion and outer membrane tangential motion, and the corresponding calculation formulas are uniformly expressed as follows:

[0083] ΔX tg (i)=X E (i+1)-X E (i)

[0084] ΔY tg (i)=Y E (i+1)-Y E (i)

[0085]

[0086] Where: ΔX tg (i) and ΔY tg (i) represent the x-axis and y-axis direction vectors of the tangential motion of the endocardium or epicardium, respectively. tg (i) represents the relative distance of tangential motion, and the coordinate matrix of the intersection point of the ray set and the inner and outer membranes (X). enp Y enp ) and (X epp Y epp ) use (X E Y E ) is used to represent.

[0087] The above formula is used to calculate each ray and finally combine them into a relative motion feature matrix with a dimension of 120×15.

[0088] Image feature extraction models such as Figure 4 As shown, the model further processes the preprocessed left ventricular image: First, the myocardial extent label is obtained by subtracting the myocardial endometrial label from the left ventricular epicardial label image, i.e., the pixel value of the myocardial part is 1, and the pixel value of the non-myocardial part is 0; then, the Hadamard product of the left ventricular image and the myocardial label is calculated to obtain 128×128 as the corresponding myocardial imaging feature.

[0089] To alleviate the data imbalance among different scarred myocardial types in the dataset, a data augmentation model is established. The model principle is as follows: Figure 5 As shown, the initial 128×128 radiographic feature image is first divided into 8×8 non-overlapping 16×16 image blocks; then, the 64 block matrices are randomly rotated, transposed, and rearranged to form an image of the same size as the original image but without the original geometric structure and semantics; finally, the enhanced radiographic feature image is combined with the relative motion features corresponding to the initial radiographic feature image to form a new set of case data.

[0090] Establish a multimodal feature fusion learning network, the network structure and corresponding data flow are as follows: Figure 6 As shown, it consists of two modules: independent training of relative motion-image multimodal features and multimodal feature fusion training. Both network modules are composed of a series of layers with the same structure. Each layer includes a linear sublayer applied across image patches and a feedforward sublayer applied across channels. Each sublayer also has a residual connection and two affine element transformations, hence the structure is called a residual perceptron. The initial 128×128 image feature image is divided into 8×8 non-overlapping patches as input, where the patch size is equal to 16×16. The patches are then independently processed by linear layers to form a set of 64 256-dimensional embeddings. The 120×15 relative motion feature matrix and the resulting 64 image feature embeddings are input into the corresponding residual multilayer perceptron sequence (composed of 6 residual perceptron layers), respectively. The two different features are trained independently, producing 120 15-dimensional output relative motion embeddings and 64 256-dimensional output image embeddings.

[0091] Then, the relative motion embedding set and output image embedding set output by the independently trained networks are average pooled to obtain 15-dimensional and 256-dimensional feature vectors, respectively. The two vectors are then concatenated to form a 271-dimensional vector to represent the myocardial features of a single movie image. Since the dataset for a single patient consists of 11 movie sequences of 20 frames each, an 11×20×271 representation matrix is ​​obtained. This matrix is ​​rearranged and dimensionality reduced in a fixed order to obtain a 220×271 multimodal feature matrix. This multimodal feature matrix is ​​then input into a multimodal feature fusion training network, which is also composed of residual perceptron sequences. This produces a 220×271 output embedding set. The output embedding set is average pooled to obtain a 271-dimensional vector, which is then fed into a linear classifier to obtain a 5-dimensional feature vector as the classification result. The 5-dimensional feature vector corresponds one-to-one with the type of scarred myocardium and represents the probability of predicting that type of scarred myocardium.

[0092] The labels were categorized according to the type of cardiovascular disease. Subjects were divided into five categories: hypertrophic cardiomyopathy with positive fibrosis, hypertrophic cardiomyopathy with negative fibrosis, myocardial infarction, myocarditis, and normal subjects, and were labeled as 0 to 5 respectively.

[0093] During training, the Adam optimizer is used, and the CrossEntropyLoss function is used to calculate the loss function value between the feature vectors and labels output by the network. The network parameters are optimized by iterative training on the data. At the end of each training round, the accuracy is checked on the test set and the model is saved.

[0094] Once the multimodal feature fusion learning network is trained and the network parameter weights are determined, preprocessing is performed on the cine magnetic resonance imaging sequence of new patients. Then, a multimodal feature matrix of scarred myocardium is extracted using a relative motion-imaging multimodal feature extraction model. This feature matrix is ​​used as input to the multimodal feature fusion learning network. The calculated feature vector represents the probability of a specific type of scarred myocardium. The type of scarred myocardium with the highest probability is used as the predicted value. If the predicted value is 0, the prediction result is hypertrophic cardiomyopathy with positive fibrosis; if the predicted value is 1, the prediction result is hypertrophic cardiomyopathy with negative fibrosis; if the predicted value is 2, the prediction result is myocardial infarction; if the predicted value is 3, the prediction result is myocarditis; and if the predicted value is 4, the prediction result is a normal subject. Ultimately, the identification and classification of scarred myocardium types are achieved.

[0095] The following experiments, based on real patient cinematic magnetic resonance imaging data, verify the effectiveness of this invention. The experiments were conducted on a Windows 11 computer equipped with a 12th Gen Intel(R) Core(TM) i7-12700H CPU, the PyTorch deep learning framework, and an NVIDIA GeForce RTX 3070Ti graphics card to accelerate code execution.

[0096] The original dataset was collected from Sir Run Run Shaw Hospital and the First Affiliated Hospital of Zhejiang University School of Medicine, containing 62,120 cine magnetic resonance images from 450 subjects. Based on the etiology of scarred cardiomyopathy, the dataset was divided into 109 cases of myocardial infarction, 44 cases of myocarditis, 273 cases of hypertrophic cardiomyopathy, and a control group of 24 cases. The hypertrophic cardiomyopathy group was further divided into 214 cases with positive fibrosis and 59 cases with negative fibrosis.

[0097] Due to individual differences and sampling errors among patients, different patients undergo imaging at 3 to 11 slices within the three short axes of the heart (basal, intermediate, and apical). Furthermore, because different MRI devices exist, the number of images in a single-slice cine sequence varies, ranging from 20 to 25 images. To address the issue of mismatched training data dimensionality, this experiment employed the following processing method: for cases with fewer than 11 slices, image sequences from different slices were sequentially and cyclically arranged until the number reached 11; for data with 25 images in a single sequence, equidistant sampling at a 4:1 ratio was used to ensure that each sequence contained 20 images. This processing standardizes the dimensionality of measurement data from different devices and patients, resulting in a dataset of 11 × 20 images for each patient, facilitating subsequent feature extraction and training.

[0098] The above data were input into the system for preprocessing and data augmentation, ultimately yielding a total of 573 cases: 214 cases of hypertrophic cardiomyopathy with positive fibrosis, 100 cases of hypertrophic cardiomyopathy with negative fibrosis, 109 cases of myocardial infarction, 100 cases of myocarditis, and 50 normal subjects. This experiment used 10-fold cross-validation to evaluate the performance of this invention in scarred myocardium identification and classification tasks.

[0099] The experimental results were as follows: the prediction accuracy for all five types of scarred myocardium exceeded 92%, with the accuracy exceeding 98% in the myocarditis and normal subject groups; the average accuracy for scarred myocardium identification was 0.964, the average precision was 0.976, the average sensitivity and average specificity were 0.976 and 0.933 respectively, and the F1 score was 0.976; the average accuracy for scarred myocardium classification was 0.929, the average precision was 0.944, the sensitivity and specificity were 0.932 and 0.980 respectively, and the F1 score reached 0.937. The ROC results are as follows: Figure 7 As shown, the AUC value is 0.953. The commonly used metrics for measuring the accuracy of the classification model in the above statistics all exceed 0.9. The experimental results indicate that the method of this invention can effectively integrate and learn multimodal myocardial features to achieve the evaluation of scarred myocardial information, and accurately identify and classify scarred myocardial tissue.

[0100] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. An automatic identification and classification method for scarred myocardium based on cinema magnetic resonance imaging and deep learning, comprising the following steps: (1) Preprocess the cardiac cine magnetic resonance images of five types of scarred myocardium: hypertrophic cardiomyopathy with positive fibrosis, hypertrophic cardiomyopathy with negative fibrosis, myocardial infarction, myocarditis, and normal myocardium, to obtain left ventricular images with myocardial intima and endocardium annotations. (2) The left ventricular image is subjected to feature extraction to obtain the relative motion feature data and imaging feature image of the myocardium; the extraction process of the myocardial imaging feature image is as follows: firstly, the myocardial epithelial labeling information and the myocardial endothelial labeling information are subtracted from the myocardial epithelial labeling information in the left ventricular image to obtain the myocardial label, and then the result of the Hadamard product operation between the myocardial label and the left ventricular image is used as the imaging feature image of the myocardium. The process of extracting myocardial relative motion feature data is as follows: First, a rectangular coordinate system is established for the left ventricular image to obtain the coordinates of each point on the inner and outer boundaries of the myocardium. The geometric centroid of the endocardium is then calculated as the centroid point C(x) of the myocardium. c ,y c ); Then, starting from the x-axis as the initial angle, rays are drawn outward from the centroid C as the endpoint, with a 3° interval between adjacent rays, resulting in a total of 120 rays. The coordinates of the intersection points of each ray with the myocardium and the endocardium are recorded. For any given ray, calculate the deformation data of the intima relative to the center of mass, the deformation data of the epicondyle relative to the center of mass, the deformation data of the myocardium relative to the center of mass, the tangential deformation data of the intima, and the tangential deformation data of the epicondyle, where: The deformation data of the inner membrane relative to the center of mass includes the x-axis component ΔX. en (i) y-axis direction component ΔY en (i) and distance D en (i); ΔX en (i)=X enp (i)-x c ΔY en (i)=Y enp (i)-y c The deformation data of the outer membrane relative to the center of mass includes the x-axis component ΔX. ep (i) y-axis direction component ΔY ep (i) and distance D ep (i); ΔX ep (I)=X epp (i)-x c ΔY ep (i)=Y epp (i)-y c The myocardial relative centroid deformation data includes the x-axis component ΔX. pn (i) y-axis direction component ΔY pn (i) and distance D pn (i); ΔX pn (i)=X epp (i)-X enp (i) ΔY pn (i)=Y epp (i)-Y enp (i) The tangential deformation data of the intima includes the x-axis component ΔX. tgn (i) y-axis direction component ΔY tgn (i) and distance D tgn (i); ΔX tgn (i)=X enp (i+1)-X enp (i) ΔY tgn (i)=Y enp (i+1)-Y enp (i) The outer membrane tangential motion deformation data includes the x-axis component ΔX. tgp (i) y-axis direction component ΔY tgp (i) and distance D tgp (i); ΔX tgp (i)=X epp (i+1)-X epp (i) ΔY tgp (i)=Y epp (i+1)-Y epp (i) Where: X enp (i) and Y enp (i) are the x-axis and y-axis coordinates of the intersection point of ray i and the myocardial endothelium, respectively. epp (i) and Y epp (i) are the x-axis coordinates and y-axis coordinates of the intersection point of ray i and the epicardium, respectively, where i is the index number of the ray; Based on the above, the motion deformation data of each ray contains 15 feature values. Combining the motion deformation data of all rays yields a relative motion feature matrix with a dimension of 120×15, which serves as the relative motion feature data of the myocardium. (3) Perform data augmentation processing on the imaging features of the myocardium; (4) Obtain a large number of samples according to steps (1) to (3). Each set of samples includes the relative motion feature data of myocardium, the enhanced myocardial imaging feature image and the scar myocardium category label of the corresponding cardiac cine magnetic resonance image. Divide all samples into training set, validation set and test set. (5) Construct a multimodal feature fusion network model and train it using training set samples to obtain an automatic identification and classification model for scarred myocardium; the multimodal feature fusion network model captures and extracts the compensatory effect of distant scarred myocardium, the non-rigid deformation motion law of myocardium, and the spatiotemporal relationship between myocardial motion and morphology based on the relative motion feature data and imaging feature images of myocardium to achieve classification and identification, and obtains the confidence level of the input sample corresponding to the above five types of scarred myocardium; (6) Input the test set sample into the automatic identification and classification model of scar myocardium, and the confidence level of the sample corresponding to the above five types of scar myocardium can be output. Based on the confidence level, it can be determined whether there is scar myocardium and the type of scar myocardium in the cardiac cine magnetic resonance image corresponding to the test set sample.

2. The method for automatic identification and classification of scarred myocardium according to claim 1, characterized in that: The cardiac cine magnetic resonance images are image sequences obtained by performing cine magnetic resonance imaging on different layers of the heart in a short-axis view.

3. The method for automatic identification and classification of scarred myocardium according to claim 1, characterized in that: In step (1), the cardiac cine magnetic resonance image is preprocessed, including detecting and cropping the complete left ventricular region image and segmenting the left ventricular myocardium and epicardium.

4. The method for automatic identification and classification of scarred myocardium according to claim 1, characterized in that: The specific implementation method of step (3) is as follows: first, the imaging feature image of the myocardium is divided into blocks, and then the image blocks are randomly rotated, transposed and recombined into a new imaging feature image.

5. The method for automatic identification and classification of scarred myocardium according to claim 1, characterized in that: The multimodal feature fusion network model first segments the myocardial imaging feature images in the sample into multiple image patches, and then passes these image patches through a linear layer to obtain a set of image embeddings. Then, the image embeddings and the relative motion feature data of the myocardium in the sample are processed sequentially through a multilayer residual perceptron and an average pooling layer to obtain their respective feature vectors, which are then concatenated. The concatenated multimodal feature matrix is ​​then passed sequentially through a multilayer residual perceptron, an average pooling layer, and a linear classifier to output a confidence vector, which contains the confidence of the input sample for the five types of scarred myocardium.

6. The method for automatic identification and classification of scarred myocardium according to claim 5, characterized in that: The multilayer residual perceptron consists of 6 cascaded ResMLP modules. Each ResMLP module consists of a linear sub-layer applied across image blocks and a feedforward sub-layer applied across channels. The linear sub-layer and the feedforward sub-layer have two affine element transformations at the beginning and end, respectively, and are connected using residuals.

7. The method for automatic identification and classification of scarred myocardium according to claim 1, characterized in that: The specific process of training the multimodal feature fusion network model in step (5) is as follows: 5.1 Initialize model parameters, including the bias vector and weight matrix of each layer, learning rate, and optimizer; 5.2 Input the myocardial relative motion feature data and myocardial imaging feature images from the training set samples into the model. The model outputs the corresponding classification and recognition result, i.e., the confidence vector, through forward propagation. Calculate the cross-entropy loss function between the confidence vector and the category label in the sample. 5.3 Based on the loss function, the optimizer iteratively updates the model parameters using gradient descent until the loss function converges, and the training is complete; 5.4 After training is completed, the model is validated using the validation set samples. The model that performs best on the validation set is used as the final automatic identification and classification model for scarred myocardium.

Images