A method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network

Abstract

The present invention proposes a method for classifying myocardial magnetic resonance image fibrosis based on a convolutional neural network, which relates to the field of image processing technology. The original magnetic resonance image of the target to be measured is obtained from a magnetic resonance imaging device, and the original magnetic resonance image is preprocessed; based on the connectivity feature, the preprocessed magnetic resonance image is segmented into multiple sub-regions, and each sub-region represents an independent connected part in the image; a gray-scale measurement function is constructed for each sub-region to quantify the gray-scale distribution characteristics of each sub-region, and by comparing the gray-scale measurement function values of each sub-region, the sub-region with the smallest gray-scale measurement function value is identified as the fibrosis tissue sub-region; features are extracted from the identified fibrosis tissue sub-region as the input of the convolutional neural network model, and the fibrosis tissue sub-region is classified to output the fibrosis stage to which the fibrosis tissue sub-region belongs, improving the accuracy of classifying the fibrosis tissue.

Description

Technical Field

[0001] The present invention proposes a method for classifying myocardial fibrosis in magnetic resonance images based on a convolutional neural network, which relates to the technical field of myocardial magnetic resonance image processing. Background Art

[0002] Myocardial fibrosis generally refers to the scar tissue formed in the heart muscle tissue due to certain diseases (such as myocardial infarction or myocarditis). Accurately classifying these fibrotic regions is very important. Magnetic resonance imaging (MRI), especially cardiac MRI (CMR), is a commonly used non-invasive examination method that can provide high-resolution images. Traditionally, the late gadolinium enhancement (LGE) technique of cardiac magnetic resonance imaging (CMR) is the gold standard for identifying fibrosis, and the normal myocardium and fibrotic regions are distinguished by the contrast agent imaging. However, this method relies on doctors to manually delineate the lesion area, and there are the following limitations:

[0003] 1. There may be significant differences in the determination of fibrotic boundaries by different physicians, resulting in low diagnostic consistency; it takes dozens of minutes to manually analyze a single CMR image, which is difficult to meet the needs of large-scale screening.

[0004] 2. Early or focal fibrosis is easily overlooked by the naked eye due to low contrast and irregular morphology.

[0005] In recent years, automatic classification methods based on deep learning have gradually become a research hotspot. Especially, convolutional neural networks (CNNs) have demonstrated powerful feature extraction capabilities in medical image analysis. Existing work is mainly divided into two categories:

[0006] End-to-end classification model: directly perform binary classification (fibrosis / normal) on CMR images, but limited by the small amount of medical data, the model is prone to overfitting.

[0007] Segmentation and classification combined model: first locate the fibrotic region through a segmentation network such as U-Net, and then classify based on the segmentation result. However, segmentation depends on pixel-level annotation, which is costly, and the classification performance is restricted by the segmentation accuracy. Although there have been progresses, the existing technologies still have the following technical problems:

[0008] 1. Data scale and quality limitations: The amount of CMR image data is small, and the annotation depends on expert experience, resulting in an insufficient training set scale; the proportion of normal regions is much higher than that of fibrotic regions, and the model is prone to bias towards the majority class.

[0009] 2. Difficulty in extracting subtle features: The gray-scale difference between fibrotic regions and normal tissues is small, and the morphology is variable (such as dot-like and patchy). The shallow features of traditional CNNs are difficult to capture their complex textures; myocardial motion artifacts and MRI noise interfere with feature learning and reduce the robustness of the model.

[0010] 3. Insufficient utilization of multi-scale information: Existing models mostly use single-resolution feature maps, ignoring the scale diversity of fibrosis (e.g., small scars and large areas of necrosis require different receptive fields); parameter differences in CMR equipment in different hospitals (such as magnetic field strength, slice thickness) lead to domain shift problems, and models trained on a single dataset have poor generalization performance. Summary of the Invention

[0011] To solve the above technical problems, the present invention proposes a method for classifying myocardial magnetic resonance image fibrosis based on a convolutional neural network, including the following steps:

[0012] Obtain the original magnetic resonance image of the target to be measured from a magnetic resonance imaging device, and preprocess the original magnetic resonance image;

[0013] Based on connected features, segment the preprocessed magnetic resonance image into multiple sub-regions, and each sub-region represents an independent connected part in the image;

[0014] Construct a gray-scale measurement function for each sub-region, quantify the gray-scale distribution characteristics of each sub-region, and by comparing the gray-scale measurement function values of each sub-region, identify the sub-region with the smallest gray-scale measurement function value as the fibrotic tissue sub-region;

[0015] Extract features from the identified fibrotic tissue sub-region as the input of the convolutional neural network model, classify the fibrotic tissue sub-region, and output the fibrosis stage to which the fibrotic tissue sub-region belongs.

[0016] In a preferred embodiment, perform density estimation on the preprocessed magnetic resonance image, select the density peak point as the central pixel point, divide the preprocessed magnetic resonance image into regular grids according to a preset number of grids, combine the color distance and the spatial distance, calculate the first total distance between the edge pixel points of each grid and the current grid central pixel point, and then calculate the second total distance between the edge pixel points of each grid and the central pixel points of adjacent grids. If the second total distance is less than the first total distance, reassign the edge pixel point to the adjacent grid;

[0017] According to the reassignment result, update the grid attribution of each edge pixel point to generate an updated grid division result.

[0018] In a preferred embodiment, perform Gaussian kernel density estimation on the magnetic resonance image, and the specific formula is as follows:

[0019] The two-dimensional Gaussian kernel function is K(u,v):

[0020] ;

[0021] where h is the bandwidth parameter that controls the smoothness of the kernel, and u and v represent the coordinate offsets between pixel points;

[0022] The density estimation function f(x, y) at the pixel point (x, y) is as follows:

[0023] ;

[0024] where: (x IJ , y IJ ) represents the pixel coordinates within the neighborhood window centered at (x, y); Z(x IJ , y IJ ) is the intensity value of this pixel; n is the size of the neighborhood window (such as an n×n window), and I, J are the pixel points located in the I-th row and J-th column within the window.

[0025] In a preferred embodiment, a feature fusion process is used to fuse the extracted features G and Q. G represents the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region, and Q represents the proportion of the fibrotic tissue sub-region in the atrial wall. Suppose G has n1 feature vectors: G 1 , G 2 , …, G a , …, G n1 ; Suppose Q has n2 feature vectors: Q 1 , Q 2 , …, Q b , …, Q n2 ;

[0026] ;

[0027] where Concat is the concatenation function, F is the high-dimensional fused feature, G a is the a-th feature vector of G, W a is the weight coefficient of the a-th feature vector of G, Q b is the b-th feature vector of Q, and W b is the weight coefficient of the b-th feature vector of Q.

[0028] In a preferred embodiment, the fused feature F is input into a convolutional neural network model, and the minimum difference value V between the fused feature F of the fibrotic tissue sub-region and the reference fused feature values of each fibrotic stage is calculated through a minimization function, and the fibrotic stage t corresponding to the minimum difference value V is output. If the minimum difference value V is greater than the difference threshold, it is determined that this fibrotic tissue sub-region does not belong to any fibrotic stage.

[0029] In a preferred embodiment, the grid edge pixel points are determined based on coordinate comparison, specifically including:

[0030] Assume that each grid is a square with side length W. For a pixel point with coordinates (x, y) in the image, when 0 ≤ x mod W ≤ 2 or W - 3 ≤ x mod W < W - 1, and 0 ≤ y mod W ≤ 2 or W - 3 ≤ y mod W ≤ W - 1, then it is determined that the pixel point is on the grid edge, where mod represents the remainder operation.

[0031] In a preferred embodiment, the variance R of the color distribution within the i-th grid is calculated i , and the average value M of the total distances between all pixel points within the i-th grid and the central pixel point of the grid is calculated i , based on the variance R i and the average value M i the connectivity feature U of the i-th grid is calculated i ; By adjusting the variance weight parameter and the position of each grid central pixel point until the connectivity features generated by each grid are all optimal values, the final grid is segmented to obtain multiple sub-regions.

[0032] In a preferred embodiment, the process of identifying the fibrotic tissue sub-region includes: calculating the point weight value between the regional center point and the regional edge point of each sub-region :

[0033] ;

[0034] In the formula, I P represents the gray value of the regional edge point p in the C-th type of sub-region, and I C represents the gray value of the regional center point of the C-th type of sub-region;

[0035] Assume that the C-th type of sub-region has M regional edge points, and the adjacent regional edge points form a set J. The edge weight value B between the adjacent regional edge points p and q is calculated as follows: pq The calculation method is as follows:

[0036] ;

[0037] In the formula, I q represents the gray value of the regional edge point q, is the variance of the gray value of the entire sub-region;

[0038] Based on the point weight value T C and the edge weight value B pq the gray scale measurement function is obtained, and the gray scale measurement function E is expressed as:

[0039] ;

[0040] The sub-region with the minimum output value of the gray scale measurement function E is the fibrotic tissue sub-region.

[0041] In a preferred embodiment, it is assumed that cardiac fibrosis is divided into 4 stages. By analyzing the magnetic resonance image data of a large number of existing cardiac samples in different fibrosis stages, features are extracted and integrated to obtain the reference fusion feature values f 1 、f 2 、f 3 、f 4 :

[0042] f 1 represents the reference fusion feature value of the mild fibrosis stage;

[0043] f 2 represents the reference fusion feature value of the moderate fibrosis stage;

[0044] f 3 represents the reference fusion feature value of the moderate-severe fibrosis stage;

[0045] f 4 represents the reference fusion feature value of the severe fibrosis stage;

[0046] Obtain the magnetic resonance image of the target to be measured, extract the fusion feature F, calculate the fusion feature F with the reference fusion feature value of each stage respectively, and output the fibrosis stage with the smallest minimized difference value.

[0047] Compared with the prior art, the present invention has the following beneficial technical effects:

[0048] 1. Accurately identify fibrotic tissues: By constructing a gray-scale measurement function and comparing the gray-scale values of each sub-region, the fibrotic tissue sub-regions can be accurately identified; the gray-scale measurement function directly quantifies the tissue uniformity and highly coincides with the fibrotic characteristics; combining with a convolutional neural network (CNN) to classify the fibrotic tissue region can further determine whether it is a fibrotic tissue and its severity, thereby improving the accuracy of judgment.

[0049] 2. Through preprocessing steps such as denoising, image enhancement, and standardization, the quality of magnetic resonance images can be significantly improved, providing a clearer image basis for subsequent fibrosis detection.

[0050] 3. The fusion of spatial distance and color distance reduces the influence of uneven illumination or artifacts; 4. Using a convolutional neural network can extract quantitative image features from images in a high-throughput manner, including the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region, the proportion of the fibrotic tissue sub-region in the atrial wall, providing quantitative statistical indicators for the detection and classification of fibrosis.

[0051] 4. Through the collaborative design of precise preprocessing, two-stage region screening (connected segmentation → grayscale measurement), and multi-modal CNN classification, the present invention achieves high-precision, high-efficiency, and high-robustness myocardial fibrosis detection and grading. The technical effects are not only reflected in the improvement of algorithm performance, but also provide a reliable tool for subsequent analysis through automated, interpretable, and extensible designs. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0053] Figure 1 It is a schematic flowchart of the method for classifying myocardial fibrosis in cardiac magnetic resonance images based on a convolutional neural network of the present invention;

[0054] Figure 2 It is a schematic diagram of the results of the system for classifying myocardial fibrosis in cardiac magnetic resonance images based on a convolutional neural network of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0056] Embodiment 1

[0057] The method for classifying myocardial fibrosis in cardiac magnetic resonance images based on a convolutional neural network of the present invention realizes the efficient detection and grading of myocardial fibrosis through a three-level architecture of precise segmentation, grayscale feature screening, and CNN deep classification. By segmenting connected features and coarsely screening candidate regions using a grayscale measurement function, the computational burden of the CNN is reduced; the end-to-end automated process requires no manual intervention from the original cardiac magnetic resonance image to the classification result, and is suitable for large-scale screening.

[0058] As Figure 1 shown, it is a schematic flowchart of the method for classifying myocardial fibrosis in cardiac magnetic resonance images based on a convolutional neural network of the present invention. The method for classifying myocardial fibrosis in cardiac magnetic resonance images based on a convolutional neural network includes the following steps:

[0059] Step 1: Obtain the original magnetic resonance image of the target to be measured from a magnetic resonance imaging device and preprocess the original magnetic resonance image.

[0060] Use professional medical image processing software or related libraries to read myocardial magnetic resonance images. These myocardial magnetic resonance images are stored in DICOM (Digital Imaging and Communications in Medicine) format.

[0061] Convert the read DICOM images into a format suitable for subsequent processing. Preferably, use image formats such as PNG, JPEG, etc. According to the noise characteristics and processing requirements of the images, select linear filtering methods such as mean filtering, median filtering, Gaussian filtering, and non-linear filtering methods such as non-local means filtering, wavelet denoising, etc.

[0062] For the selected denoising algorithm, adjust the corresponding parameters to achieve the best denoising effect. For Gaussian filtering, it is necessary to determine the Gaussian kernel size and standard deviation; for non-local means filtering, it is necessary to set parameters such as the search window size and similarity threshold.

[0063] By adjusting the contrast of the image, the details in the image can be made more clearly visible. Preferably, use histogram equalization and adaptive histogram equalization. Histogram equalization can equalize the gray histogram of the image and expand the gray dynamic range of the image; adaptive histogram equalization is to perform histogram equalization in a local area, which can better preserve the local details of the image.

[0064] According to the actual situation of the image, appropriately adjust the brightness of the image. The brightness adjustment is achieved by performing linear transformation or non-linear transformation on the gray values of the image. Preferably, multiply the gray values of the image by a constant or add an offset to increase or decrease the overall brightness of the image.

[0065] After the above preprocessing steps, myocardial magnetic resonance images with higher quality and more suitable for fibrosis classification based on convolutional neural networks can be obtained.

[0066] Step 2: Based on the connectivity feature, segment the preprocessed magnetic resonance image into multiple sub-regions, and each sub-region represents an independent connected part in the image.

[0067] Perform density estimation on the preprocessed magnetic resonance image, and determine the desired number of grids according to the overall density distribution of the magnetic resonance image and the size of the magnetic resonance image.

[0068] First, perform density estimation on the preprocessed magnetic resonance image. Preferably, use the Gaussian kernel density estimation method to calculate the density values of the surrounding pixels at each pixel point of the magnetic resonance image, and understand the density changes in different regions of the magnetic resonance image by analyzing the overall density distribution of the image.

[0069] For the Gaussian kernel density estimation of the magnetic resonance image, the specific formula is as follows:

[0070] Two-dimensional Gaussian kernel function K(u, v):

[0071] ;

[0072] where h is the bandwidth parameter that controls the smoothness of the kernel, and u and v represent the coordinate offsets between pixel points.

[0073] The density estimation function f(x, y) at the pixel point (x, y) is:

[0074] ;

[0075] where: (x IJ , y IJ ) represents the pixel coordinates within the neighborhood window centered at (x, y); Z(x IJ , y IJ ) is the intensity value of this pixel; n is the size of the neighborhood window (such as an n×n window), and I, J are the pixel points located in the I-th row and J-th column within the window.

[0076] The normalization factor 1 / n ensures that the probability density integral is 1.

[0077] From the density distribution, select representative pixel points as the centers of the initial grid. Preferably, select the density peak points as the initial grid centers, and the density peak points represent the cores of different feature regions in the magnetic resonance image.

[0078] Determine the number of grids and the initial division: Based on the selected initial grid centers, according to the spatial distribution of the magnetic resonance image and the number of grids, each grid consists of the pixel points closest to its center, which can ensure that each grid has a certain degree of independence and uniformity in space.

[0079] Set the initial grid and the grid center pixel points.

[0080] According to the size of the magnetic resonance image and the desired number of grids N, calculate the size of each grid;

[0081] The size of the grid C = image area S / number of grids N.

[0082] A grid center pixel point is initially selected on each grid of the magnetic resonance image, and care should be taken to avoid the grid center pixel point falling on the edge or noise of the grid.

[0083] For example, each grid cell can be 10×10 pixels; when calculating the number of grids, the size of the grid size C is calculated according to the total size (width and height) of the magnetic resonance image and the number of grids.

[0084] In a preferred embodiment, after initially determining the grid center pixel point, a local fine-tuning is performed on it. Check whether there are more suitable pixel points with less noise influence within a certain range around the grid center pixel point. If so, adjust the grid center to that point.

[0085] After completing the grid division and determining the center pixel point, the results are verified multiple times. Some statistical indicators can be used, such as calculating the degree of difference between the gray value of the grid center pixel point and the gray values of the surrounding pixel points, to determine whether the grid center is likely to fall on noise. If problems are found, re-perform the grid division or adjust the center pixel point.

[0086] Combining the color distance and spatial distance of each grid, calculate the total distance between each pixel point in the grid and the grid center pixel point.

[0087] Color distance d C is the distance d between two pixel points in the color space (such as CIELAB) C .

[0088] Color distance d C is calculated as follows in the CIELAB color space:

[0089] ;

[0090] where: L 1 , a 1 , b 1 are the CIELAB color space coordinates of the first pixel point; L 2 , a 2 , b 2 are the CIELAB color space coordinates of the second pixel point.

[0091] Spatial distance d S is the distance between two pixel points in the image plane. The spatial distance d S is calculated as follows:

[0092] ;

[0093] where: X 1 , Y 1 are the coordinates of the first pixel point in the image plane; X2 , Y 2 is the coordinate of the second pixel on the image plane.

[0094] Using a weighted sum method to combine the color distance and the spatial distance, calculate the total distance D between each pixel in the grid and the central pixel of the grid, which is used to measure the similarity or difference between pixels.

[0095] The calculation formula for the total distance D between each pixel and the central pixel of the grid is as follows:

[0096] ;

[0097] where d C is the color distance between the pixel in the grid and the central pixel of the grid, d S is the spatial distance between the pixel in the grid and the central pixel of the grid, W is the size of the grid, and m is a parameter used to control the relative importance of the color and spatial distances.

[0098] Calculate the total distance between each pixel on the grid edge in the magnetic resonance image and the neighboring central pixel of the grid. If the total distance from the neighboring central pixel of the grid is less than the total distance from the current central pixel of the grid, then assign the pixel on the grid edge in the magnetic resonance image to the neighboring grid. According to the reallocation result, update the grid attribution of each edge pixel to generate an updated grid division result.

[0099] First, determine the grid edge pixels. Traverse each pixel in the image. For each pixel, check the grid to which its neighboring pixels belong. If there are pixels belonging to different grids among the neighboring pixels, then the current pixel is located on the grid edge.

[0100] Secondly, use different colors or labels to visualize the pixels in each grid to view the assignment result. Through the above steps, the distance between the grid edge pixels in the magnetic resonance image and the neighboring grid center can be calculated, and the pixels can be reallocated according to the distance. This method can optimize the image segmentation result and make the edge pixels more reasonably belong to the neighboring region.

[0101] In a preferred embodiment, the grid edge pixels are determined based on coordinate comparison, specifically as follows:

[0102] In this embodiment, the grids are regularly divided, and each grid is a square with a side length of W. For a pixel with coordinates (x, y) in the image, if the remainder of its x coordinate divided by W is close to 0 or W - 1, and the remainder of its y coordinate divided by W is close to 0 or W - 1, then the pixel may be located on the grid edge.

[0103] For example, when 0 ≤ x mod W ≤ 2 or W - 3 ≤ x mod W < W - 1, and 0 ≤ y mod W ≤ 2 or W - 3 ≤ y mod W ≤ W - 1, it is determined that the pixel is on the grid edge, where mod represents the modulo operation.

[0104] Calculate the connectivity features of each grid, and based on the optimal values of the connectivity features, implement the segmentation method that conforms to the final grid to obtain multiple sub-regions.

[0105] For the i-th grid, first, calculate the variance R of the color distribution within the i-th grid i . A high variance indicates a large color diversity in the grid, that is, a high uniqueness of the grid.

[0106] Secondly, calculate the average value M of the total distance between all pixel points within the i-th grid and the central pixel point of the grid i . The smaller the average distance, the higher the compactness of the grid.

[0107] Combine R i and M i by weighting to form the connectivity feature U i :

[0108] ;

[0109] where is the variance weight parameter, which is used to adjust the relative importance between uniqueness and compactness.

[0110] Finally, by adjusting the variance weight parameter and the position of the initial central pixel point of the grid until the connectivity feature U i generated by each grid is the optimal value, the final grid segmentation method is realized to obtain multiple sub-regions.

[0111] In practical applications, the atrial wall mainly has the following four sub-regions: blood pool, normal myocardium, pericardial fluid, and fibrotic tissue. The identification of the fibrotic tissue sub-region is further carried out in Step 3 for these four sub-regions.

[0112] Step 3: Construct a gray-scale measurement function for each sub-region, quantify the gray-scale distribution characteristics of each sub-region, and by comparing the gray-scale measurement function values of each sub-region, identify the sub-region with the smallest gray-scale measurement function value as the fibrotic tissue sub-region.

[0113] Regarding the identification of the fibrotic tissue sub-region as a problem of assigning labels to each pixel point p within the four sub-regions.

[0114] Specifically, the number of labels is set to four, corresponding to the above four types of regions (blood pool, normal myocardium, pericardial fluid, and fibrotic tissue). The sub-region with the minimum value of the gray-scale measurement function E after the final segmentation is taken as the fibrotic tissue sub-region, that is, the fibrotic tissue.

[0115] Calculate the point weight value between the regional center point and the regional edge point of each sub-region , calculated by the following formula:

[0116] ;

[0117] In the formula, I P represents the gray value of each regional edge point p in the image, and I C represents the gray value of the regional center point of the C-th type of sub-region. There are M regional edge points.

[0118] The adjacent regional edge points form a set J, and the edge weight value B pq between the adjacent regional edge points p and q is calculated as follows:

[0119] ;

[0120] In the formula, I q represents the gray value of the regional edge point q, and is the variance of the entire sub-region.

[0121] Based on the point weight value T C and the edge weight value B pq a gray-scale measurement function is obtained, and the gray-scale measurement function E is expressed as:

[0122] ;

[0123] The sub-region with the minimum output value of the gray-scale measurement function E is the fibrotic tissue sub-region, and this fibrotic tissue sub-region is extracted.

[0124] In a preferred embodiment, since the real atrial fibrous layer is generally thin, only accounting for 2-3 pixels, the single-threshold method has an unsatisfactory classification effect on images with large gray-scale differences in the fibrotic region. Therefore, the present invention uses a feature fusion process to fuse different extracted features.

[0125] Step Four: Extract features from the identified fibrotic tissue sub-region as the input of the convolutional neural network model, classify the fibrotic tissue sub-region, and output the fibrotic stage to which the fibrotic tissue sub-region belongs.

[0126] Since the real atrial fibrous layer is generally thin, only accounting for 2-3 pixels, the single-threshold method has an unsatisfactory classification effect on images with large gray-scale differences in the fibrotic region. Therefore, the present invention uses a feature fusion process to fuse different extracted features.

[0127] Feature fusion can integrate features from different sources, and a convolutional neural network model is used to classify the fibrosis of the extracted fibrotic tissue region.

[0128] The features extracted in this embodiment include: the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region and the proportion of the fibrotic tissue in the atrial wall.

[0129] Among them, in the proportion of the fibrotic tissue in the atrial wall, the numerator is the number of pixel points of the fibrotic tissue, and the denominator is the number of pixel points of the atrial wall.

[0130] The feature fusion process is used to fuse the different extracted features, and the specific fusion process can be described by the following formula:

[0131] F = Concat(G, Q);

[0132] Among them, G represents the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region, Q represents the proportion of the fibrotic tissue sub-region in the atrial wall, and F represents the fused feature generated by fusion.

[0133] It should be noted that for the fibrotic tissue sub-region with a determined boundary, calculating its longest diameter and shortest diameter can be achieved by geometric analysis of the points on the contour, calculating the distance between any two points on the contour, traversing all possible point pairs, finding the maximum value of the distance as the longest diameter, the minimum value as the shortest diameter, and then calculating the ratio of the longest diameter to the shortest diameter to obtain feature G.

[0134] After completing the segmentation of the atrial wall region, the total number of pixel points contained in the atrial wall is counted, which is used as the total area of the atrial wall. Calculate the area of the fibrotic tissue sub-region: count the number of pixel points within the fibrotic tissue sub-region as the area of this sub-region.

[0135] Calculate the proportion: Divide the area of the fibrotic tissue sub-region by the total area of the atrial wall to obtain the proportion of the fibrotic tissue sub-region in the atrial wall, that is, feature Q.

[0136] The Fuse function is a fusion function that combines multiple features in different ways to improve the comprehensiveness of data processing. The Fuse function includes: weighted summation or concatenation.

[0137] Weighted summation assigns different weights to different data sources or features, and then performs weighted summation to obtain a comprehensive output.

[0138] Concatenation is to directly connect multiple feature vectors together to form a new high-dimensional feature vector.

[0139] Input features:

[0140] Let G have n1 feature vectors: G1 , G 2 , …, G a , …, G n1 ;

[0141] Suppose Q has n2 eigenvectors: Q 1 , Q 2 , …, Q b , …, Q n2 ;

[0142] Through the method of weighted summation and concatenation, first perform weighted summation on multiple eigenvectors of each feature, and then concatenate multiple features together to obtain the fused feature F.

[0143] ;

[0144] Concat is a concatenation function, that is, directly connect multiple eigenvectors together to form a new high-dimensional eigenvector. F is the high-dimensional fused feature, G a is the ath eigenvector of G, W a is the weight coefficient of the ath eigenvector of G, Q b is the bth eigenvector of Q, W b is the weight coefficient of the bth eigenvector of Q.

[0145] Input the fused feature F into the convolutional neural network model. The convolutional neural network model classifies the sub-regions of fibrotic tissue and outputs the fibrotic stage to which the sub-regions of fibrotic tissue belong.

[0146] Specifically, the convolutional neural network model first calculates the minimized difference value between the fused feature and the reference fused feature values of each stage through the minimization function.

[0147] The minimization function is , and the output minimized difference value V is:

[0148] ; t = 1, 2 …… T;

[0149] where, f t is the reference fused feature value of the tth stage of fibrosis, T is the total number of fibrosis stages, and the total number of stages and the reference fused feature values of each stage are selected according to actual needs.

[0150] For the scenario of the convolutional neural network model diagnosing cardiac fibrosis in medical images, assume that cardiac fibrosis is divided into 4 stages (T = 4). By analyzing the magnetic resonance image data of a large number of cardiac samples with different diagnosed fibrosis stages, features are extracted and integrated to obtain the reference fused feature value f t :

[0151] f1 =[0.23, 0.45], representing the reference fusion eigenvalue for the mild fibrosis stage (stage 1),

[0152] f 2 =[0.35, 0.62], corresponding to the reference fusion eigenvalue for the moderate fibrosis stage (stage 2);

[0153] f 3 =[0.48, 0.78], indicating the reference fusion eigenvalue for the moderate - severe fibrosis stage (stage 3);

[0154] f 4 =[0.60, 0.95], being the reference fusion eigenvalue for the severe fibrosis stage (stage 4);

[0155] Finally, the same magnetic resonance imaging examination is performed on the target to be measured, and the fusion feature F = [0.46, 0.75] is extracted. The fusion feature F is respectively input into the trained convolutional neural network model. By calculating the fusion feature F with the reference fusion eigenvalues of each stage, the fibrosis stage t with the minimum minimized difference value is output.

[0156] Set a difference threshold. If the minimized difference value V is greater than the difference threshold, it is determined that the fibrotic tissue does not conform to any fibrosis stage.

[0157] In an alternative embodiment, the reference fusion eigenvalues of each stage can be replaced by the reference fusion eigenvalues of each fibrotic form.

[0158] In this embodiment, the fibrotic images are divided into 3 different forms, namely: reactive interstitial fibrosis, reparative fibrosis, and perivascular fibrosis.

[0159] Reactive interstitial fibrosis is caused by persistent long - term excessive pressure load (such as hypertension, obstructive hypertrophic cardiomyopathy, etc.), inflammatory reactions, and age - dependent increase, etc. Its characteristic is the excessive deposition of extracellular matrix (without accompanied by myocardial cell reduction).

[0160] Reparative fibrosis is the result of the repair of myocardial tissue after myocardial infarction. Scar tissue produced by fibroblasts replaces the necrotic myocardial tissue, which is mostly seen at the insertion points of the left and right ventricles of the interventricular septum and can also be found in the middle layer of the myocardium.

[0161] In the form of perivascular fibrosis, the dilation of the adventitia of microvessels is its characteristic, reflecting the transformation of fibroblasts or the fibrotic activation of vascular endothelial and wall cells, leading to the disruption of coronary recanalization and causing microvascular dysfunction, thus leading to diastolic dysfunction and aggravating the severity of ischemia.

[0162] Example 2

[0163] This embodiment proposes a classification system for implementing the myocardial magnetic resonance image fibrosis classification method based on a convolutional neural network in Embodiment 1. The structural schematic diagram of this classification system is as shown in Figure 2 the following figure.

[0164] This classification system includes: a data acquisition module, an image preprocessing module, an image segmentation module, a fibrotic tissue detection module, a feature extraction module, and a classification module. Each module works collaboratively to complete the entire process from the original magnetic resonance image to the classification of fibrotic tissue.

[0165] Data acquisition module: It is used to acquire the original magnetic resonance image of the target to be measured from a magnetic resonance imaging device.

[0166] Specifically, connect to the magnetic resonance imaging device through a network interface or a data storage device, and use a medical image reading library (such as SimpleITK) to read the original magnetic resonance image data.

[0167] Image preprocessing module: It is used to preprocess the original magnetic resonance image to improve the quality of the magnetic resonance image and the accuracy of subsequent processing.

[0168] Image segmentation module: It is used to segment the preprocessed magnetic resonance image into multiple sub-regions based on connectivity features, and each sub-region represents an independent connected part in the image.

[0169] The image segmentation module includes: an initialization unit, a distance calculation unit, a pixel assignment unit, and a connectivity feature calculation unit.

[0170] Initialization unit: According to the image size and the desired number of grids, calculate the size of each grid, and select a grid center pixel on each grid.

[0171] Distance calculation unit: Combine the color distance and the spatial distance to calculate the first total distance between the edge pixels of each grid and the center point of the current grid, and calculate the second total distance between the edge pixels of each grid and the center points of adjacent grids.

[0172] Pixel assignment unit: Judge the relationship between the second total distance and the first total distance. If the second total distance is less than the first total distance, reassign this edge pixel to the adjacent grid.

[0173] Connectivity feature calculation unit: Calculate the connectivity feature of each grid, and adjust the variance weight parameter and the position of the center pixel of each grid until the connectivity feature generated by each grid reaches the optimal value, so as to realize the segmentation of the final grid and obtain multiple sub-regions.

[0174] Fibrosis tissue detection module: Construct a grayscale measurement function for each sub-region, quantify the grayscale distribution characteristics of each sub-region, and identify the sub-region with the smallest grayscale measurement function value as the fibrosis tissue sub-region by comparing the grayscale measurement function values of each sub-region.

[0175] The fibrosis tissue detection module includes: a grayscale measurement function construction unit and a sub-region comparison unit.

[0176] Grayscale measurement function construction unit: Construct a grayscale measurement function for each sub-region to quantify the grayscale distribution characteristics of each sub-region.

[0177] Sub-region comparison unit: Calculate the grayscale measurement function value of each sub-region, and find the sub-region with the smallest grayscale measurement function value as the fibrosis tissue sub-region by comparing the grayscale measurement function values of each sub-region.

[0178] Feature extraction and classification module: Used to extract features from the identified fibrosis tissue sub-region as the input of the convolutional neural network model.

[0179] Classification module: Used to classify the fibrosis tissue region using the convolutional neural network model and output the fibrosis stage to which the fibrosis tissue sub-region belongs.

[0180] In a preferred embodiment, the classification module includes: a model construction unit, a model training unit, and a classification prediction unit.

[0181] Model construction unit: Construct a convolutional neural network model, including a convolutional layer, a pooling layer, a fully connected layer, etc. Preferably, use a deep learning framework (such as TensorFlow, PyTorch) to construct the model.

[0182] Model training unit: Use the labeled training data to train the convolutional neural network model and adjust the parameters of the convolutional neural network model to improve the classification accuracy.

[0183] Classification prediction unit: Input the extracted features into the trained convolutional neural network model for classification prediction to obtain the fibrosis stage to which the fibrosis tissue sub-region belongs.

[0184] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or a data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0185] As described above, the foregoing is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for classifying myocardial magnetic resonance image fibrosis based on a convolutional neural network, characterized in that: The steps are as follows: Obtain the original magnetic resonance image of the target to be measured from a magnetic resonance imaging device, and preprocess the original magnetic resonance image; perform Gaussian kernel density estimation on the magnetic resonance image, and the two-dimensional Gaussian kernel function is K(u, v): ； where h is the bandwidth parameter that controls the smoothness of the kernel, and u and v represent the coordinate offsets between pixel points; the density estimation function f(x, y) at pixel point (x, y) is: ； Where: (x IJ ,y IJ ) represents the pixel coordinates in the neighborhood window centered at (x, y); n is the size of the neighborhood window; Z(x IJ ,y IJ ) is the intensity value of the pixel located at the Ith row and the Jth column in the window; Select the density peak point as the central pixel point, divide the preprocessed magnetic resonance image into regular grids according to the preset number of grids, combine the color distance and the spatial distance, calculate the first total distance between the edge pixel points of each grid and the central pixel point of the current grid, and then calculate the second total distance between the edge pixel points of each grid and the central pixel point of the adjacent grid. If the second total distance is less than the first total distance, reassign the edge pixel point to the adjacent grid; According to the reassignment result, update the grid attribution of each edge pixel point to generate an updated grid division result; Determine the grid edge pixel points based on coordinate comparison. Assume that each grid is a square with side length W. For a pixel point with coordinates (x, y) in the image, when 0 ≤ x mod W ≤ 2 or W - 3 ≤ x mod W < W - 1, and 0 ≤ y mod W ≤ 2 or W - 3 ≤ y mod W ≤ W - 1, then it is determined that the pixel point is on the grid edge, where mod represents the remainder operation; Calculate the variance R of the color distribution in the i-th grid i , calculate the average value M of the total distance between all pixels in the i-th grid and the center pixel of the grid i , based on the variance R i and the mean value M i Calculate the connectivity feature U of the i-th grid i : ； By adjusting the variance weight parameter and the position of the central pixel of each grid, until the connectivity features generated by each grid are all optimal values, and the final grid segmentation is achieved, and the preprocessed magnetic resonance image is segmented into multiple sub-regions, each sub-region represents an independent connected part in the image; Construct a gray scale measurement function for each sub-region to quantify the gray scale distribution characteristics of each sub-region. By comparing the gray scale measurement function values of each sub-region, identify the sub-region with the smallest gray scale measurement function value as the fibrotic tissue sub-region; Extract features from the identified fibrotic tissue sub-region as the input of the convolutional neural network model, classify the fibrotic tissue sub-region, and output the fibrotic stage to which the fibrotic tissue sub-region belongs.

2. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 1, characterized in that: Let G represent the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region, Q represent the proportion of the fibrotic tissue sub-region in the atrial wall, and let G have n1 eigenvectors: G1, G2, …, G a ,…,G n1 ; Assume Q has n2 eigenvectors: Q1, Q2, …, Q b ,…,Q n2 ; Use the feature fusion process to fuse the extracted features G and Q: ； Among them, Concat is the concatenation function, F is the high-dimensional fusion feature, G a is the ath eigenvector of G, W a is the weight coefficient of the ath eigenvector of G, Q b is the bth eigenvector of Q, W b is the weight coefficient of the bth eigenvector of Q.

3. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 2, characterized in that: Input the fusion feature F into the convolutional neural network model, calculate the minimum difference value V between the fusion feature F of the fibrotic tissue sub-region and the reference fusion feature value of each fibrotic stage through the minimization function, and output the fibrotic stage t corresponding to the minimum difference value V. If the minimum difference value V is greater than the difference threshold, it is determined that the fibrotic tissue sub-region does not belong to any fibrotic stage.

4. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 1, characterized in that: The process of identifying fibrotic tissue subregions includes: calculating the point weights between the region center point and the region edge point of each subregion : ； In the formula, I P represents the gray value of the edge point p in the C-th sub-region, I C Represents the gray value of the center point of the C-th sub-region; Assume that there are M regional edge points in the C-th sub-region, the adjacent regional edge points constitute a set J, and the edge weight between adjacent regional edge points p and q is B pq The calculation method is as follows: ； In the formula, I q Represents the gray value of the region edge point q, is the gray value variance of the entire sub-region; Based on the point weight T C and edge weight B pq The grayscale measurement function E is obtained as follows: ； The sub-region with the smallest output value of the gray scale measurement function E is the fibrotic tissue sub-region.

5. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 3, characterized in that: Assume that cardiac fibrosis is divided into 4 stages. By analyzing the magnetic resonance image data of a large number of existing cardiac samples with different fibrotic stages, extract and integrate features to obtain the reference fusion feature values f1, f2, f3, f4 of each fibrotic stage: f1 represents the reference fusion feature value of the mild fibrotic stage; f2 represents the reference fusion feature value of the moderate fibrotic stage; f3 represents the reference fusion feature value of the moderately severe fibrotic stage; f4 represents the reference fusion feature value of the severe fibrotic stage; Obtain the magnetic resonance image of the target to be measured, extract the fusion feature F, calculate the fusion feature F with the reference fusion feature value of each stage respectively, and output the fibrotic stage with the smallest minimum difference value.

Images