Cardiac muscle magnetic resonance image fibrosis classification method based on convolutional neural network

Abstract

The invention proposes a myocardial magnetic resonance image fibrosis classification method based on a convolutional neural network, and relates to the technical field of image processing, and the method comprises the steps: obtaining an original magnetic resonance image of a to-be-detected target from a magnetic resonance imaging device, and carrying out the preprocessing of the original magnetic resonance image; segmenting the preprocessed magnetic resonance image into a plurality of sub-regions based on the connected features, each sub-region representing an independent connected part in the image; constructing a gray scale measurement function for each sub-region, quantifying the gray scale distribution characteristics of each sub-region, and identifying the sub-region with the minimum gray scale measurement function value as a fibrosis tissue sub-region by comparing the gray scale measurement function values of the sub-regions; features are extracted from the identified fibrosis tissue sub-regions to serve as input of the convolutional neural network model, the fibrosis tissue sub-regions are classified, the fibrosis stages to which the fibrosis tissue sub-regions belong are output, and the accuracy of classification of the fibrosis tissue is improved.

Description

Technical Field

[0001] The invention proposes a myocardial magnetic resonance image fibrosis classification method based on a convolutional neural network, and relates to the technical field of myocardial magnetic resonance image processing. Background Art

[0002] Myocardial fibrosis generally refers to scar tissue formed in cardiac muscle tissue due to certain diseases (such as myocardial infarction or myocarditis), and it is very important to accurately classify these fibrotic areas. Magnetic resonance imaging (MRI), especially cardiac MRI (CMR), is a commonly used non-invasive examination method that can provide high-resolution images. Traditionally, delayed gadolinium enhancement (LGE) technology of cardiac magnetic resonance imaging (CMR) is the gold standard for identifying fibrosis, and contrast agents are used to distinguish normal myocardium from fibrotic areas. However, this method relies on doctors to manually outline the lesion area, which has the following limitations: 1. Different physicians may have significant differences in their judgment of fibrosis boundaries, resulting in low diagnostic consistency; manual analysis of a single CMR image takes tens of minutes, which is difficult to meet large-scale screening needs.

[0003] 2. Early or focal fibrosis can be easily overlooked by the naked eye due to its low contrast and irregular shape.

[0004] In recent years, automatic classification methods based on deep learning have gradually become a research hotspot. In particular, convolutional neural networks (CNNs) have demonstrated powerful feature extraction capabilities in medical image analysis. Existing work can be divided into two categories: End-to-end classification model: CMR images are directly classified into two categories (fibrosis / normal). However, due to the small amount of medical data, the model is prone to overfitting.

[0005] Segmentation and classification joint model: First, locate the fibrosis area through segmentation networks such as U-Net, and then classify based on the segmentation results. However, segmentation relies on pixel-level annotation, which is costly, and classification performance is constrained by segmentation accuracy. Despite the progress, the existing technology still has the following technical problems: 1. Data size and quality limitations: The amount of CMR image data is small, and labeling relies on expert experience, resulting in insufficient training set size; the proportion of normal areas is much higher than that of fibrosis areas, and the model tends to be biased towards the majority class.

[0006] 2. Difficulty in extracting subtle features: The grayscale difference between fibrosis areas and normal tissues is small, and the morphology is variable (such as dots and sheets). The shallow features of traditional CNNs are difficult to capture their complex textures; myocardial motion artifacts and MRI noise interfere with feature learning and reduce the robustness of the model.

[0007] 3. Insufficient utilization of multi-scale information: Existing models mostly use single-resolution feature maps, ignoring the scale diversity of fibrosis (e.g., small-scale scars and large-scale necrosis require different receptive fields). The parameter differences of CMR equipment in different hospitals (e.g., magnetic field strength, layer thickness) lead to domain shift problems, and the generalization performance of models trained with a single dataset is poor. Summary of the invention

[0008] In order to solve the above technical problems, the present invention proposes a myocardial magnetic resonance image fibrosis classification method based on a convolutional neural network, comprising the following steps: Acquire an original magnetic resonance image of the target to be measured from a magnetic resonance imaging device, and preprocess the original magnetic resonance image; Based on the connectivity features, the preprocessed magnetic resonance image is segmented into multiple sub-regions, each of which represents an independent connected part of the image; A grayscale measurement function is constructed for each sub-region to quantify the grayscale distribution characteristics of each sub-region. By comparing the grayscale measurement function values ​​of each sub-region, the sub-region with the smallest grayscale measurement function value is identified as the fibrotic tissue sub-region; Features are extracted from the identified fibrotic tissue sub-regions and used as input to the convolutional neural network model to classify the fibrotic tissue sub-regions and output the fibrosis stage to which the fibrotic tissue sub-regions belong.

[0009] In a preferred embodiment, density estimation is performed on the preprocessed magnetic resonance image, and a density peak point is selected as a central pixel point. The preprocessed magnetic resonance image is divided into regular grids according to a preset number of grids. The first total distance between the edge pixel point of each grid and the central pixel point of the current grid is calculated by combining the color distance and the spatial distance. Then, the second total distance between the edge pixel point of each grid and the central pixel point of the adjacent grid is calculated. If the second total distance is less than the first total distance, the edge pixel point is reallocated to the adjacent grid. According to the result of the reallocation, the grid affiliation of each edge pixel is updated to generate an updated grid division result.

[0010] In a preferred embodiment, Gaussian kernel density estimation is performed on the magnetic resonance image, and the specific formula is as follows: The two-dimensional Gaussian kernel function is K(u,v): ; Among them, h is the bandwidth parameter, which controls the smoothness of the kernel, and u and v represent the coordinate offset between pixels; The density estimation function f(x,y) at the pixel point (x,y) is: ; Where: (x IJ ,y IJ) represents the pixel coordinates within the neighborhood window centered at (x, y); Z(x IJ , y IJ ) is the intensity value of this pixel; n is the size of the neighborhood window (such as an n×n window), and I, J are the pixel points located in the I-th row and J-th column within the window.

[0011] In a preferred embodiment, a feature fusion process is used to fuse the extracted features G and Q. G represents the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region, and Q represents the proportion of the fibrotic tissue sub-region in the atrial wall. Suppose G has n1 feature vectors: G1, G2, …, G a , …, G n1 ; suppose Q has n2 feature vectors: Q1, Q2, …, Q b , …, Q n2 ; ; where Concat is the concatenation function, F is the high-dimensional fused feature, G a is the a-th feature vector of G, W a is the weight coefficient of the a-th feature vector of G, Q b is the b-th feature vector of Q, and W b is the weight coefficient of the b-th feature vector of Q.

[0012] In a preferred embodiment, the fused feature F is input into a convolutional neural network model, and the minimum difference value V between the fused feature F of the fibrotic tissue sub-region and the reference fused feature values of each fibrotic stage is calculated through a minimization function, and the fibrotic stage t corresponding to the minimum difference value V is output. If the minimum difference value V is greater than the difference threshold, it is determined that the fibrotic tissue sub-region does not belong to any fibrotic stage.

[0013] In a preferred embodiment, grid edge pixel points are determined based on coordinate comparison, specifically including: Suppose each grid is a square with side length W. For a pixel point with coordinates (x, y) in the image, when 0 ≤ x mod W ≤ 2 or W - 3 ≤ x mod W < W - 1, and 0 ≤ y mod W ≤ 2 or W - 3 ≤ y mod W ≤ W - 1, then it is determined that this pixel point is on the grid edge, where mod represents the remainder operation.

[0014] In a preferred embodiment, the variance R of the color distribution within the i-th grid is calculated i , the average value M of the total distance between all pixel points within the i-th grid and the center pixel point of the grid is calculated i , and the connectivity feature U of the i-th grid is calculated based on the variance R i and the average value M i i ​; By adjusting the variance weight parameters and the position of each grid center pixel, until the connectivity features generated by each grid are the optimal values, the final grid segmentation is achieved to obtain multiple sub-regions.

[0015] In a preferred embodiment, the process of identifying fibrotic tissue sub-regions includes: calculating the point weight between the region center point and the region edge point of each sub-region : ; In the formula, I P represents the gray value of the regional edge point p in the C-th sub-region, I C Represents the gray value of the center point of the C-th sub-region; Assume that there are M regional edge points in the C-th sub-region, the adjacent regional edge points constitute a set J, and the edge weight between adjacent regional edge points p and q is B pq The calculation method is as follows: ; In the formula, I q Represents the gray value of the region edge point q, is the gray value variance of the entire sub-region; Based on the point weight T C and edge weight B pq The grayscale measurement function E is obtained as follows: ; The sub-region with the smallest output value of the grayscale measurement function E is the fibrotic tissue sub-region.

[0016] In a preferred embodiment, cardiac fibrosis is divided into four stages. By analyzing the magnetic resonance image data of a large number of cardiac samples at different fibrosis stages, features are extracted and integrated to obtain reference fusion feature values ​​f1, f2, f3, and f4 for each fibrosis stage: f1 represents the reference fusion feature value of the mild fibrosis stage; f2 represents the reference fusion feature value of the moderate fibrosis stage; f3 represents the reference fusion feature value for moderate to severe fibrosis stages; f4 represents the reference fusion feature value of the severe fibrosis stage; The magnetic resonance image of the target is obtained, the fusion feature F is extracted, the fusion feature F is calculated with the reference fusion feature value of each stage, and the fibrosis stage with the smallest minimized difference value is output.

[0017] Compared with the prior art, the present invention has the following beneficial technical effects: 1. Accurately identify fibrotic tissue: By constructing a grayscale measurement function and comparing the grayscale values ​​of each sub-region, the fibrotic tissue sub-region can be accurately identified; the grayscale measurement function directly quantifies tissue uniformity, which is highly consistent with the characteristics of fibrosis; combined with the convolutional neural network (CNN) to classify the fibrotic tissue region, it can further determine whether it is fibrotic tissue and its severity, thereby improving the accuracy of the judgment.

[0018] 2. Through preprocessing steps such as denoising, image enhancement and standardization, the quality of magnetic resonance images can be significantly improved, providing a clearer image basis for subsequent fibrosis detection.

[0019] 3. The spatial distance is fused with the color distance to reduce the impact of uneven lighting or artifacts; 4. The convolutional neural network can be used to extract quantitative image features from the image with high throughput, including the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-area, the proportion of the fibrotic tissue sub-area in the atrial wall, and provide quantitative statistical indicators for the detection and classification of fibrosis.

[0020] 4. The present invention realizes high-precision, high-efficiency and high-robustness myocardial fibrosis detection and classification through the collaborative design of precise preprocessing, two-level region screening (connectivity segmentation → grayscale measurement) and multimodal CNN classification. The technical effect is not only reflected in the improvement of algorithm performance, but also provides a reliable tool for subsequent analysis through automation, explainability and scalability design. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 It is a schematic diagram of the process of the myocardial magnetic resonance image fibrosis classification method based on convolutional neural network of the present invention; Figure 2 It is a schematic diagram of the results of the convolutional neural network-based myocardial magnetic resonance image fibrosis classification system of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] Example 1 The convolutional neural network-based myocardial magnetic resonance image fibrosis classification method of the present invention achieves efficient detection and classification of myocardial fibrosis through a three-level architecture of precise segmentation, grayscale feature screening and CNN deep classification. The candidate regions are roughly screened by connected feature segmentation and grayscale measurement function to reduce the computational burden of CNN; the end-to-end automated process does not require manual intervention from the original myocardial magnetic resonance image to the classification result, and is suitable for large-scale screening.

[0025] like Figure 1 FIG. 1 is a flow chart of a method for classifying myocardial magnetic resonance images for fibrosis based on a convolutional neural network according to the present invention. The method for classifying myocardial magnetic resonance images for fibrosis based on a convolutional neural network comprises the following steps: Step 1: Obtain an original magnetic resonance image of the target to be measured from a magnetic resonance imaging device and preprocess the original magnetic resonance image.

[0026] Use professional medical image processing software or related libraries to read myocardial magnetic resonance images. These myocardial magnetic resonance images are stored in DICOM (Digital Imaging and Communications in Medicine) format.

[0027] Convert the read DICOM image into a format suitable for subsequent processing, preferably using image formats such as PNG and JPEG. According to the noise characteristics of the image and processing requirements, select linear filtering methods such as mean filtering, median filtering, and Gaussian filtering, as well as non-linear filtering methods such as non-local mean filtering and wavelet denoising.

[0028] For the selected denoising algorithm, adjust the corresponding parameters to achieve the best denoising effect. For Gaussian filtering, you need to determine the Gaussian kernel size and standard deviation; for non-local mean filtering, you need to set parameters such as search window size and similarity threshold.

[0029] By adjusting the contrast of the image, the details in the image are made clearer and more visible. Preferably, histogram equalization or adaptive histogram equalization is used. Histogram equalization can equalize the grayscale histogram of the image and expand the grayscale dynamic range of the image; adaptive histogram equalization performs histogram equalization in a local area and can better preserve the local details of the image.

[0030] According to the actual situation of the image, the brightness of the image is appropriately adjusted by performing linear or nonlinear transformation on the grayscale value of the image to achieve brightness adjustment. Preferably, the grayscale value of the image is multiplied by a constant or an offset is added to increase or decrease the overall brightness of the image.

[0031] After the above preprocessing steps, a myocardial magnetic resonance image with higher quality and more suitable for fibrosis classification based on convolutional neural network can be obtained.

[0032] Step 2: Based on the connectivity features, the preprocessed magnetic resonance image is segmented into multiple sub-regions, each of which represents an independent connected part of the image.

[0033] Density estimation is performed on the preprocessed magnetic resonance image, and the desired number of grids is determined according to the overall density distribution of the magnetic resonance image and the size of the magnetic resonance image.

[0034] First, density estimation is performed on the preprocessed magnetic resonance image. Preferably, a Gaussian kernel density estimation method is used to calculate the density value of the surrounding pixels at each pixel point in the magnetic resonance image. By analyzing the density distribution of the entire image, the density changes in different areas of the magnetic resonance image are understood.

[0035] For the Gaussian kernel density estimation of magnetic resonance images, the specific formula is as follows: Two-dimensional Gaussian kernel function K(u,v): ; Among them, h is the bandwidth parameter, which controls the smoothness of the kernel, and u and v represent the coordinate offset between pixels.

[0036] The density estimation function f(x,y) at the pixel point (x,y) is: ; Where: (x IJ ,y IJ ) represents the pixel coordinates in the neighborhood window centered at (x, y); Z(x IJ ,y IJ ) is the intensity value of the pixel; n is the size of the neighborhood window (such as an n×n window), and I,J is the pixel point located at the Ith row and the Jth column in the window.

[0037] The normalization factor 1 / n ensures that the probability density integrates to 1.

[0038] From the density distribution, a representative pixel point is selected as the center of the initial grid. Preferably, a density peak point is selected as the center of the initial grid, and the density peak point represents the core of different feature areas in the magnetic resonance image.

[0039] Determine the number of grids and initial division: Based on the selected initial grid center, according to the spatial distribution of the magnetic resonance image and the number of grids, each grid is composed of the pixels closest to its center, which can ensure that each grid has a certain degree of independence and uniformity in space.

[0040] Set the initial grid and the center pixel of the grid.

[0041] Calculate the size of each grid according to the size of the magnetic resonance image and the desired number of grids N; The size of the grid C = image area S / number of grids N.

[0042] A grid center pixel point is preliminarily selected on each grid of the magnetic resonance image, and care should be taken to avoid the grid center pixel point falling on the edge of the grid or on the noise.

[0043] For example, each grid unit may be 10×10 pixels; when calculating the number of grids, the size of the grid size C is calculated based on the total size (width and height) of the magnetic resonance image and the number of grids.

[0044] In a preferred embodiment, after the grid center pixel point is initially determined, a local fine adjustment is performed on it to check whether there is a more suitable pixel point with less noise influence in a certain range around the grid center pixel point, and if so, the grid center is adjusted to this point.

[0045] After completing the grid division and determining the center pixel, verify the results several times. You can use some statistical indicators, such as calculating the difference between the grayscale value of the center pixel of the grid and the grayscale value of the surrounding pixels, to determine whether the center of the grid may fall on the noise. If a problem is found, re-grid or adjust the center pixel.

[0046] Combining the color distance and spatial distance of each grid, the total distance between each pixel in the grid and the center pixel of the grid is calculated.

[0047] Color distance d C is the distance d between two pixels in the color space (such as CIELAB) C .

[0048] Color distance d C In CIELAB color space it is calculated as follows: ; Among them: L1, a1, b1 are the CIELAB color space coordinates of the first pixel; L2, a2, b2 are the CIELAB color space coordinates of the second pixel.

[0049] Spatial distance d S is the distance between two pixels on the image plane, and the spatial distance d S The calculation is as follows: ; Among them: X1, Y1 are the coordinates of the first pixel on the image plane; X2, Y2 are the coordinates of the second pixel on the image plane.

[0050] Use the weighted sum method to combine the color distance and the spatial distance, and calculate the total distance D between each pixel point in the grid and the central pixel point of the grid, which is used to measure the similarity or difference between pixel points.

[0051] The calculation formula for the total distance D between each pixel point and the central pixel point of the grid is as follows: ; where, d C is the color distance between the pixel point in the grid and the central pixel point of the grid, and d S is the spatial distance between the pixel point in the grid and the central pixel point of the grid. W is the size of the grid, and m is a parameter used to control the relative importance of the color and spatial distances.

[0052] Calculate the total distance between each pixel point on the grid edge in the magnetic resonance image and the adjacent central pixel point of the grid. If the total distance from the adjacent central pixel point of the grid is less than the total distance from the current central pixel point of the grid, then assign the pixel point on the grid edge in the magnetic resonance image to the adjacent grid. According to the reallocation result, update the grid attribution of each edge pixel point to generate an updated grid division result.

[0053] First, determine the grid edge pixel points. Traverse each pixel point in the image. For each pixel point, check the grid to which its neighboring pixels belong. If there are pixels belonging to different grids among the neighboring pixels, then the current pixel point is located on the grid edge.

[0054] Secondly, use different colors or labels to visualize the pixel points in each grid to view the assignment result. Through the above steps, the distance between the grid edge pixel points in the magnetic resonance image and the adjacent grid center can be calculated, and the pixel points can be reallocated according to the distance. This method can optimize the image segmentation result and make the edge pixel points more reasonably belong to the neighboring regions.

[0055] In a preferred embodiment, the grid edge pixel points are determined based on coordinate comparison, specifically as follows: In this embodiment, the grid is regularly divided, and each grid is a square with a side length of W. For a pixel point with coordinates (x, y) in the image, if its x coordinate modulo W is close to 0 or W - 1, and its y coordinate modulo W is close to 0 or W - 1, then the pixel point may be located on the grid edge.

[0056] For example, when 0 ≤ x mod W ≤ 2 or W - 3 ≤ x mod W < W - 1, and 0 ≤ y mod W ≤ 2 or W - 3 ≤ y mod W ≤ W - 1, then it is determined that the pixel point is on the grid edge, where mod represents the modulo operation.

[0057] The connectivity features of each grid are calculated, and a segmentation method that conforms to the final grid is implemented based on the optimal value of the connectivity features to obtain multiple sub-regions.

[0058] For the i-th grid, first, calculate the variance R of the color distribution within the i-th grid i A high variance indicates that the color diversity of the grid is large, that is, the uniqueness of the grid is high.

[0059] Secondly, calculate the average value M of the total distance between all pixels in the i-th grid and the center pixel of the grid i The smaller the average distance, the more compact the grid is.

[0060] R i and M i Weighted combination into connected feature U i : ; in, is the variance weight parameter that adjusts the relative importance between uniqueness and compactness.

[0061] Finally, by adjusting the variance weight parameter and the initial grid center pixel position until the connected feature U generated by each grid i All of them are optimal values, realizing the final grid segmentation method and obtaining multiple sub-areas.

[0062] In practical applications, the atrial wall mainly consists of the following four sub-regions: blood pool, normal myocardium, pericardial fluid and fibrotic tissue. The four sub-regions are further identified as fibrotic tissue sub-regions in step three.

[0063] Step 3: Construct a grayscale measurement function for each sub-region, quantify the grayscale distribution characteristics of each sub-region, and identify the sub-region with the smallest grayscale measurement function value as the fibrotic tissue sub-region by comparing the grayscale measurement function values ​​of each sub-region.

[0064] The identification of fibrotic tissue sub-regions is regarded as a label assignment problem for each pixel p in the four sub-regions.

[0065] Specifically, the number of labels is set to four, corresponding to the above four types of regions (blood pool, normal myocardium, pericardial fluid and fibrotic tissue), and the sub-region where the grayscale measurement function E reaches the minimum after the final segmentation is completed is taken as the fibrotic tissue sub-region, that is, the fibrotic tissue.

[0066] Calculate the point weight between the center point and the edge point of each sub-region , calculated by the following formula: ; Where IP Represents the gray value of the edge point p in each region of the image, I C Represents the gray value of the center point of the C-th sub-region. There are M region edge points in total.

[0067] The edge points of adjacent regions constitute a set J, and the edge weight between the edge points p and q of adjacent regions is B. pq The calculation method is as follows: ; In the formula, I q Represents the gray value of the region edge point q, is the variance of the entire sub-region.

[0068] Based on the point weight T C and edge weight B pq The grayscale measurement function E is obtained as follows: ; The sub-region with the smallest output value of the grayscale measurement function E is the fibrotic tissue sub-region, and the fibrotic tissue sub-region is extracted.

[0069] In a preferred embodiment, fibrotic tissue is usually distributed in strips, so it is necessary to perform connectivity analysis on the extracted fibrotic tissue: isolated fibrotic tissue sub-regions are removed, and isolated points or small blocks are not formed.

[0070] Step 4: Extract features from the identified fibrotic tissue sub-regions and use them as input to the convolutional neural network model to classify the fibrotic tissue sub-regions and output the fibrosis stage to which the fibrotic tissue sub-regions belong.

[0071] Since the real atrial fiber layer is generally thin, occupying only 2-3 pixels, the single threshold method is not ideal for classifying images with large grayscale differences in fibrosis areas. Therefore, the present invention uses a feature fusion process to fuse the extracted different features.

[0072] Feature fusion can be used to integrate features from different sources, and a convolutional neural network model can be used to classify the extracted fibrotic tissue areas.

[0073] In this embodiment, the extracted features include: the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region and the proportion of the fibrotic tissue in the atrial wall.

[0074] Among them, in the proportion of fibrotic tissue in the atrial wall, the numerator is the number of pixels of fibrotic tissue, and the denominator is the number of pixels of the atrial wall.

[0075] The feature fusion process is used to fuse the extracted different features. The specific fusion process can be described by the following formula: F = Concat(G,Q); Among them, G represents the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-area, Q represents the proportion of the fibrotic tissue sub-area in the atrial wall, and F represents the fusion feature generated by fusion.

[0076] It should be noted that for the fibrotic tissue sub-region with a determined boundary, the calculation of its longest diameter and shortest diameter can be achieved by performing geometric analysis on the points on the contour, calculating the distance between any two points on the contour, traversing all possible point pairs, finding the maximum value of the distance as the longest diameter, and the minimum value as the shortest diameter, and then calculating the ratio of the longest diameter to the shortest diameter to obtain the feature G.

[0077] After completing the segmentation of the atrial wall region, the total number of pixels contained in the atrial wall is counted as the total area of ​​the atrial wall. Calculating the area of ​​the fibrotic tissue sub-region: Counting the number of pixels in the fibrotic tissue sub-region as the area of ​​the sub-region.

[0078] Calculate the proportion: divide the area of ​​the fibrotic tissue sub-region by the total area of ​​the atrial wall to obtain the proportion of the fibrotic tissue sub-region in the atrial wall, that is, the characteristic Q.

[0079] Fuse function is a fusion function that combines multiple features in different ways to improve the comprehensiveness of data processing. Fuse functions include: weighted summation or concatenation.

[0080] Weighted summation assigns different weights to different data sources or features, and then performs weighted summation to obtain a comprehensive output.

[0081] Concatenation is to directly connect multiple feature vectors together to form a new high-dimensional feature vector.

[0082] Input features: Assume G has n1 eigenvectors: G1, G2, …, G a ,…,G n1 ; Assume Q has n2 eigenvectors: Q1, Q2, …, Q b ,…,Q n2 ; Through the weighted summation and splicing method, the weighted summation of multiple feature vectors of each feature is first performed, and then the multiple feature vectors are spliced ​​together to obtain the fused feature F.

[0083] ; Concat is a concatenation function that directly connects multiple feature vectors together to form a new high-dimensional feature vector. F is a high-dimensional fusion feature, G is a is the ath eigenvector of G, W ais the weight coefficient of the ath eigenvector of G, Q b is the bth eigenvector of Q, W b is the weight coefficient of the bth eigenvector of Q.

[0084] The fusion feature F is input into the convolutional neural network model, which classifies the fibrotic tissue sub-regions and outputs the fibrosis stage to which the fibrotic tissue sub-regions belong. Specifically, the convolutional neural network model first calculates the minimized difference between the fusion feature and the reference fusion feature value at each stage through a minimization function.

[0085] The minimization function is , the output minimized difference value V is: ; t=1,2…T; Among them, f t is the reference fusion feature value of the tth stage of fibrosis, T is the total number of fibrosis stages, and the total number of stages and the reference fusion feature value of each stage are selected according to actual needs.

[0086] The convolutional neural network model is designed for the scenario of medical imaging diagnosis of cardiac fibrosis. It is assumed that cardiac fibrosis is divided into four stages (T=4). By analyzing the magnetic resonance image data of a large number of cardiac samples with different fibrosis stages, the features are extracted and integrated to obtain the reference fusion feature value f for each stage. t : f1=[0.23,0.45], represents the reference fusion feature value of the mild fibrosis stage (stage 1), f2 = [0.35, 0.62], corresponding to the reference fusion eigenvalue of the moderate fibrosis stage (stage 2); f3 = [0.48, 0.78], indicating the reference fusion feature value of moderate to severe fibrosis stage (stage 3); f4 = [0.60, 0.95], the reference fusion feature value for the severe fibrosis stage (stage 4); Finally, the same magnetic resonance image examination is performed on the target to be tested, and the fusion feature F = [0.46, 0.75] is extracted. The fusion feature F is input into the trained convolutional neural network model respectively, and the fibrosis stage t with the smallest difference value is output by calculating the fusion feature F with the reference fusion feature value of each stage.

[0087] A difference threshold is set. If the minimized difference value V is greater than the difference threshold, it is determined that the fibrotic tissue does not meet any fibrosis stage.

[0088] In an alternative embodiment, the reference fused feature value for each stage may be replaced by a reference fused feature value for each fibrosis form.

[0089] In this embodiment, fibrosis images are divided into three different forms, namely: reactive interstitial fibrosis, reparative fibrosis and perivascular fibrosis.

[0090] Reactive interstitial fibrosis is caused by persistent long-term excessive pressure load (such as hypertension, obstructive hypertrophic cardiomyopathy, etc.), inflammatory response and age-dependent increase, and is characterized by excessive deposition of extracellular matrix (without accompanying myocardial cell reduction).

[0091] Reparative fibrosis is the result of repair of myocardial tissue after myocardial infarction. It is a process in which scar tissue produced by fibroblasts replaces necrotic myocardial tissue. It is often seen at the insertion point of the left and right ventricles of the ventricular septum, and can also be found in the middle layer of the myocardium.

[0092] In the form of perivascular fibrosis, dilation of the microvascular adventitia is the characteristic, reflecting fibroblast transformation or fibrotic activation of endothelial and mural cells, leading to disruption of coronary recanalization and causing microvascular dysfunction, thereby causing diastolic dysfunction and aggravating the severity of ischemia.

[0093] Example 2 This embodiment proposes a classification system for implementing the myocardial magnetic resonance image fibrosis classification method based on convolutional neural network in embodiment 1. The structural schematic diagram of the classification system is shown in FIG. Figure 2 shown.

[0094] The classification system includes: data acquisition module, image preprocessing module, image segmentation module, fibrosis tissue detection module, feature extraction module and classification module. Each module works together to complete the entire process from original magnetic resonance images to fibrosis tissue classification.

[0095] Data acquisition module: used to obtain the original magnetic resonance image of the target to be measured from the magnetic resonance imaging device.

[0096] Specifically, the magnetic resonance imaging device is connected via a network interface or a data storage device, and the raw magnetic resonance image data is read using a medical image reading library (such as SimpleITK).

[0097] Image preprocessing module: used to preprocess the original magnetic resonance images to improve the quality of the magnetic resonance images and the accuracy of subsequent processing.

[0098] Image segmentation module: used to segment the preprocessed magnetic resonance image into multiple sub-regions based on connectivity features, each sub-region represents an independent connected part in the image.

[0099] The image segmentation module includes: an initialization unit, a distance calculation unit, a pixel point allocation unit and a connected feature calculation unit.

[0100] Initialization unit: Calculate the size of each grid according to the image size and the expected number of grids, and select a grid center pixel on each grid.

[0101] Distance calculation unit: Combines color distance and spatial distance to calculate the first total distance between the edge pixel of each grid and the center point of the current grid, and calculates the second total distance between the edge pixel of each grid and the center point of the adjacent grid.

[0102] A pixel point allocating unit: determines the relationship between the second total distance and the first total distance, and if the second total distance is smaller than the first total distance, reallocates the edge pixel point to an adjacent grid.

[0103] Connectivity feature calculation unit: Calculate the connectivity features of each grid, adjust the variance weight parameters and the position of the central pixel of each grid until the connectivity features generated by each grid are the optimal values, and finally segment the grid to obtain multiple sub-regions.

[0104] Fibrosis tissue detection module: construct a grayscale measurement function for each sub-region, quantify the grayscale distribution characteristics of each sub-region, and identify the sub-region with the smallest grayscale measurement function value as the fibrosis tissue sub-region by comparing the grayscale measurement function values ​​of each sub-region.

[0105] The fibrosis tissue detection module includes: a grayscale measurement function construction unit and a sub-region comparison unit.

[0106] Grayscale measurement function construction unit: construct a grayscale measurement function for each sub-region to quantify the grayscale distribution characteristics of each sub-region.

[0107] Sub-region comparison unit: calculates the grayscale measurement function value of each sub-region, and by comparing the grayscale measurement function values ​​of each sub-region, finds the sub-region with the smallest grayscale measurement function value as the fibrotic tissue sub-region.

[0108] Feature extraction and classification module: used to extract features from the identified fibrotic tissue sub-regions as input to the convolutional neural network model.

[0109] Classification module: used to classify fibrotic tissue areas using a convolutional neural network model and output the fibrosis stage to which the fibrotic tissue sub-area belongs.

[0110] In a preferred embodiment, the classification module includes: a model building unit, a model training unit and a classification prediction unit.

[0111] Model building unit: Construct a convolutional neural network model, including convolutional layers, pooling layers, fully connected layers, etc. It is preferred to use a deep learning framework (such as TensorFlow, PyTorch) to build the model.

[0112] Model training unit: Use labeled training data to train the convolutional neural network model and adjust the parameters of the convolutional neural network model to improve classification accuracy.

[0113] Classification prediction unit: The extracted features are input into the trained convolutional neural network model for classification prediction to obtain the fibrosis stage to which the fibrotic tissue sub-region belongs.

[0114] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for classifying myocardial magnetic resonance image fibrosis based on a convolutional neural network, characterized in that: It includes the following steps: Obtain the original magnetic resonance image of the target to be measured from a magnetic resonance imaging device, and preprocess the original magnetic resonance image; Based on the connectivity feature, segment the preprocessed magnetic resonance image into multiple sub-regions, and each sub-region represents an independent connected part in the image; Construct a gray-scale measurement function for each sub-region, quantify the gray-scale distribution characteristics of each sub-region, and by comparing the gray-scale measurement function values of each sub-region, identify the sub-region with the smallest gray-scale measurement function value as the fibrotic tissue sub-region; Extract features from the identified fibrotic tissue sub-region as the input of the convolutional neural network model, classify the fibrotic tissue sub-region, and output the fibrotic stage to which the fibrotic tissue sub-region belongs.

2. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 1, characterized in that: Perform density estimation on the preprocessed magnetic resonance image, select the density peak point as the central pixel point, divide the preprocessed magnetic resonance image into regular grids according to the preset number of grids, combine the color distance and the spatial distance, calculate the first total distance between the edge pixel points of each grid and the central pixel point of the current grid, and then calculate the second total distance between the edge pixel points of each grid and the central pixel points of adjacent grids. If the second total distance is less than the first total distance, reassign the edge pixel point to the adjacent grid; According to the reassignment result, update the grid attribution of each edge pixel point to generate an updated grid division result.

3. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 2, characterized in that: Perform Gaussian kernel density estimation on the magnetic resonance image, and the specific formula is as follows: The two-dimensional Gaussian kernel function is K(u,v): ； where h is the bandwidth parameter that controls the smoothness of the kernel, and u and v represent the coordinate offsets between pixel points; The density estimation function f(x,y) at the pixel point (x,y) is: ； Where: (x IJ ,y IJ ) represents the pixel coordinates in the neighborhood window centered at (x, y); Z(x IJ ,y IJ ) is the intensity value of the pixel; n is the size of the neighborhood window (such as an n×n window), and I,J is the pixel point located at the Ith row and the Jth column in the window.

4. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 1, characterized in that: Let G represent the ratio of the longest diameter to the shortest diameter of the fibrotic tissue sub-region, Q represent the proportion of the fibrotic tissue sub-region in the atrial wall, and let G have n1 eigenvectors: G1, G2, …, G a ,…,G n1 ; Assume Q has n2 eigenvectors: Q1, Q2, …, Q b ,…,Q n2 ; Use the feature fusion process to fuse the extracted features G and Q: ； Among them, Concat is the concatenation function, F is the high-dimensional fusion feature, G a is the ath eigenvector of G, W a is the weight coefficient of the ath eigenvector of G, Q b is the bth eigenvector of Q, W b is the weight coefficient of the bth eigenvector of Q.

5. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 4, characterized in that: Input the fusion feature F into the convolutional neural network model, calculate the minimum difference value V between the fusion feature F of the fibrotic tissue sub-region and the reference fusion feature values of each fibrotic stage through the minimization function, output the fibrotic stage t corresponding to the minimum difference value V. If the minimum difference value V is greater than the difference threshold, it is determined that the fibrotic tissue sub-region does not belong to any fibrotic stage.

6. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 2, characterized in that: Determine the grid edge pixel points based on coordinate comparison, specifically including: Assume that each grid is a square with a side length of W. For a pixel point with coordinates (x,y) in the image, when 0 ≤ x mod W ≤ 2 or W - 3 ≤ x mod W < W - 1, and 0 ≤ y mod W ≤ 2 or W - 3 ≤ y mod W ≤ W - 1, then it is determined that the pixel point is on the grid edge, where mod represents the remainder operation.

7. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 2, characterized in that: Calculate the variance R of the color distribution in the i-th grid i , calculate the average value M of the total distance between all pixels in the i-th grid and the center pixel of the grid i , based on the variance R i and the mean value M i Calculate the connectivity feature U of the i-th grid i ; By adjusting the variance weight parameters and the position of each grid center pixel, until the connectivity features generated by each grid are the optimal values, the final grid segmentation is achieved to obtain multiple sub-regions.

8. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 7, characterized in that: The process of identifying fibrotic tissue subregions includes: calculating the point weights between the region center point and the region edge point of each subregion : ； In the formula, I P represents the gray value of the regional edge point p in the C-th sub-region, I C Represents the gray value of the center point of the C-th sub-region; Assume that there are M regional edge points in the C-th sub-region, the adjacent regional edge points constitute a set J, and the edge weight between adjacent regional edge points p and q is B pq The calculation method is as follows: ； In the formula, I q Represents the gray value of the region edge point q, is the gray value variance of the entire sub-region; Based on the point weight T C and edge weight B pq The grayscale measurement function E is obtained as follows: ； The sub-region with the smallest output value of the gray-scale measurement function E is the fibrotic tissue sub-region.

9. The method for classifying myocardial magnetic resonance image fibrosis based on convolutional neural network according to claim 5, characterized in that: Assume that cardiac fibrosis is divided into 4 stages. By analyzing the magnetic resonance image data of a large number of existing cardiac samples with different fibrotic stages, extract and integrate features to obtain the reference fusion feature values f1, f2, f3, f4 of each fibrotic stage: f1 represents the reference fusion feature value of the mild fibrotic stage; f2 represents the reference fusion feature value of the moderate fibrotic stage; f3 represents the reference fusion feature value of the moderate-severe fibrotic stage; f4 represents the reference fusion feature value of the severe fibrotic stage; The magnetic resonance image of the target is obtained, the fusion feature F is extracted, the fusion feature F is calculated with the reference fusion feature value of each stage, and the fibrosis stage with the smallest minimized difference value is output.

Images