A spinal MRI image segmentation method based on deep learning

Abstract

The present invention discloses a spinal MRI image segmentation method based on deep learning. The present invention takes into account the unclear imaging of bones such as the spine in MRI data, utilizes EfficientNetV2 2D and EfficientNetV2 3D with good image classification effect for feature extraction, improves the feature extraction capability through ASPP 2D, ASPP 3D, Attention Mechanism 2D, and Attention Mechanism 3D, and finally obtains the segmentation result by upsampling through bilinear interpolation.

Description

Technical Field

[0001] The present invention relates to a spinal MRI image segmentation method based on deep learning. Background Art

[0002] With the development of medical image segmentation, the spine can be segmented from MRI images. Combined with 3D imaging software, the original 2D tomographic data can be converted into 3D data, allowing doctors to more easily and quickly understand the patient's spinal condition. Traditional spine segmentation methods have the shortcomings of low segmentation accuracy and low efficiency. The segmentation accuracy cannot meet medical requirements, and 3D modeling is meaningless. In the past decade, deep learning has developed rapidly, performing well in feature extraction, and is increasingly being applied to the field of image segmentation. Currently, many deep learning methods have been applied to image segmentation. Spine image segmentation is more targeted at CT image segmentation. Spine MRI image segmentation is still one of the difficulties in image segmentation because MRI has low sensitivity to bones. Summary of the Invention

[0003] The present invention aims to solve the problem that the bone imaging of spinal MRI images is unclear and difficult to segment, and proposes a spinal MRI image segmentation method based on deep learning.

[0004] The technical solutions adopted in the present invention are:

[0005] A spinal MRI image segmentation method based on deep learning includes the following steps:

[0006] 1) Divide each subject's original MRI slice image into several 2D images to obtain 2D data. At the same time, use data stitching to stitch all 2D images of the original MRI slice image to obtain a 3D image and 3D data;

[0007] 2) EfficientNetV2 is renamed EfficientNetV2 2D. EfficientNetV2 3D is obtained by replacing the 2D convolution and 2D pooling in EfficientNetV2 2D with 3D convolution and 3D pooling. EfficientNetV2 2D and EfficientNetV2 3D are used to extract the feature matrix of the image segmentation target from the MRI 2D and 3D images obtained in step 1, respectively, to complete the downsampling operation.

[0008] 3) ASPP is named ASPP 2D, and ASPP 3D is obtained by replacing the 2D convolution and 2D pooling in ASPP 2D with 3D convolution and 3D pooling;

[0009] Attention is named Attention 2D. By replacing the two-dimensional weight matrix in the Attention mechanism 2D with a three-dimensional weight matrix, the Attention mechanism 3D is obtained.

[0010] ASPP 2D is used to perform multi-scale feature fusion on the feature matrix of the image segmentation target extracted from the MRI 2D image and expand the receptive field of the image segmentation extracted features. Attention 2D is used to improve the correlation between individual data in the feature matrix of the image segmentation target extracted from the MRI 2D image.

[0011] ASPP 3D is used to perform multi-scale feature fusion on the feature matrix of the image segmentation target extracted from the MRI 3D image and expand the receptive field of the image segmentation extracted features. Attention mechanism 3D is used to improve the correlation between individual data in the feature matrix of the image segmentation target extracted from the MRI 3D image.

[0012] 4) The feature matrices of the image segmentation targets of the MRI 2D image and the MRI 3D image obtained in step 3) are concatenated and up-sampled using bilinear interpolation. The feature matrices of the image segmentation targets of the extracted MRI 2D and MRI 3D images are restored to the original MRI slice image size, and the segmentation results of the deep learning network are obtained.

[0013] 5) The segmentation results of the MRI 2D image are fused with the segmentation results of the MRI 3D image, ensuring that the segmentation results have both segmentation accuracy and position information and 3D data.

[0014] Furthermore, the original MRI slice image is divided into 12 2D images, each of which has a size of 880×880. By data splicing, the 12 880×880 2D images are spliced ​​into 12×880×880 using code to obtain a 3D image and 3D data.

[0015] Furthermore, the EfficientNetV2 2D and EfficientNetV2 3D are used to extract feature matrices of image segmentation targets from MRI 2D images and 3D images, respectively. The size of the feature matrix is ​​32×256×256, where 32 is the number of channels and 256×256 is the size of the feature matrix.

[0016] Furthermore, the ASPP 2D and ASPP 3D are used to perform multi-scale feature fusion on the feature matrix of the extracted image segmentation target and to expand the receptive field of the corresponding image segmentation extraction feature; both ASPP 2D and ASPP 3D use three dilated convolutions with ratios of 6, 12 and 18, and the size of the convolution kernel is 3×3, resulting in an image segmentation extraction feature of 32×256×256, where 32 is the number of channels and 256×256 is the size of the image segmentation extraction feature.

[0017] Furthermore, after the upsampling operation is completed by bilinear interpolation in step 4), the feature matrix of the image segmentation target is restored to the original MRI slice image size of 880×880.

[0018] The present invention has the following beneficial effects:

[0019] This paper takes into account the unclear imaging of bones such as the spine in MRI data, and uses EfficientNetV2 2D and EfficientNetV2 3D, which have good image classification effects, for feature extraction. The feature extraction capability is improved through ASPP 2D, ASPP 3D, Attention Mechanism 2D, and Attention Mechanism 3D. Finally, upsampling is performed through bilinear interpolation to obtain the final segmentation result. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a system principle block diagram of the deep learning-based spinal MRI image segmentation method implemented in the present invention.

[0021] Figure 2 Flowchart of the overall present invention.

[0022] Figure 3 Schematic diagram of the principles of ASPP 2D and ASPP 3D.

[0023] Figure 4 Network module diagram for EfficientNetV2 2D and EfficientNetV2 3D. DETAILED DESCRIPTION

[0024] The present invention will be further described below with reference to the accompanying drawings.

[0025] like Figure 1 and Figure 2 The present invention provides a spinal MRI image segmentation method based on deep learning, comprising the following steps:

[0026] 1) Each subject's 12 880×880 original MRI slice images were divided into 12 2D images, each 880×880 in size, to obtain 2D data. Simultaneously, the 12 880×880 original MRI images were concatenated into 12×880×880 images using code to obtain 3D images and 3D data.

[0027] 2) EfficientNetV2 is renamed EfficientNetV2 2D. EfficientNetV2 3D is obtained by replacing the 2D convolution and 2D pooling in EfficientNetV2 2D with 3D convolution and 3D pooling. EfficientNetV2 2D and EfficientNetV2 3D are used to extract the feature matrices of the image segmentation targets from the MRI 2D and 3D images obtained in step 1, respectively. Compared with the original data, the feature matrices are significantly smaller, thus completing the downsampling operation.

[0028] 3) ASPP is named ASPP 2D, and ASPP 3D is obtained by replacing the 2D convolution and 2D pooling in ASPP 2D with 3D convolution and 3D pooling;

[0029] Attention is named Attention 2D. By replacing the two-dimensional weight matrix in the Attention mechanism 2D with a three-dimensional weight matrix, the Attention mechanism 3D is obtained.

[0030] ASPP 2D is used to perform multi-scale feature fusion on the feature matrix of the image segmentation target extracted from the MRI 2D image and expand the receptive field of the image segmentation extracted features. Attention 2D is used to improve the correlation between individual data in the feature matrix of the image segmentation target extracted from the MRI 2D image.

[0031] ASPP 3D is used to perform multi-scale feature fusion on the feature matrix of the image segmentation target extracted from the MRI 3D image and expand the receptive field of the image segmentation extracted features. Attention mechanism 3D is used to improve the correlation between individual data in the feature matrix of the image segmentation target extracted from the MRI 3D image.

[0032] 4) The feature matrices of the image segmentation targets of the MRI 2D image and the MRI 3D image obtained in step 3) are concatenated and up-sampled using bilinear interpolation. The feature matrices of the image segmentation targets of the extracted MRI 2D and MRI 3D images are restored to the original MRI slice image size, and the segmentation results of the deep learning network are obtained.

[0033] 5) The segmentation results of the MRI 2D image are fused with the segmentation results of the MRI 3D image, ensuring that the segmentation results have both segmentation accuracy and position information and 3D data.

[0034] The present invention demonstrates that the deep learning image segmentation model can achieve a test accuracy of over 86% under the condition that MRI images do not clearly image bones, and the detection results are effective and reliable. This method has great prospects and potential value in the research of spinal MRI image segmentation methods.

[0035] like Figure 3 As shown in the figure, both ASPP 2D and ASPP 3D are used to expand the receptive field and improve the depth of features. They include 3 dilated convolutions, global average pooling, 1×1 convolution, batch normalization, and activation functions. Multi-scale features are fused through dilated convolutions of different ratios, and 1×1 convolution is used to reduce the number of channels and the amount of computation.

[0036] ASPP 2D and ASPP 3D are used to expand the size of the feature matrix by using dilated convolutions with different ratios to enlarge the receptive field. Three dilated convolutions with ratios of 6, 12, and 18 are used, and the size of the convolution kernel is 3×3. The size of the resulting feature matrix is ​​32×12×256×256, where 32 is the number of channels and 12×256×256 is the size of the feature matrix.

[0037] EfficientNetV2 2D and EfficientNetV2 3D are used to downsample and extract the feature matrix of the image. The size of the feature matrix is ​​32×12×256×256, where 32 is the number of channels and 12×256×256 is the size of the feature matrix. Figure 3 N=6 (i.e. 6 times).

[0038] like Figure 4 The EfficientNetV2 2D and EfficientNetV2 3D shown in the figure include: 3×3 convolution, SE module, residual connection and 1×1 convolution to form a network block. EfficientNetV22D and 3D are used to extract the features of MRI images, downsample the original images by 16 times, and finally combine the features together.

[0039] Attention 2D and Attention 3D use position encoding to extract the position information of training set images. After encoding, the attention mechanism is used to extract key information, so that the network only pays attention to important information and ignores unimportant information. Finally, residual connections and Sigmoid activation functions are combined to improve the network's computing speed and convergence ability.

[0040] The above description is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principles of the present invention. These improvements should also be regarded as the scope of protection of the present invention.

Claims

1. A spinal MRI image segmentation method based on deep learning, characterized by: The steps include: 1) Divide each subject's original MRI slice image into several 2D images to obtain 2D data. At the same time, use data stitching to stitch all 2D images of the original MRI slice image to obtain a 3D image and 3D data; 2) EfficientNetV2 3D is obtained by replacing the 2D convolution and 2D pooling in EfficientNetV2 2D with 3D convolution and 3D pooling. The feature matrices of the image segmentation targets are extracted from the MRI 2D image and 3D image obtained in step 1 using EfficientNetV2 2D and EfficientNetV2 3D, respectively, to complete the downsampling operation; 3) ASPP 3D is obtained by replacing the 2D convolution and 2D pooling in ASPP 2D with 3D convolution and 3D pooling; By replacing the two-dimensional weight matrix in the Attention mechanism 2D with a three-dimensional weight matrix, we get the Attention mechanism 3D. ASPP 2D is used to perform multi-scale feature fusion on the feature matrix of the image segmentation target extracted from the MRI 2D image and expand the receptive field of the image segmentation extracted features. Attention 2D is used to improve the correlation between individual data in the feature matrix of the image segmentation target extracted from the MRI 2D image. ASPP 3D is used to perform multi-scale feature fusion on the feature matrix of the image segmentation target extracted from the MRI 3D image and expand the receptive field of the image segmentation extracted features. Attention mechanism 3D is used to improve the correlation between individual data in the feature matrix of the image segmentation target extracted from the MRI 3D image. 4) The feature matrices of the image segmentation targets of the MRI 2D image and the MRI 3D image obtained in step 3) are concatenated and up-sampled using bilinear interpolation. The feature matrices of the image segmentation targets of the extracted MRI 2D and MRI 3D images are restored to the original MRI slice image size, and the segmentation results of the deep learning network are obtained. 5) The segmentation results of the MRI 2D image are fused with the segmentation results of the MRI 3D image, ensuring that the segmentation results have both segmentation accuracy and position information and 3D data.

2. The deep learning-based spinal MRI image segmentation method according to claim 1, wherein: The original MRI slice image is divided into 12 2D images, each of which has a size of 880×880. By data splicing, the 12 880×880 2D images are spliced ​​into 12×880×880 using code to obtain a 3D image and 3D data.

3. The deep learning-based spinal MRI image segmentation method according to claim 1, wherein: The EfficientNetV2 2D and EfficientNetV2 3D are used to extract feature matrices of image segmentation targets from MRI 2D images and 3D images, respectively. The size of the feature matrix is ​​32×256×256, where 32 is the number of channels and 256×256 is the size of the feature matrix.

4. The deep learning-based spinal MRI image segmentation method according to claim 1, wherein: The ASPP2D and ASPP 3D are used to perform multi-scale feature fusion on the feature matrix of the extracted image segmentation target and to expand the receptive field of the corresponding image segmentation extracted features; Both ASPP 2D and ASPP 3D use three dilated convolutions with ratios of 6, 12, and 18, and the size of the convolution kernel is 3×3. The resulting image segmentation feature extraction is 32×256×256, where 32 is the number of channels and 256×256 is the size of the image segmentation feature extraction.

5. The deep learning-based spinal MRI image segmentation method according to claim 1, wherein: After the upsampling operation is completed by bilinear interpolation in step 4), the feature matrix of the image segmentation target is restored to the original MRI slice image size of 880×880.

Images