A multimodal MRI image acoustic neuroma segmentation method

Abstract

A method for segmenting acoustic nerve schwannomas in multimodal MRI images includes the following steps: 1) Obtaining T1-weighted and T2-weighted acoustic nerve schwannomas MRI datasets, preprocessing them, and obtaining training samples from the datasets; 2) Using the training samples to train a proposed network model based on improved 3D convolution and Wnet networks, and using the network model to segment acoustic nerve schwannomas in multimodal MRI images; 3) Constructing a spatial heatmap of acoustic nerve schwannomas location in standard space, and using the trained acoustic nerve schwannomas location spatial heatmap as a constraint to verify the obtained acoustic nerve schwannomas segmentation results, removing segmentation results with unreasonable locations. This invention achieves automatic and accurate segmentation of acoustic nerve schwannomas.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method for segmenting acoustic nerve schwannomas in multimodal MRI images. Background Technology

[0002] Brain tumors are abnormal tissue proliferations that lead to increased intracranial pressure, damage to the central nervous system, and ultimately, death. Acoustic neuroma, originating from the sheath of the acoustic nerve, is a benign tumor, more accurately termed acoustic neuroma, and is one of the most common intracranial tumors. Acoustic neuromas, compared to common gliomas, are relatively fixed in location and easier to segment. Currently, there is no universal algorithm that can solve the segmentation problem for all brain tumors. Therefore, designing and developing an automatic and precise segmentation algorithm specifically for acoustic neuromas to assist doctors in preoperative diagnosis remains a challenging task. Summary of the Invention

[0003] To overcome the problems of existing tumor segmentation methods based on single-modal images failing to express all tumor information, tumor segmentation methods based on multimodal images failing to accurately fuse multimodal information, and existing 3D convolutional deep learning networks consuming excessive resources and having slow training speeds, this invention proposes a multimodal MRI image acoustic neuroma segmentation method based on improved 3D convolutional and Wnet networks, achieving automatic and accurate segmentation of acoustic neuroma.

[0004] The technical solution adopted by this invention to solve its technical problem is:

[0005] A method for segmenting acoustic nerve schwannomas in multimodal MRI images includes the following steps:

[0006] Step 1, Image Preprocessing: Acquire the acoustic neuroma MRI dataset, and perform spatial resampling, image registration, and image grayscale thresholding preprocessing. Align the T1-enhanced weighted image and T2-weighted image with the same voxel size in space, and further generate the corresponding brain region / background region mask through grayscale thresholding and connected component information.

[0007] Step 2: Generate training samples: Combine the brain region / background region mask image generated in Step 1 with the tumor ground truth annotation in the training samples, and simultaneously cut the image and ground truth. Input the cut blocks into the network for training to reduce the pressure on subsequent network training.

[0008] Step 3: Training the network model: Based on the 3D convolution in the network model, a new type of convolution is used, which replaces the 3D convolution with a one-dimensional convolution and a two-dimensional convolution; the Wnet network is improved on the Unet network architecture, which simultaneously inputs the T1 enhanced weighted image and the T2 weighted image for downsampling, and uses an attention mechanism to perform fusion segmentation in the high-dimensional feature space. The network model is trained using the training samples generated in step 2, and the network model is used to perform acoustic tumor segmentation on multimodal MRI images.

[0009] Step 4: Add spatial constraints for the location of the otitis media: During the training process, a spatial heatmap of the location of the otitis media in a standard space is constructed. The spatial heatmap of the location of the otitis media obtained in the training is used as a constraint to examine the segmentation results of the otitis media obtained in Step 3 and remove segmentation results with unreasonable locations.

[0010] Furthermore, in step 1, the resampling algorithm used is the bilinear interpolation algorithm, and the registration process uses the mutual information B-spline registration method.

[0011] In step 3, a one-dimensional convolution and a two-dimensional convolution are used instead of a three-dimensional convolution. The one-dimensional convolution is used to extract the connection information between adjacent layers of the MRI image, and the two-dimensional convolution is used to extract information from a single layer of the MRI image. Using these two convolutions instead of the three-dimensional convolution can connect the information between adjacent layers of the MRI image, while reducing the number of network parameters and improving the training and segmentation speed.

[0012] In step 3, an improved Wnet network is proposed based on the Unet network architecture. The T1-weighted image and the T2-weighted image are simultaneously input for dual-channel downsampling. The multimodal inputs are fused using an attention mechanism in the high-dimensional feature space. Since different modalities of MRI images contain different information, the T1-weighted image and the T2-weighted image are simultaneously input for downsampling to obtain features of multiple channels. The features of multiple channels have different importance. Features that are beneficial to the segmentation result are selected and features that are detrimental to the segmentation result are suppressed. The multi-channel features are weighted using an attention mechanism. After weighted fusion, upsampling is performed to restore the feature map and obtain the segmentation result.

[0013] In step 4, each MRI image is matched to an average standard spatial brain region. A plane is constructed through the bottom and midline of the internal channels on both sides of the brain, another plane is constructed through the midline of the internal channels perpendicular to the bottom of the internal channels, and a coronal plane is also constructed to obtain a three-dimensional coordinate system. Each MRI image is matched to this standard spatial coordinate system, and the locations of acoustic tumors that appear during training are marked in the standard space to obtain the high-frequency occurrence locations of acoustic tumors. A heat map is then constructed. The acoustic tumor location spatial heat map obtained from training is used as a constraint to verify the acoustic tumor segmentation results, and segmentation results with unreasonable locations are removed.

[0014] This invention combines an improved 3D convolution module with the Wnet network. First, a one-dimensional convolution and a two-dimensional convolution are used instead of a three-dimensional convolution. The one-dimensional convolution is used to extract the connection information between adjacent layers of the MRI image, while the two-dimensional convolution is used to extract information from a single layer of the MRI image. Then, T1-weighted and T2-weighted images are simultaneously input into the network for downsampling to obtain features from multiple channels. These multi-channel features are then weighted using an attention mechanism, and after weighted fusion, upsampling is performed to recover the feature map, resulting in the segmentation result.

[0015] The beneficial effect of this invention is that it enables automatic and accurate segmentation of acoustic nerve schwannomas. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the network model for an embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of the convolution model established by the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further explained below in conjunction with specific embodiments and accompanying drawings.

[0019] Reference Figure 1 and Figure 2 A method for segmenting acoustic neuroma from multimodal MRI images, which can fully utilize the information between multimodal MRI images, requires fewer resources, and has a fast training speed, can achieve automatic and accurate segmentation of acoustic neuromas, including the following steps:

[0020] Step 1, Image Preprocessing: Acquire the acoustic neuroma MRI dataset, and perform preprocessing such as spatial resampling, image registration, and image grayscale thresholding. Align the T1-enhanced weighted image and T2-weighted image with the same voxel size in space, and further generate the corresponding brain region / background region mask by grayscale thresholding and connected component information.

[0021] In step 1, the resampling algorithm used is the bilinear interpolation algorithm, and the registration process uses the B-spline registration method with mutual information.

[0022] Step 2: Generate training samples: In order to train the network model used in this invention, three-dimensional sub-images are extracted from the training set. In the implementation of this invention, 256 sub-images of size 32×32×32 are extracted from each training set image of size 256×256×128. The cut blocks are input into the network for training to reduce the pressure on subsequent network training.

[0023] Step 3: Training the Network Model: The improved 3D convolutional WNet framework used in this invention comprises an encoder module with four convolutional layers and a max-pooling layer, containing 32, 64, 128, and 256 feature maps respectively; an attention mechanism module with two fully connected layers and an average pooling layer, containing 1024, 512, and 256 feature maps for different dimensional features respectively; and a decoder module with four deconvolutional layers and convolutional layers, containing 256, 128, 64, and 32 feature maps respectively. In the convolutional layers, improved 3D convolution is applied, including a 3×3×1 convolutional kernel and a 1×1×3 convolutional kernel. For all max-pooling layers, the pooling size is 2×2×2, and the stride is 2. For all deconvolutional layers, the deconvolutioned feature maps are weighted and combined with the corresponding features in the encoder module through the attention mechanism module. After decoding, a sigmoid activation function is used to generate voxel-level predictions. The network model is trained using the training samples generated in step 2, and then the trained network is applied to segment the T1-enhanced weighted image and the T2-weighted image to be segmented.

[0024] In step 3, a one-dimensional convolution and a two-dimensional convolution are used to replace the three-dimensional convolution. The one-dimensional convolution is used to extract the connection information between adjacent layers of the MRI image, and the two-dimensional convolution is used to extract the information of a single layer of the MRI image. Using these two convolutions to replace the three-dimensional convolution can connect the information between adjacent layers of the MRI image, while reducing the number of network parameters and improving the training segmentation speed.

[0025] In step 3, an improved Wnet network is proposed based on the Unet network architecture. The T1-weighted image and the T2-weighted image are simultaneously input for dual-channel downsampling. The multimodal inputs are fused using an attention mechanism in the high-dimensional feature space. Different modalities of MRI images contain different information. The simultaneous input of the T1-weighted image and the T2-weighted image for downsampling yields features from multiple channels. The features from multiple channels have different importance. Features that are beneficial to the segmentation result are selected to enhance and features that are detrimental to the result are suppressed. The multi-channel features are weighted using an attention mechanism. After weighted fusion, upsampling is performed to restore the feature map and obtain the segmentation result.

[0026] Step 4: Add spatial constraints for the location of the acoustic tumor: Match each MRI image to an average standard spatial brain region. Construct a plane through the bottom and midline of the internal channels on both sides of the brain, construct another plane through the midline of the internal channels perpendicular to the bottom of the internal channels, and the coronal plane to obtain a three-dimensional coordinate system. Match each MRI image to this standard spatial coordinate system, and then mark the locations of acoustic tumors that appear during training in the standard space to obtain the high-frequency occurrence locations of acoustic tumors and construct a heat map. Then use the spatial heat map of acoustic tumor locations obtained from training as a constraint to check the acoustic tumor segmentation results and remove segmentation results with unreasonable locations.

[0027] The specific implementation described above is only one optimal way of implementing the present invention and is not intended to limit the patent scope of the present invention. Any equivalent structural or procedural transformations made using the spirit and principles of the present invention and the contents of the accompanying drawings should be included within the patent protection scope of the present invention.

Claims

1. A method for segmenting acoustic nerve schwannomas in multimodal MRI images, characterized in that, The method includes the following steps: Step 1, Image Preprocessing: Acquire the acoustic neuroma MRI dataset, and perform spatial resampling, image registration, and image grayscale thresholding preprocessing. Align the T1-enhanced weighted image and T2-weighted image with the same voxel size in space, and further generate the corresponding brain region / background region mask through grayscale thresholding and connected component information. Step 2: Generate training samples: Combine the brain region / background region mask image generated in Step 1 with the tumor ground truth annotation in the training samples, and simultaneously cut the image and ground truth. Input the cut blocks into the network for training to reduce the pressure on subsequent network training. Step 3: Training the network model: Based on the 3D convolution in the network model, a new type of convolution is used, which replaces the 3D convolution with a one-dimensional convolution and a two-dimensional convolution; the Wnet network is improved on the Unet network architecture, which simultaneously inputs the T1 enhanced weighted image and the T2 weighted image for downsampling, and uses an attention mechanism to perform fusion segmentation in the high-dimensional feature space. The network model is trained using the training samples generated in step 2, and the network model is used to perform acoustic tumor segmentation on multimodal MRI images. An improved 3D convolutional WNet framework is proposed. The encoder module contains four convolutional layers and a max-pooling layer, with 32, 64, 128, and 256 feature maps respectively. The attention mechanism module contains two fully connected layers and an average pooling layer, with 1024, 512, and 256 feature maps for different feature dimensions. The decoder module contains four deconvolutional layers and convolutional layers, with 256, 128, 64, and 32 feature maps respectively. In the convolutional layers, an improved 3D convolution is applied, including a 3×3×1 convolutional kernel and a 1×1×3 convolutional kernel. For all max-pooling layers, the pooling size is 2×2×2 with a stride of 2. For all deconvolutional layers, the deconvolutioned feature maps are weighted and combined with the corresponding features in the encoder module through the attention mechanism module. Step 4: Add spatial constraints for the location of the otitis media: During the training process, a spatial heatmap of the location of the otitis media in a standard space is constructed. The spatial heatmap of the location of the otitis media obtained in the training is used as a constraint to examine the segmentation results of the otitis media obtained in Step 3 and remove segmentation results with unreasonable locations.

2. The method for segmenting acoustic nerve schwannomas in multimodal MRI images as described in claim 1, characterized in that, In step 1, the resampling algorithm used is the bilinear interpolation algorithm, and the registration process uses the B-spline registration method with mutual information.

3. A method for segmenting acoustic nerve schwannomas in multimodal MRI images as described in claim 1 or 2, characterized in that, In step 3, a one-dimensional convolution and a two-dimensional convolution are used to replace the three-dimensional convolution. The one-dimensional convolution is used to extract the connection information between adjacent layers of the MRI image, and the two-dimensional convolution is used to extract the information of a single layer of the MRI image. Using these two convolutions to replace the three-dimensional convolution can connect the information between adjacent layers of the MRI image, while reducing the number of network parameters and improving the training segmentation speed.

4. A method for segmenting acoustic nerve schwannomas in multimodal MRI images as described in claim 1 or 2, characterized in that, In step 3, an improved Wnet network is proposed based on the Unet network architecture. The T1-weighted image and the T2-weighted image are simultaneously input for dual-channel downsampling. The multimodal inputs are fused using an attention mechanism in the high-dimensional feature space. Since different modalities of MRI images contain different information, the T1-weighted image and the T2-weighted image are simultaneously input for downsampling to obtain features of multiple channels. The features of multiple channels have different importance. Features that are beneficial to the segmentation result are selected and features that are detrimental to the segmentation result are suppressed. The multi-channel features are weighted using an attention mechanism. After weighted fusion, upsampling is performed to restore the feature map and obtain the segmentation result.

5. A method for segmenting acoustic nerve schwannomas in multimodal MRI images as described in claim 1 or 2, characterized in that, In step 4, each MRI image is matched to an average standard spatial brain region. A plane is constructed through the bottom and midline of the internal channels on both sides of the brain, and another plane is constructed through the midline of the internal channels perpendicular to the bottom of the internal channels, as well as the coronal plane, to obtain a three-dimensional coordinate system. Each MRI image is matched to this standard spatial coordinate system. Then, the locations of acoustic tumors that appear during the training process are marked in the standard space to obtain the high-frequency occurrence locations of acoustic tumors, and a heat map is constructed. The acoustic tumor location spatial heat map obtained from the training is used as a constraint to check the acoustic tumor segmentation results and remove segmentation results with unreasonable locations.

Images