A super-resolution reconstruction method for head 3D MRI

Abstract

The present invention discloses a method for super-resolution reconstruction of head 3D MRI. The method first selects high-resolution MRI image data, pre-processes the selected MRI data to obtain small data blocks, and uses the small data blocks as training data; then establishes a 3D MRI super-resolution reconstruction model based on a convolutional neural network and a Transformer encoder, trains the model with the small data blocks as input, and obtains a 3D MRI super-resolution reconstruction model; finally, three axial MRI data are used for processing to obtain cube cavity data, and the cavity data is input into the trained model to obtain a small cube filled with the cavity, and the small cubes filled with the cavity are spliced ​​to form a complete data space. The method has a small model scale, fast image reconstruction speed, high reconstructed image quality, and provides more precise data for medical image processing.

Description

Technical Field

[0001] The invention relates to the technical field of medical image processing, and in particular to a head three-dimensional MRI super-resolution reconstruction method. Background Art

[0002] During magnetic resonance imaging (MRI), the signal-to-noise ratio and spatial resolution of the image are affected due to the limitations of the image acquisition sensor. On the other hand, due to the limitations of image data storage space, some image data that are not closely related to diagnostic information are omitted during image acquisition to reduce the storage pressure of the hospital. For example, in the MRI diagnosis of nasopharyngeal carcinoma, doctors often obtain clinical image information from three views: coronal, sagittal, and transverse. In order to save storage space, this type of image data has a higher on-chip resolution to meet the doctor's diagnostic needs, and the image layer spacing is larger to save storage space. The layer spacing is usually 3-10mm. In the image processing task using only one axial data, for current medical image machine vision tasks, in the frequently used 3D convolution operation, due to the influence of the image layer spacing, the real space covered by each convolution kernel is not a cube, but a cuboid with a very long side. The spatial resolution of the image extracted by the convolution kernel in different axes is different, resulting in a decrease in model performance. When using three-axis data for image processing tasks, if the image information of the three azimuth planes cannot be effectively integrated, it will affect the subsequent image processing tasks, such as affecting the experimental accuracy of image segmentation and disease classification problems. It is necessary to design a three-dimensional medical image super-resolution reconstruction method to improve the analysis accuracy of three-dimensional medical images. Summary of the invention

[0003] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a head three-dimensional MRI super-resolution reconstruction method, which has a small model scale, fast image reconstruction speed and high reconstructed image quality.

[0004] The technical solution for achieving the purpose of the present invention is:

[0005] A head three-dimensional MRI super-resolution reconstruction method specifically comprises the following steps:

[0006] 1) Select high-resolution MRI image data of the head, process the data, remove the black background area data, and use it as the data range of step 2);

[0007] 2) Preprocessing the MRI data selected in step 1), sliding and selecting a cube data block on the data, decomposing the obtained cube data block into six surface images and internal 3D data blocks covered by the surface data, and the obtained data is used as the model training data in step 3);

[0008] 3) Establishing a 3D MRI super-resolution reconstruction model based on a convolutional neural network and a Transformer encoder, using the data blocks described in step 2) as network training data to obtain a 3D MRI super-resolution reconstruction model;

[0009] 4) Select three axial MRI data, align the three axial data, cross the three directional plane data to obtain the cavity data block, and decompose the cavity data block to obtain 6 planes;

[0010] 5) Input the data obtained in step 4) into the model established in step 3) to obtain output data;

[0011] 6) Reintegrate the output data obtained in step 5), select the position of the cavity data block in step 4), and reconstruct the entire data space.

[0012] In the step 1), the selected data is high-resolution MRI data with a layer interval of zero, and the data is a DICOM format image, and the real space size of each pixel is 1mm×1mm×1mm; the specific operation of removing the black background area data is first to convert the DICOM data into bitmap data, convert the pixel value range to within 0-1, and then select a threshold of 0.1 to remove the noise pixel information in the background area.

[0013] The step 2) is to slide the data block on the high-resolution data, and the sliding step should be (target MRI image layer spacing / target MRI image pixel actual size-1); for the six surface images and the internal 3D data blocks covered by the surface data, the six surface images are used as input data for step 3) model training, and the internal cube data blocks covered by the surface data are used as the target output of step 3) model training.

[0014] In the step 3), the 3D MRI super-resolution reconstruction model based on convolutional neural network and Transformer encoder is composed of four parts: linear transformation layer, CNN layer, DNN layer and Transformer encoder layer. The model data is input into the linear transformation transformation layer and CNN layer in parallel. After the data is processed by the linear transformation layer, it is input into the Transformer encoder layer; after being processed by the Transformer encoder layer and CNN layer, it is integrated by the DNN layer to output the final result; wherein the CNN layer is composed of convolution, pooling, activation and residual connection structure; the Transformer encoder layer is composed of multiple Transformer encoders connected in series, and each encoder is composed of two sub-layer connection structures, The first sub-layer connection structure includes a multi-head self-attention sub-layer, a normalization layer and a residual connection, and the second sub-layer connection structure includes a feed-forward fully connected sub-layer, a normalization layer and a residual connection; the function of the linear transformation layer is to convert multi-dimensional data into one dimension, the function of the CNN layer is to extract image contours and partial texture information, the function of the Transformer encoder is to extract texture information, and the function of the DNN layer is to integrate information; after the model is built, the data of step 2) is input into the model for training to obtain an MRI super-resolution reconstruction model. During the training process, the 3D data blocks obtained by the model are compared with the real data blocks, and the absolute error and gradient error are used as the loss function of the loss value between them. The loss value is continuously iterated through the back propagation algorithm to optimize the parameters in the model.

[0015] The step 4) specifically records the spatial position relationship of each image of the transverse, coronal and sagittal planes during the imaging process, establishes a spatial surface equation for all three axial MRI slice images, and obtains a cavity cube generated by the intersection of any two adjacent transverse sections, any two adjacent coronal planes and any two adjacent sagittal planes by solving the equation. The data format required for input in step 3) is obtained through the obtained cavity cube.

[0016] The step 6) is specifically to combine the data block obtained in step 5) with the cavity data obtained in step 4) to complete the cavity filling in step 4), and then restore the combined data to the position where the cavity is generated in step 4).

[0017] Beneficial effects: The present invention provides a head three-dimensional MRI super-resolution reconstruction method, which has a small model scale, fast image reconstruction speed, high reconstructed image quality, and provides more precise data for medical image processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Provide a technical roadmap for super-resolution reconstruction of head 3D MRI;

[0019] Figure 2It is a schematic diagram of the intersection of coronal, sagittal and transverse images;

[0020] Figure 3 This is a schematic diagram of the reconstruction area;

[0021] Figure 4 Schematic diagram of the neural network for 3D MRI super-resolution reconstruction. DETAILED DESCRIPTION

[0022] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited thereto.

[0023] like Figure 1 As shown in the figure, the technical roadmap of head 3D MRI super-resolution reconstruction mainly includes the following steps:

[0024] 1) Three axial MRI image scans

[0025] A magnetic resonance imaging device was used to collect MRI image data of the real head in three planes: coronal, sagittal and transverse.

[0026] 2) Three-axis MRI image registration

[0027] During the imaging process, the magnetic resonance device will record the spatial position relationship of each image, and establish a spatial surface equation for all three axial MRI slice images. By solving it, we can get Figure 2 The cavity cube is produced by the intersection of any two adjacent transverse sections, any two adjacent coronal planes and any two adjacent sagittal planes.

[0028] 3) Feature extraction

[0029] like Figure 3 As shown in the figure, the cavity cube produced by the intersection of any two adjacent cross sections, any two adjacent coronal planes, and any two adjacent sagittal planes is disassembled to obtain 6 small plane images as data input for the three-dimensional MRI super-resolution reconstruction model based on convolutional neural network and Transformer encoder.

[0030] 4) 3D MRI super-resolution reconstruction model based on convolutional neural network and Transformer encoder

[0031] like Figure 4The figure shows a schematic diagram of the neural network for 3D MRI super-resolution reconstruction. The model is mainly composed of CNN convolution module, linear transformation module, Transformer encoder module and DNN deep neural network module. The data input into the model is divided into two parallel parts, CNN module and linear transformation module. The data of the linear transformation module is then input into the Transformer encoder module. Finally, the results of the two parts are input into the DNN for feature fusion and output of the target result.

[0032] In the CNN module, since the size of the input image is a small image such as 5×5, 7×7, 9×9 or 11×11, a convolution kernel of size 3×3 is selected during the convolution process, which can not only extract the correlation information between adjacent pixels but also take into account the size of the input image.

[0033] The Transformer model originally proposed consists of 6 encoders and 6 decoders. Usually only the encoder is used in image processing tasks. Since the Transformer encoder requires the input to be a vector, for the input data, the linear transformation module is first used to convert the data of the 6 faces into 6 one-dimensional sequences, and then input into the Transformer encoder. After experiments, this model uses 8 series-connected Transformer encoders. The encoder consists of a multi-head attention layer and a feedforward layer. In the multi-head attention mechanism, the model will focus on the correlation between different pixel intervals and extract and migrate the textures in the six faces covering the cavity.

[0034] Since a part of the image space information will be lost in the model when a two-dimensional image is converted into a one-dimensional sequence of all pixels when using the Transformer encoder, this model combines the convolution and Transformer encoders to combine the advantages of both. In the entire model, CNN provides the main image contour generation, and the Transformer part is responsible for the extraction, migration and regression of some high-frequency texture information. Compared with a single CNN structure network, the model with the Transformer encoder has a stronger ability to extract texture features of non-continuous image fragments.

[0035] The model uses absolute error and gradient error as the loss function for training, and uses Adam as the optimizer for the model. In the process of model training, the transfer learning method is used. The experiment first trains a model with only one layer of Transformer encoder, and then uses the existing model parameters to train a model with two layers of Transformer encoder layers. The depth of the Transformer encoder layer is continuously increased, and finally a model with a Transformer encoder layer depth of 8 is trained. Compared with the model that trains 8 layers of Transformer encoders at one time, this training strategy has a faster training convergence speed.

[0036] 5) Image 3D reconstruction

[0037] The 3D data blocks output by the 3D MRI super-resolution reconstruction model based on convolutional neural networks and Transformer encoders are combined with the input data of the model to obtain larger data blocks. This process fills the cavities inside the original cavity data and places the data blocks according to the original data positions to complete the entire 3D super-resolution reconstruction process.

Claims

1. A method for super-resolution reconstruction of head three-dimensional MRI, characterized in that: The steps include: 1) Select high-resolution MRI image data of the head, process the data, remove the black background area data, and use it as the data range of step 2); 2) Preprocessing the MRI data selected in step 1), sliding and selecting a cube data block on the data, decomposing the obtained cube data block into six surface images and internal 3D data blocks covered by the surface data, and the obtained data is used as the model training data in step 3); 3) Establishing a 3D MRI super-resolution reconstruction model based on a convolutional neural network and a Transformer encoder, using the data blocks described in step 2) as network training data to obtain a 3D MRI super-resolution reconstruction model; 4) Select three axial MRI data, align the three axial data, cross the three directional plane data to obtain the cavity data block, and decompose the cavity data block to obtain 6 planes; 5) Input the data obtained in step 4) into the model established in step 3) to obtain output data; 6) Reintegrate the output data obtained in step 5), select the position of the cavity data block in step 4), and reconstruct the entire data space; In the step 3, the 3D MRI super-resolution reconstruction model based on convolutional neural network and Transformer encoder is composed of four parts: linear transformation layer, CNN layer, DNN layer and Transformer encoder layer. The model data is input into the linear transformation layer and CNN layer in parallel. The data is input into the Transformer encoder layer after being processed by the linear transformation layer. After being processed by the Transformer encoder layer and the CNN layer, the DNN layer integrates and outputs the final result; the CNN layer consists of convolution, pooling, activation and residual connection structures; the Transformer encoder layer is composed of multiple Transformer encoders in series, each encoder consists of two sub-layer connection structures, the first sub-layer connection structure includes a multi-head self-attention sub-layer, a normalization layer and a residual connection, and the second sub-layer connection structure includes a feed-forward fully connected sub-layer, a normalization layer and a residual connection; the function of the linear transformation layer is to convert multi-dimensional data into one dimension, the function of the CNN layer is to extract image contours and partial texture information, the function of the Transformer encoder is to extract texture information, and the function of the DNN layer is to integrate information; after the model is constructed, the data in step 2) is input into the model for training to obtain an MRI super-resolution reconstruction model. During the training process, the 3D data blocks obtained by the model are compared with the real data blocks, and the absolute error and gradient error are used as the loss function of the loss value between them. The loss value is continuously iterated through the back propagation algorithm to optimize the parameters in the model; The step 4) specifically records the spatial position relationship of each image of the transverse, coronal and sagittal planes during the imaging process, establishes a spatial surface equation for all three axial MRI slice images, and obtains a cavity cube generated by the intersection of any two adjacent transverse sections, any two adjacent coronal planes and any two adjacent sagittal planes by solving the equation. The data format required for input in step 3) is obtained through the obtained cavity cube.

2. A head three-dimensional MRI super-resolution reconstruction method according to claim 1, characterized in that: In the step 1), the selected data is high-resolution MRI data with a layer interval of zero, and the data is a DICOM format image, and the real space size of each pixel is 1mm×1mm×1mm; the specific operation of removing the black background area data is first to convert the DICOM data into bitmap data, convert the pixel value range to within 0-1, and then select a threshold of 0.1 to remove the noise pixel information in the background area.

3. The head three-dimensional MRI super-resolution reconstruction method according to claim 1, characterized in that: The step 2) is to slide the data block on the high-resolution data, and the sliding step should be (target MRI image layer spacing / target MRI image pixel actual size-1); for the six surface images and the internal 3D data blocks covered by the surface data, the six surface images are used as input data for step 3) model training, and the internal cube data blocks covered by the surface data are used as the target output of step 3) model training.

4. The head three-dimensional MRI super-resolution reconstruction method according to claim 1, characterized in that: The step 6) is specifically to combine the data block obtained in step 5) with the cavity data obtained in step 4) to complete the cavity filling in step 4), and then restore the combined data to the position where the cavity is generated in step 4).

Images