An automatic scanning and positioning method for MRI head 3D images based on deep learning

Abstract

The present invention discloses an automatic scanning and positioning method for MRI head 3D images based on deep learning, comprising the following steps: step s1: collecting MRI head 3D images and manually marking them, dividing the manually marked MRI head 3D images into training set, verification set and test set, and then preprocessing the training set, verification set and test set respectively; step s2: building a 3D convolutional neural network model; step s3: gradually inputting the preprocessed training set into the 3D convolutional neural network model for training, and verifying it through the verification set; step s4: running the preprocessed test set through the trained 3D convolutional neural network model, comparing the output result with the manually marked MRI head 3D image and calculating the error. The present invention searches for feature points in the MRI head 3D image through the 3D convolutional neural network model, realizes automatic positioning of MRI scanning, and improves efficiency through automated processes while maintaining a small error.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to an automatic scanning and positioning method for MRI head 3D images based on deep learning. Background Art

[0002] Magnetic resonance imaging (MRI) is a high-end medical imaging technology. The advantages of MRI are harmlessness, imaging in any direction, and high tissue contrast. Therefore, head imaging is generally the preferred equipment. In the scanning of various parts of MRI, positioning must be performed first. The head scanning positioning requirements are: the cross-sectional scan is perpendicular to the sagittal plane, and passes through or is parallel to the scanning baseline formed by AC-PC, and the range includes the entire cranial brain; the sagittal scan is required to be parallel to the sagittal plane presented by the patient's AC-PC, and parallel to the longitudinal fissure of the brain in the cross section, and the range includes the entire cranial brain. Therefore, it is of great significance to locate the sagittal plane of the patient's scan. Among them, in the line drawing baseline AC-PC, AC is the anterior commissure (AC) point, which is a nerve fiber bundle (white matter) connecting the two hemispheres of the brain, and PC is the posterior commissure (PC) point, connecting the superior colliculus on both sides, adjacent to the oculomotor nucleus. In addition, the interpeduncular cistern (IC) vertex is also a feature point in the image, which can form a standard sagittal plane with the AC and PC points. Therefore, by finding the above three feature points, we have the AC-PC scanning baseline and the standard sagittal plane, which can solve the positioning problem of head scanning.

[0003] In clinical head scans, it is generally necessary to pre-scan one or more positioning phase sequences, and manually determine the required sagittal plane. Sometimes, the sagittal plane in the strict sense cannot be found; in sequence scans related to the AC-PC baseline, the operating technician will generally first scan a sagittal plane sequence, and find the AC point and PC point from the same or different layers as the scanning baseline. If the point cannot be found in some cases, it is necessary to continue scanning one or more sequences. The clinical positioning AC-PC baseline scan is as follows Figure 1 shown.

[0004] The disadvantages of manual positioning are self-evident: time-consuming, error-prone, and increased burden on the operator; it cannot achieve the advanced clinical application of MRI - fully automatic scanning. In some special patients or special cases, the patient's head is offset to the left or right. In this case, the operator may be able to find the AC and PC points, but it is difficult to find the accurate sagittal plane, which may lead to inaccurate scanning and waste of time.

[0005] The disadvantage of finding the AC and PC points in 2D images through a neural network model is that, more often than not, the AC and PC points do not exist in the same image at the same time during scanning, and the scanning not only refers to the AC-PC scanning baseline but also to the standard sagittal plane, so the application of the scheme is limited. Summary of the invention

[0006] The present invention aims to provide a method for solving the problem of fully automatic positioning of head scanning by combining 3D convolutional neural network technology with nuclear magnetic resonance 3D scanning technology.

[0007] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0008] A method for automatic scanning and positioning of MRI head 3D images based on deep learning, comprising the following steps:

[0009] Step s1: Acquire MRI head 3D images and manually mark them, divide the manually marked MRI head 3D images into a training set, a validation set, and a test set, and then preprocess the training set, the validation set, and the test set respectively;

[0010] Step s2: Build a 3D convolutional neural network model;

[0011] Step s3: gradually input the preprocessed training set into the 3D convolutional neural network model for training, and verify it through the verification set, set a training completion mechanism and save the trained 3D convolutional neural network model;

[0012] Step s4: Run the preprocessed test set through the trained 3D convolutional neural network model, compare the output results with the manually labeled MRI head 3D image and calculate the error.

[0013] Preferably, the manual marking is to manually mark feature points on the acquired MRI head 3D image, and the feature points include AC point coordinates, PC point coordinates and IC point coordinates.

[0014] Preferably, in step s1, the preprocessing includes normalization, image resizing and image rotation, and the size of the acquired MRI head 3D image received by the 3D convolutional neural network model is 128*128*128.

[0015] Preferably, the 3D convolutional neural network model includes 5 3D convolutional modules, 1 channel attention module and 3 fully connected layers. The 3D convolutional neural network model is output through the last fully connected layer. The 3D convolutional neural network model uses MSE as a convergence function, and the method for updating and optimizing the parameters of the 3D convolutional neural network model is the Adam gradient descent method.

[0016] Preferably, the generalization ability of the 3D convolutional neural network model is judged by the Loss function of the validation set, and the 3D convolutional neural network model parameters when the Loss function value is the smallest are always taken as the optimal parameters, and the Loss function is as follows:

[0017]

[0018] Among them, the subscript with pred represents the points output by the 3D convolutional neural network model, and the subscript with true represents the points in the manually labeled image.

[0019] Furthermore, in step s4, the output result includes the coordinates of the feature points, and the last fully connected layer of the 3D convolutional neural network model includes 9 neurons, and 9 numerical values ​​are output through the 9 neurons, and every 3 numerical values ​​represent the coordinates of 1 feature point.

[0020] The present invention has the following beneficial effects:

[0021] 1. The present invention can find the AC, PC and IC points in the MRI head 3D image, and based on the three points, form the sagittal plane and AC-PC form the scanning baseline. It is more reasonable and faster to apply to the head scanning positioning planning, and can adapt to the scanning positioning of patients in various complex situations; it can reduce the workload of MRI clinical operators and save the time of scanning planning;

[0022] 2. The present invention is applied to fully automatic MRI head scanning, and the process is fully automated without manual intervention;

[0023] 3. The present invention has good repeatability for patients undergoing reexamination and is very good for comparing recovery conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of scanning positioning parallel to the AC-PC baseline;

[0025] Figure 2 This is a structural diagram of the 3D convolutional neural network model;

[0026] Figure 3 This is the structure diagram of the 3D convolution module;

[0027] Figure 4 This is the structure diagram of the channel attention module;

[0028] Figure 5 This is the convergence trend diagram of the Loss function during the training process of the 3D convolutional neural network model;

[0029] Figure 6 Output image for the 3D convolutional neural network model. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0031] A method for automatic scanning and positioning of MRI head 3D images based on deep learning, comprising the following steps:

[0032] Step s1: Acquire MRI head 3D images and manually label them, divide the manually labeled MRI head 3D images into a training set, a validation set, and a test set, and then preprocess the training set, the validation set, and the test set, respectively.

[0033] The manual marking is to manually mark feature points on the acquired MRI head 3D image, and the feature points include AC point coordinates, PC point coordinates and IC point coordinates.

[0034] In step s1, the preprocessing includes normalization, image size adjustment and image rotation, and the size of the acquired MRI head 3D image received by the 3D convolutional neural network model is 128*128*128.

[0035] Step s2: Build a 3D convolutional neural network model.

[0036] like Figure 2 The 3D convolutional neural network model includes 5 3D convolutional modules, 1 channel attention module and 3 fully connected layers. The 3D convolutional neural network model is output through the last fully connected layer. The 3D convolutional neural network model uses MSE as the convergence function. The method for updating and optimizing the parameters of the 3D convolutional neural network model is the Adam gradient descent method.

[0037] The 3D convolution module is used to extract features in 3D space. As the depth of the network layer increases, the number of feature map channels increases. Each 3D convolution module is connected to a 3D maximum pooling layer to reduce the dimension of the feature map. After passing through 5 convolution modules and pooling layers, the dimension of the feature map becomes 4x4x4x128. Through this process, the features are constantly converging to the target area.

[0038] 3D convolution module such as Figure 3As shown. First, according to the principle of Google Inception, two 3x3 convolutions are equivalent to 5x5 convolutions, and three 3x3 convolutions are equivalent to 7x7 convolutions, with fewer parameters. Therefore, this design principle is also used in this module. The input feature map of the module is increased by a 3D convolution layer with a convolution kernel of 1 to increase the number of channels of the feature map. Then, the feature map Conv3D_3 is obtained by a 3x3x3 3D ​​convolution result, and a 3x3x3 3D ​​convolution is connected after Conv3D_3 to obtain the feature map Conv3D_5. A 3x3x3 3D ​​convolution is connected after Conv3D_5 to obtain the feature map Con3D_7x7x7. The feature maps Conv3D_3, Conv3D_5, and Con3D_7 are connected in the channel dimension to form a feature map that combines three field of view sizes. Finally, a 1x1x1 convolution operation is performed to reduce the number of feature map channels and fuse feature information. All 3D convolution operations in this module include Batch Normalization and LeakyRelu activation operations.

[0039] Channel attention module such as Figure 4 As shown in the figure, the module input extracts features through a 1x1x1 convolution layer, and the extracted feature map is ShortCut. AvgPool is used on the feature map ShortCut to reduce the dimension. The size of the feature map after dimensionality reduction is 1x1x1xC, where C represents the channel, and the feature map becomes C feature values. Then two 3D convolution layers are used for learning. At this time, the 3D convolution layer is completely equivalent to the fully connected layer. Finally, the Sigmoid activation function is used to convert the C feature values ​​into decimal values ​​between 0 and 1.0, which represent the ratio of the feature information of each channel on the feature map ShortCut. During the training process, the corresponding ratio of unimportant feature channels will become smaller and smaller, or even disappear, while the important feature channels will become larger and larger. The feature value is multiplied by the ShortCut feature map to activate the feature map ShortCut. Finally, the activated ShortCut is added to the input feature map Input of the module as the output of this module. This module can make the model output more accurate and accelerate the convergence of the model.

[0040] At the same time, the channel attention module is a mechanism for evaluating the amount of information in the feature map channels. This mechanism increases the ratio of channels with rich feature information, and reduces or even eliminates the ratio of channels with less feature information. This mechanism can make the model converge more accurately and quickly.

[0041] Step s3: gradually input the preprocessed training set into the 3D convolutional neural network model for training, and verify it through the verification set, set the training completion mechanism and save the trained 3D convolutional neural network model.

[0042] The generalization ability of the 3D convolutional neural network model is judged by the Loss function of the validation set, and the 3D convolutional neural network model parameters with the minimum Loss function value are always taken as the optimal parameters. The Loss function is as follows:

[0043]

[0044] Among them, the subscript with pred represents the points output by the 3D convolutional neural network model, and the subscript with true represents the points in the manually labeled image.

[0045] In the specific implementation process, training is performed on NVIDIA GPU 1080Ti. The number of images trained in each step is 2, that is, batch size = 2, and the number of iterations is 200, that is, epoch = 200. The initial learning rate is set to 1e-3, and the learning rate is gradually reduced by exponential descent during training. The descent method is to reduce the learning rate by 0.08 every 200 steps. During the training process, the convergence performance of the model is verified once every 10 steps through the validation set, and the generalization ability of the model is judged by the Loss of the validation set. During the entire training process, the model parameters with the smallest Loss during verification will always be saved as the optimal model parameters. The model convergence during this implementation training process is as follows Figure 5 , red represents the downward trend of Loss during training, and blue represents the downward trend of Loss during verification. Through observation, the model does not have overfitting phenomenon, and the convergence of Loss of training and verification reaches expectations.

[0046] Step s4: Run the preprocessed test set through the trained 3D convolutional neural network model, compare the output results with the manually labeled MRI head 3D image and calculate the error.

[0047] After the training is completed, the optimal model parameters are saved as the final training results of the model, the test set is input into the model in sequence, and the output result is the model prediction result, such as Figure 6 .

[0048] In step s4, the output result includes the coordinates of the feature points. The last fully connected layer of the 3D convolutional neural network model includes 9 neurons, and 9 values ​​are output through the 9 neurons. Every 3 values ​​represent the coordinates of 1 feature point. The coordinates of each point predicted by the model are compared with the coordinates of each point manually marked. Table 1 shows the average error between the manually marked coordinates and the model predicted coordinates, as follows:

[0049] Table 1 Average error

[0050]

[0051] The error of the coordinates of each point on each axis does not exceed 1.5 pixels, and the AC-PC scanning baseline angle is 4.5 degrees on average. Combined with the actual physical spacing between points in the MRI image plane (XY plane) and between layers (Z axis), the error is within the clinically acceptable range.

[0052] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, technicians familiar with the field may make various corresponding changes and deformations based on the present invention, but these corresponding changes and deformations should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for automatic scanning and positioning of MRI head 3D images based on deep learning, characterized in that: The following steps are involved: Step s1: Acquire MRI head 3D images and manually mark them, divide the manually marked MRI head 3D images into a training set, a validation set, and a test set, and then preprocess the training set, the validation set, and the test set respectively; Step s2: building a 3D convolutional neural network model; the 3D convolutional neural network model includes 5 3D convolutional modules, 1 channel attention module and 3 fully connected layers, each 3D convolutional module is connected to a 3D maximum pooling layer, the 3D convolutional neural network model is output through the last fully connected layer, the 3D convolutional neural network model uses MSE as a convergence function, and the method for updating and optimizing the parameters of the 3D convolutional neural network model is the Adam gradient descent method; Step s3: gradually input the preprocessed training set into the 3D convolutional neural network model for training, and verify it through the verification set, set a training completion mechanism and save the trained 3D convolutional neural network model; judge the generalization ability of the 3D convolutional neural network model through the Loss function of the verification set, and always take the 3D convolutional neural network model parameters with the minimum Loss function value as the optimal parameters, and the Loss function is as follows: Among them, the points with the subscript pred represent the output points of the 3D convolutional neural network model, and the points with the subscript true represent the points of the manually marked image; the manual marking is to manually mark the feature points of the collected MRI head 3D image, and the feature points include the AC point coordinates, the PC point coordinates and the IC point coordinates; Step s4: Run the preprocessed test set through the trained 3D convolutional neural network model, compare the output results with the manually labeled MRI head 3D image and calculate the error.

2. The method for automatic scanning and positioning of MRI head 3D images based on deep learning according to claim 1, characterized in that: In step s1, the preprocessing includes normalization, image size adjustment and image rotation, and the size of the acquired MRI head 3D image received by the 3D convolutional neural network model is 128*128*128.

3. The method for automatic scanning and positioning of MRI head 3D images based on deep learning according to claim 1, characterized in that: In step s4, the output result includes the coordinates of the feature points. The last fully connected layer of the 3D convolutional neural network model includes 9 neurons, and 9 values ​​are output through the 9 neurons. Every 3 values ​​represent the coordinates of 1 feature point.

Images