Medical image segmentation method and system based on scale-guided convolutional neural network

Abstract

The application discloses a medical image segmentation method and system based on a scale guide convolutional neural network, wherein a scale map learning module is added in the network structure to learn a scale map from a feature map; the scale map label is obtained by calculating and converting a boundary map of the original label of the medical image, and the learned scale map in the network is supervised correspondingly; and the network learns the scale map, calculates the offset to guide the receptive field of the convolution. The method can expand the receptive field of the convolution in the neural network, enables the network to better learn the in-class information in the global rather than being limited to local features, and can guide the convolution to better learn the boundary information and improve the segmentation effect of the boundary. The application can significantly improve the medical image segmentation effect.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to a medical image segmentation method and system, in particular to a medical image segmentation method and system based on a scale-guided convolutional neural network. BACKGROUND

[0002] With the rapid development of the convolutional neural network (CNN) in deep learning, its application in medical image segmentation has achieved good results, which can effectively help medical staff to perform auxiliary work of disease diagnosis and treatment, and greatly improves work efficiency. However, the traditional convolutional neural network has the following problems for medical image segmentation: (1) the receptive field of convolution is fixed and limited, and can only be used to extract local information, and cannot integrate global information; (2) the fitting effect of the network on the boundary, and many networks have general boundary segmentation effect in the segmentation task. SUMMARY

[0003] In order to solve the problems of limited receptive field of the existing neural network and poor overall lesion boundary segmentation effect, and improve the accuracy of lesion segmentation of the network, the application provides a medical image segmentation method and system based on a scale-guided convolutional neural network.

[0004] The technical scheme of the method of the application is that a medical image segmentation method based on a scale-guided convolutional neural network inputs a medical image to be segmented into the scale-guided convolutional neural network for image segmentation.

[0005] The scale-guided convolutional neural network comprises an encoder, a scale module and a decoder.

[0006] The scale module is arranged after the second-to-last layer of the encoder and comprises a feature processing layer, a scale map learning module and a scale-guided offset convolution module. The feature processing layer comprises a 3*3 convolution and a batch normalization layer. The scale map learning module is arranged after the feature processing layer and comprises a 3*3 convolution, a batch normalization layer, a 1*1 convolution and a 1*1 ReLU activation function layer. The scale-guided offset convolution module is arranged after the feature processing layer and the scale map learning module and comprises an offset calculation layer, a 3*3 deformable convolution and a batch normalization layer.

[0007] The output of the scale-guided offset convolution module is input into the decoder after the first-to-last layer of the encoder to obtain an image segmentation result.

[0008] The technical scheme of the system of the application is that a medical image segmentation system based on a scale-guided convolutional neural network comprises:

[0009] one or more processors;

[0010] a storage device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the medical image segmentation method based on the scale guided convolutional neural network.

[0011] The application adds a scale map learning module in the network structure to learn a scale map from a feature map; the application obtains a label of the scale map by calculating a boundary map of an original label of a medical image and a series of conversions, and performs corresponding supervision on the learned scale map in the network; the network learns the scale map, calculates an offset to guide a receptive field of convolution, and the method can expand the receptive field of convolution in the neural network, enables the network to better learn in-class information in the global rather than local features, and can guide the convolution to better learn boundary information and improve the segmentation effect of the boundary. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 a network structure based on the scale guided convolutional neural network of the embodiment of the application;

[0013] Figure 2 a scale module structure diagram of the embodiment of the application. DETAILED DESCRIPTION

[0014] In order to facilitate those skilled in the art to understand and implement the application, the application is further described in detail below in combination with the drawings and embodiments, and it should be understood that the described embodiments are only used to illustrate and explain the application, and are not used to limit the application.

[0015] Please see Figure 1 The medical image segmentation method based on the scale guided convolutional neural network provided by the application inputs a medical image to be segmented into the scale guided convolutional neural network to perform image segmentation.

[0016] The scale guided convolutional neural network of the embodiment includes an encoder, a scale module and a decoder.

[0017] Please see Figure 2 The scale module of the embodiment is arranged after the second-to-last layer of the encoder, and is composed of a feature processing layer, a scale map learning module and a scale guided offset convolution module.

[0018] The feature processing layer of the embodiment is composed of a 3*3 convolution and a batch normalization layer; the scale map learning module is arranged after the feature processing layer, and is composed of a 3*3 convolution, a batch normalization layer, a 1*1 convolution and a 1*1 ReLU activation function layer.

[0019] The scale-guided offset convolution module of the embodiment is arranged after the feature processing layer and the scale map learning module, and is composed of an offset calculation layer, a 3*3 deformable convolution, and a batch normalization layer;

[0020] The output of the scale-guided offset convolution module of the embodiment is input into the decoder after passing through the inverse first layer of the encoder to obtain an image segmentation result.

[0021] The offset calculation layer in the scale-guided offset convolution module of the embodiment calculates the offset of each pixel point, up, down, left and right, respectively, by calculating the corresponding offset through the predicted scale map, and accumulates the same category of points to calculate the offset of the four convolution points, up, down, left and right. The above, down, left and right four offset values are used to calculate the offset of the four convolution points, upper left, lower left, upper right and lower right. The final offset of the convolution kernel is generated as the parameter of the deformable convolution to guide the receptive field of the deformable convolution, and the output feature map is input into the next convolution layer.

[0022] The scale-guided convolutional neural network of the embodiment is a trained network; the training process includes the following steps:

[0023] Step 1: Construct a training sample set, calculate the scale map label according to the image label in the training sample set, and generate the scale map label after Gaussian filtering processing;

[0024] The specific implementation of step 1 includes the following sub-steps:

[0025] Step 1.1: Obtain the original label size, and newly create a scale map with the same size as the original label size;

[0026] Step 1.2: Traverse all pixel points of the original label, if the value of the current pixel point is inconsistent with the value of the pixel point below or to the left, it means that the pixel point is at the boundary junction, and the pixel point value at the same coordinate in the scale map is set to 10, and the rest of the pixel point value is set to 0;

[0027] Step 1.3: Perform Gaussian filtering operation on the scale map to smooth it and obtain the final scale map label.

[0028] Step 2: Input the scale map label and the image, image label into the scale-guided convolutional neural network for training;

[0029] The scale map label is obtained after the scale map learning module, and the MSE loss between the predicted scale map and the scale map label is calculated and added to the original loss function, and the network parameter information is updated by back propagation;

[0030] The MSE loss is: wherein y i is the value of the scale map label at the i-th pixel point, is the value of the predicted scale map at the i-th pixel point; n refers to the number of pixel points.

[0031] The present application calculates the scale map label according to the image label in the training sample, sets the values of the points at the boundary to 10, and generates the scale map label after Gaussian filtering processing; adds a scale map learning module to the encoder part of the neural network, outputs the predicted scale map by convolution and other operations on the feature map; uses the above calculated scale map label as a supervisory signal, adds the MSE loss function between the predicted scale map and the scale map label into the loss function for training; adds a scale guide offset convolution module after the scale map learning module in the encoder part of the neural network, calculates and generates the receptive field offset of the corresponding deformable convolution by the predicted scale map; uses the offset to guide the receptive field of the deformable convolution, so that the deformable convolution can learn the global information with the largest possible receptive field without crossing the boundary in the scale map; the feature map output by the deformable convolution is then input into the next convolution layer for training; the combination of the basic neural network and the above modules, supervision and loss function is used as a segmentation network to segment the medical image to be segmented. The present application first calculates the corresponding scale map label through the label, then adds a scale map learning module in the neural network, learns the scale map based on the feature map and uses the scale map label as a supervisory signal, and then generates the corresponding offset using the learned scale map to guide the following deformable convolution, which can widen the receptive field of the deformable convolution as much as possible, so that the network can better learn the global information, and the receptive field of the convolution guided by the scale map can not cross the boundary, so the boundary information is fused in the learning, thereby improving the segmentation effect at the boundary.

[0032] The present application is further described through specific experiments, and the experimental results are shown in the following table:

[0033]

[0034] As can be seen from the table, based on the three benchmark networks, the segmentation effect is obviously improved after adding the scale module of the present embodiment, and the Dice index of the segmentation is improved by about 2.12% after adding the scale module to the encoder part of SANet. Based on the four benchmark networks, the segmentation effect is obviously improved after adding the scale module, and the average Dice index is improved by about 1.23% after adding the scale module to the encoder part of nnUNet, and the Dice index is improved by 1.91% after adding to U-Net.

[0035] It should be understood that the above description is merely a detailed explanation of the preferred embodiments and is not intended to limit the patent protection scope of the present application. Any modification or alternation made by those skilled in the art without departing from the scope of the present application shall fall within the patent protection scope of the present application. The patent protection scope of the present application shall be subject to the appended claims.

Claims

1. A method for medical image segmentation based on scale-guided convolutional neural network, characterized in that: Input a medical image to be segmented into a scale-guided convolutional neural network based on the image segmentation; The scale-guided convolutional neural network based on the scale-guided convolutional neural network comprises an encoder, a scale module and a decoder; The scale module is arranged after the second last layer of the encoder and comprises a feature processing layer, a scale map learning module and a scale-guided offset convolution module; the feature processing layer comprises a 3*3 convolution and a batch normalization layer; the scale map learning module is arranged after the feature processing layer and comprises a 3*3 convolution, a batch normalization layer, a 1*1 convolution and a 1*1 ReLU activation function layer; the scale-guided offset convolution module is arranged after the feature processing layer and the scale map learning module and comprises an offset calculation layer, a 3*3 deformable convolution and a batch normalization layer; The output of the scale-guided offset convolution module is input into the decoder after the first last layer of the encoder to obtain an image segmentation result; The scale-guided convolutional neural network based on the scale-guided convolutional neural network is a trained network; The training process comprises the following steps: Step 1: constructing a training sample set, calculating a scale map label according to an image label in the training sample set, and generating the scale map label after Gaussian filtering processing; Step 1.1: obtaining an original label size, and newly creating a scale map with the same size as the original label size; Step 1.2: traversing all pixel points of the original label, if the value of a current pixel point is inconsistent with the value of a pixel point below or to the left of the current pixel point, it is indicated that the current pixel point is at a boundary junction, all pixel point values of the same coordinates in the scale map are set to 10, and the rest of the pixel point values are set to 0; Step 1.3: performing Gaussian filtering operation on the scale map to smooth the scale map, and obtaining a final scale map label; Step 2: inputting the scale map label and the image and the image label into the scale-guided convolutional neural network based on the scale-guided convolutional neural network for training; The scale map label is input into the scale map learning module to obtain a predicted scale map; an MSE loss between the predicted scale map and the scale map label is calculated, added to an original loss function, and back-propagated to update parameter information of the network; The MSE loss is: MSE = ,in The scale map label is in the first The value at each pixel. Is the prediction scale map in the 1st The value at each pixel; n Refers to the number of pixels.

2. The method of claim 1, wherein the method is a method of medical image segmentation based on a scale-guided convolutional neural network. The offset calculation layer in the scale-guided offset convolution module calculates corresponding offsets by the predicted scale map, calculates how many points of the same category are above, below, to the left and to the right of each pixel point, and accumulates the offsets to calculate offsets of four convolution points, i.e., the upper left, lower left, upper right and lower right, of each pixel point; the offsets of the four convolution points are used to generate final offsets of a convolution kernel, which are used as parameters of a deformable convolution to guide a receptive field of the deformable convolution, and the output feature map is input into a subsequent convolution layer. 3.A medical image segmentation system based on scale-guided convolutional neural network, characterized by, One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the medical image segmentation method based on the scale-guided convolutional neural network according to any one of claims 1 to 2. ​

Images