Full-region learning brain glioma segmentation method based on multi-layer reversal calibration

Abstract

The invention relates to a full-region learning brain glioma segmentation method based on multilayer reversal calibration, and belongs to the field of image processing. The method comprises the following steps: data construction, data preprocessing, an image segmentation network, a multi-scale feature fusion mechanism, a network training module, reversal calibration and discrete region learning, and building of a training model, and building of a network model based on the multi-scale feature fusion mechanism by using a Pytorch deep learning framework in combination with preprocessed data. The method comprises the following steps of: firstly, obtaining an initial segmentation result by training a network model, reversing a background and a foreground of each layer through a reversal calibration step, highlighting an area of a segmentation target, calibrating integral area learning in combination with a loss function, and finally focusing a discrete area by applying opening operation and closing operation to generate a precise brain glioma segmentation result. According to the method, the segmentation efficiency can be improved, and the problems of fuzzy tumor boundary and tumor region discretization in brain glioma segmentation can be solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and specifically provides a method for segmenting gliomas by full-region learning based on multi-layer reverse calibration Background Art

[0002] Gliomas are the most common type of malignant brain tumors and the primary malignant tumors with the highest incidence in the adult central nervous system. The annual incidence is approximately 6.4 per 100,000 cases, accounting for 50.1% of all primary malignant tumors in the central nervous system

[0003] The progress of medical imaging technology provides data support for diagnosis and treatment. The development of medical image segmentation algorithms improves the diagnostic efficiency, reduces the deviation of doctors' subjective judgments, and provides the possibility for personalized treatment. The morphology of gliomas is diverse, and it is difficult to distinguish the tumor boundary from the surrounding tissues, which is also the difficulty in glioma segmentation

[0004] Existing methods for solving the boundary segmentation of gliomas include: combining depth information and boundary information, and capturing local details by shape boundary perception and boundary prior knowledge guidance. These methods can assist in processing the boundary information of gliomas, but they rely on extracting abstract features to process the boundary information, resulting in problems such as boundary localization deviation, complex parameter adjustment, and fixed matching. The present invention designs a method for segmenting gliomas by full-region learning based on multi-layer reverse calibration, extracts the segmentation region of gliomas separately, and constrains the accuracy of region segmentation. The parameters therein can be automatically learned by the algorithm without additional parameter settings to more accurately segment the tumor region Summary of the Invention

[0005] The purpose of the present invention is to propose a method for segmenting gliomas by full-region learning based on segmentation map boundary calibration to solve the limitations of existing glioma image boundaries being blurred, inaccurate segmentation region positioning, and difficult segmentation, and improve the accuracy of glioma segmentation

[0006] To achieve the above purpose, the technical solutions adopted by the present invention are as follows

[0007] A method for segmenting gliomas by full-region learning based on multi-layer reverse calibration, specifically including the following steps

[0008] Step S1: Construct a database and process the data set

[0009] Step S2: Integrate the multi-scale feature mechanism

[0010] Step S3: Train the network model

[0011] Step S4: Perform reverse calibration

[0012] Step S5: Learn the discrete region.

[0013] Further, step S1 is specifically as follows:

[0014] Collect the MR image datasets of glioma patients from multiple medical institutions. First, perform skull stripping on the collected datasets, use FSL tools to extract the brain tissues, and use FSL tools to perform co-registration on the extracted brain tissues, and co-register the brain tissues to the MN152 template.

[0015] The dimension of the initially collected dataset is 244×244×155. First, process the images according to the intensity of the 155-dimensional images, normalize the intensity of the images to between 0 and 1, adopt min-max normalization, obtain the maximum and minimum values of the pixels of each image by identifying the image matrix, and then use the following normalization formula to map the image intensity range to between 0 and 1.

[0016]

[0017] Secondly, traverse the images, loop through each image slice, use numpy functions to calculate the pixel values of the images. When the sum of the pixel values of the image is 0, it means that the image slice is all background and does not contain the segmentation target, and the current image slice is excluded. When the sum of the pixel values of the image is greater than 0, it means that the image slice contains the segmentation target excluding the background, and the current image slice needs to be retained. The retained image slices are used to detect the brain region by scanning the pixel values in a top-down manner, and the redundant background regions are ignored. After removing the background interference, the dimension of each image slice is 192×192. Each processed patient contains four modalities of MRI imaging: FLAIR, T1, T1-ce, and T2;

[0018] Further, step S2 is specifically as follows:

[0019] Divide and merge the total features to facilitate the downward transmission of the original feature information; refer to Figure 3, the multi-scale feature fusion mechanism consists of four parallel branches. The number and arrangement of convolutional layers in each branch are different. The first branch only contains a 3×3 convolutional layer with a dilation rate of 1, and then outputs features. The second branch first performs a 3×3 convolution with a dilation rate of 3, fuses the convolution result with the features output by the first branch, then follows a 1×1 convolution, and then outputs features. The third branch first performs a 3×3 convolution with a dilation rate of 1, then follows a 3×3 convolution with a dilation rate of 3, fuses the convolution result with the features output by the second branch, then follows a 1×1 convolution, and then outputs features. The fourth branch first performs a 3×3 convolution with a dilation rate of 1, then follows a 3×3 convolution with a dilation rate of 3, then follows a 3×3 convolution with a dilation rate of 1, fuses the convolution result with the features output by the third branch, then follows a 1×1 convolution, and then outputs features. The features output by each branch are all fused together as the final output;

[0020] Further, step S3 is specifically as follows:

[0021] Based on the encoder-decoder mechanism and the multi-scale feature fusion mechanism, a new backbone network is obtained. The preprocessed data is used as the input of the network, and the glioma segmentation result is reconstructed through multi-layer feature extraction, and the loss function is calculated.

[0022] The segmentation network is an encoder-decoder structure. The same multi-scale feature fusion mechanism is embedded in each layer of the encoder and decoder. A supervision mechanism is set in the middle layer to assist in reconstructing accurate segmentation results. The supervision mechanism in the middle layer provides the ground truth for the segmentation results reconstructed in the middle layer, and uses the loss function to constrain the similarity between the reconstruction results of each layer and the ground truth. The calculation of the loss function is as follows:

[0023]

[0024] G Rec =Decoder(G i ), i = 1, 2, 3, 4 (3)

[0025]

[0026] Finally, the segmentation result of glioma is output, and the loss function is calculated by comparing the segmentation result with the corresponding ground truth. The calculation of the loss function is as follows:

[0027]

[0028] Further, step S4 is specifically as follows:

[0029] The glioma segmentation result is obtained through network training. The inverse function is used to highlight the segmentation target, and the original features are calibrated through feature combination.

[0030] First, reduce the scale of the current feature map by bilinear interpolation for the segmentation results of each network layer to achieve the focus of the current feature information. Secondly, activate the feature map information using the sigmoid activation function for the compared feature information. The numerical range of the activated feature map is between 0 and 1. The closer the value is to 1, the higher the information content of the feature map; otherwise, the information content is low. Then, use the reverse function to retain the features with high information content and suppress the features with low information content in the feature map. Thirdly, expand the original features provided by each layer and combine the reverse information of the features with the original information in a multiplicative manner, and enhance the high-throughput information based on the original information to calibrate the segmentation region. The reverse calibration of each layer's output features constrains the correlation between the segmentation region and the ground truth through the loss function. Finally, the combined feature map undergoes convolution and ReLU activation, and the size of the feature map is restored in the same way of bilinear interpolation and passed down layer by layer. The calculation of the loss function is as follows:

[0031]

[0032] Further, step S5 is specifically as follows:

[0033] First, use opening operation and closing operation to separate the continuous region and discrete region of glioma. The opening operation consists of a dilation layer and an erosion layer. The erosion layer is executed first and then the dilation layer. Both the dilation layer and the erosion layer consist of two steps: convolution and pooling. The convolution operation expands the segmentation result with a 3×3 convolution kernel, and then the pooling operation extracts important features. The closing operation consists of a dilation layer and an erosion layer. The dilation layer is executed first and then the erosion layer. The convolution and pooling operations are the same as those in the opening operation. The discrete region learning aims to focus on discrete points by constraining the region between the segmentation result and the ground truth through the loss function. The calculation of the loss function is as follows;

[0034]

[0035] During the training of the segmentation network, there are four major parts of the loss function, namely the feature reconstruction loss function, the segmentation result loss function, the reverse calibration loss function, and the discrete region learning loss function. The total loss function is as follows:

[0036]

[0037] where α , β , γ and λ are hyperparameters during model training and can adjust the weights of the loss function according to actual needs.

[0038] The combination of the loss function and the discrete region learning method provides a new and efficient glioma segmentation algorithm, which can accurately segment the tumor core region, tumor enhancement region and whole tumor region of glioma, providing a basis for clinicians to diagnose glioma and make reasonable treatment decisions.

[0039] The beneficial effects of the present invention are as follows: on the premise of ensuring that the segmentation network realizes glioma segmentation, a multi-layer reverse calibration mechanism is used to strengthen the learning degree of the glioma segmentation region and improve the accuracy of regional learning. Then, discrete region learning is used to connect the discrete pixel points of glioma into regions to expand the discrete points, improving the overall segmentation accuracy of glioma and providing a basis for clinicians to diagnose glioma and make reasonable treatment decisions. Brief Description of the Drawings

[0040] Figure 1 It is a schematic diagram of the overall structure of the loss function calculation and each execution step of the present invention;

[0041] Figure 2 It is a schematic diagram of the logical relationship between the execution steps of the present invention;

[0042] Figure 3 It is a schematic diagram of the multi-scale feature fusion of the present invention. Detailed Embodiments

[0043] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the embodiments of the present invention.

[0044] Please refer to Figures 1-3 , in this embodiment, a glioma segmentation method based on multi-layer reverse calibration and full-region learning is provided. This solution mainly includes the following steps: data construction, data preprocessing, image segmentation network, multi-scale feature fusion mechanism, network training module, boundary calibration and discrete region learning, so as to obtain the segmentation result of glioma. The logical relationship between each step is shown in Figure 2 ; specifically, it includes the following steps:

[0045] Step S1: Construct a database and process the data set:

[0046] Collect the MR image data set of glioma patients from multiple medical institutions. First, perform skull removal on the collected data set, use the FSL tool to extract the brain tissue, and use the FSL tool to perform common registration on the extracted brain tissue, and register the brain tissue to the MN152 template.

[0047] The dimension of the initially collected dataset is 244×244×155. First, process the images according to the intensity of the 155-dimensional images, normalize the intensity of the images to between 0 and 1, using min-max normalization. By identifying the image matrix, obtain the maximum and minimum values of each pixel in each image, and then use the following normalization formula to map the image intensity range to between 0 and 1.

[0048]

[0049] Secondly, traverse the images, loop through each image slice, use numpy functions to calculate the pixel values of the images. When the sum of the pixel values of an image is 0, it means that the image slice is all background and does not contain the segmentation target, and the current image slice is removed. When the sum of the pixel values of an image is greater than 0, it means that the image slice contains the segmentation target except for the background, and the current image slice needs to be retained. Use a top-down method to detect the pixel values of the retained image slices to scan out the brain region, and the redundant background regions are ignored. After removing the background interference, the dimension of each image slice is 192×192. Each processed patient contains MRI images of four modalities: FLAIR, T1, T1-ce, and T2;

[0050] Step S2: Multiscale feature fusion mechanism:

[0051] Divide and merge the total features to facilitate the downward transfer of the original feature information; refer to Figure 3 , the multiscale feature fusion mechanism consists of four parallel branches. The number and arrangement of convolutional layers in each branch are different. The first branch only contains a 3×3 convolutional layer with a dilation rate of 1, and then outputs the features immediately. The second branch first performs a 3×3 convolution with a dilation rate of 3, fuses the convolution result with the features output by the first branch, then follows a 1×1 convolution, and then outputs the features immediately. The third branch first performs a 3×3 convolution with a dilation rate of 1, then follows a 3×3 convolution with a dilation rate of 3, fuses the convolution result with the features output by the second branch, then follows a 1×1 convolution, and then outputs the features immediately. The fourth branch first performs a 3×3 convolution with a dilation rate of 1, then follows a 3×3 convolution with a dilation rate of 3, then follows a 3×3 convolution with a dilation rate of 1, fuses the convolution result with the features output by the third branch, then follows a 1×1 convolution, and then outputs the features immediately. The features output by each branch are all fused together as the final output;

[0052] Step S3: Train the network model:

[0053] Based on the encoder-decoder architecture and the multiscale feature fusion mechanism, obtain a new backbone network. The preprocessed data is used as the input of the network, and the glioma segmentation result is reconstructed through multi-layer feature extraction, and the loss function is calculated.

[0054] The segmentation network is an encoder-decoder structure. The same multi-scale feature fusion mechanism is embedded in each layer of the encoder and decoder. A supervision mechanism is set in the intermediate layer to assist in reconstructing accurate segmentation results. The supervision mechanism in the intermediate layer provides the ground truth for the segmentation results reconstructed in the intermediate layer, and uses a loss function to constrain the similarity between the reconstruction results of each layer and the ground truth. The calculation of the loss function is as follows:

[0055]

[0056] G Rec = Decoder(G i ), i = 1, 2, 3, 4 (3)

[0057]

[0058] Finally, the segmentation results of gliomas are output, and the loss function is calculated by comparing the segmentation results with the corresponding ground truth. The calculation of the loss function is as follows:

[0059]

[0060] Step S4: Perform reverse calibration:

[0061] The segmentation results of gliomas are obtained through network training. The inverse function is used to highlight the segmentation target, and the original features are calibrated through feature combination.

[0062] First, the segmentation results of each network layer are used to reduce the scale of the current feature map through bilinear interpolation to achieve the focus of the current feature information. Secondly, the compared feature information is used to activate the feature map information with the sigmoid activation function. The numerical range of the activated feature map is between 0 and 1. The closer the value is to 1, the higher the information content of the feature map, and vice versa. Then, the feature map retains the features with high information content by the inverse function and suppresses the features with low information content. Thirdly, the original features provided by each layer are expanded, and the inverse information of the features is combined with the original information in a multiplicative way to enhance the high-throughput information on the basis of the original information and calibrate the segmentation region. The inverse calibration of the features output by each layer is constrained by the loss function to correlate the segmentation region with the ground truth. Finally, the combined feature map is convolved and activated by ReLU, and the size of the feature map is restored in the same way of bilinear interpolation and passed down layer by layer. The calculation of the loss function is as follows:

[0063]

[0064] Step S5: Learn discrete regions:

[0065] First, the continuous and discrete regions of the brain glioma are separated by opening and closing operations. The opening operation consists of a dilation layer and a corrosion layer. The corrosion layer is executed first and then the dilation layer. The dilation layer and the corrosion layer consist of two steps: convolution and pooling. The convolution operation is to expand the segmentation result by a 3×3 convolution kernel, and then extract important features by pooling. The closing operation consists of a dilation layer and a corrosion layer. The dilation layer is executed first and then the corrosion layer. The convolution and pooling operations are the same as the opening operation. Discrete region learning is to achieve the purpose of focusing on discrete points by constraining the area between the segmentation result and the true value through the loss function. The loss function is calculated as follows;

[0066]

[0067] The segmentation network training includes four major loss functions, namely, feature reconstruction loss function, segmentation result loss function, reversal calibration loss function and discrete region learning loss function. The total loss function is as follows:

[0068]

[0069] where α , β ,γ and λ are hyperparameters during model training, and the weight of the loss function can be adjusted according to actual needs.

[0070] The combination of loss function and discrete region learning method provides a new and efficient glioma segmentation algorithm, which can accurately segment the tumor core area, tumor enhancement area and whole tumor area of ​​glioma, providing a basis for clinicians to diagnose glioma and make reasonable treatment decisions.

[0071] The segmentation algorithm provided in this application can accurately segment the tumor area and provide accurate tumor volume, shape and location data, which helps doctors evaluate the disease, choose treatment options and monitor the efficacy. Secondly, the automated segmentation process can help improve diagnostic efficiency, reduce manual errors and ensure higher accuracy. In addition, with the development of deep learning technology, segmentation algorithms can better cope with complex imaging features, identify subtle changes in tumors, and help doctors diagnose the disease early and evaluate the prognosis. Therefore, the brain glioma segmentation algorithm provides doctors with a reliable auxiliary means and promotes the scientificity and accuracy of clinical decision-making.

[0072] It should be understood that, although the embodiments of the present invention have been described and illustrated as much as possible, it is acceptable for a person skilled in the art to improve, change or replace the above description without departing from the principles of the present invention, and all changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for segmenting gliomas in the whole brain region based on multi-layer reverse calibration, characterized in that Specifically, it includes the following steps: Step S1: Construct a database and process the data set; Step S2: Integrate the multi-scale feature mechanism; Step S3: Train the network model; Step S4: Perform reverse calibration; Step S5: Learn the discrete region.

Images