A brain MR image segmentation method based on superpixels and fully convolutional neural network
By combining superpixels and fully convolutional neural networks, the shortcomings of the existing brain MR image segmentation algorithm in fine structure segmentation and spatial information considerations are solved, and high-accuracy image segmentation is achieved and noise processing is performed. show superiority.
Patent Information
- Application Number
- CN202111427359.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-29
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2041-11-29
AI Technical Summary
When existing image segmentation algorithms process complex medical images, especially brain MR images, it is difficult to accurately segment targets with fine structures, and lack spatial information considerations and high computational cost. The traditional method has poor results under the influence of noise.
Using a method based on superpixel and fully convolutional neural network, superpixel segmentation is performed through a simple linear iterative clustering algorithm, a weighted fusion fully convolutional neural network is constructed, and most voting rules are designed for pixel classification, combining global features and local details. Information, realize high accuracy segmentation of brain MR images.
It improves the accuracy of brain MR image segmentation, is better than other segmentation methods, can effectively process complex images and retain edge information and details, reducing noise impact.
Smart Images
Figure CN114092494B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of image segmentation. Specifically, an image segmentation method based on superpixels and a fully convolutional neural network is designed to achieve accurate segmentation of brain MR images. Background Art
[0002] Many image segmentation algorithms we are familiar with operate on pixels. In the segmentation process, the image is treated as a two-dimensional matrix for data processing, without considering the spatial organization relationship between pixels, which leads to low algorithm processing efficiency. Superpixel refers to an image block composed of adjacent pixels with similar texture, color, brightness and other characteristics. It groups pixels by the similarity of features between pixels, which can obtain redundant information of the image and greatly reduce the complexity of subsequent image processing tasks. Superpixel segmentation algorithm has a very wide range of applications due to its unique characteristics, and plays an important role in many image segmentation algorithms. With the continuous deepening of superpixel research in recent years, many excellent superpixel generation algorithms have emerged. These algorithms can be mainly divided into two categories, one is based on graph theory, and the other is based on gradient descent. Among them, the graph theory-based method is to convert the image into a weighted undirected graph, and then divide the generated graph into subgraphs according to certain rules. The division standard is to maximize the similarity within the subgraph and minimize the similarity between subgraphs. In essence, it is to remove specific edges in the weighted undirected graph to generate subgraphs that meet the requirements. The gradient-based method is essentially to cluster pixels, and then iterate continuously until the requirements are met. The resulting clusters are superpixels. Superpixel segmentation algorithms generally have problems with the number of superpixels, the mutual constraints between compactness and segmentation quality, and the practicality of the algorithm. It is also difficult to accurately segment targets with fine structures. Traditional image segmentation algorithms use low-level semantic information such as color, texture, and brightness to segment images. They run fast, have low complexity, and can also maintain good edge information. However, in practical applications, the content in the image may be more complex and have relatively large differences, which results in the phenomenon that the traditional segmentation method is not ideal. In recent years, with the continuous development of deep learning, it has been applied to various fields and has achieved very significant research results. The application of artificial neural networks in image segmentation has further improved the effect of image segmentation. The FCN network model is one of the very classic image semantic segmentation network models, and many subsequent network models are improved on the basis of this network. FCN replaces the last fully connected layer of CNN with a convolutional layer. Its biggest feature is that it can accept input images of any size. It upsamples the feature map of the last convolutional layer through deconvolution operations to restore it to the same size as the input image, thus achieving end-to-end semantic segmentation. The segmentation based on FCN lacks consideration of spatial information, and there are problems such as large amount of training data and high computational cost. For medical images, there is less feature information such as texture and color, and the boundary is not obvious. These factors will affect the segmentation effect. Summary of the invention
[0003] The purpose of the present invention is to overcome the shortcomings of the prior art, construct a fully convolutional neural network that considers both global features and local detail information, realize pixel re-division through the majority voting rule, and propose a brain MR image segmentation method based on superpixels and fully convolutional neural network to improve the accuracy of brain MR image segmentation, and its segmentation result is better than other segmentation methods.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is:
[0005] A brain MR image segmentation method based on superpixels and fully convolutional neural networks, the key steps of which include:
[0006] Step 1: Use a simple linear iterative clustering algorithm to preprocess the source image to obtain superpixel segmentation results;
[0007] Step 2: Construct a fully convolutional neural network based on weighted fusion. During the upsampling process of different paths, weighted fusion is performed on different image layers to obtain different segmentation results.
[0008] Step three, design a pixel classification method based on the majority voting rule. By comparing the classification labels of pixels in different segmentation results, the ambiguous pixels are re-divided to obtain the final refined segmentation result.
[0009] The simple linear iterative clustering algorithm described in step 1 is described as follows:
[0010] Given a source image I , N is the number of all pixels in the source image, in grid spacing initialization K The distance measurement formula is as follows:
[0011]
[0012] Among them, represents the pixel p With pixels q Grayscale distance, indicating pixel p With pixels q The spatial distance m and S Represents two parameters for balancing grayscale distance and spatial distance, m The value range is [1, 40].
[0013] In step 2, a fully convolutional neural network based on weighted fusion is constructed. In the upsampling process of different paths, different image layers are weighted fused. The construction method is:
[0014] By constructing 5 different network paths, 5 image segmentation results containing different amounts of information are obtained;
[0015] In the first path, 32 times deconvolution is performed on the pooling layer of the fifth layer to obtain the segmentation result FCN-32s;
[0016] In the second path, the prediction results of the fifth layer are upsampled by 2 times and weighted fused with the prediction results of the fourth layer (the weights are all 1 / 2) to obtain the segmentation result FCN-16s;
[0017] In the third path, the prediction results of the fifth layer are upsampled by 4 times, and the prediction results of the fourth layer are upsampled by 2 times. The two are weighted fused with the prediction results of the third layer (the weights are both 1 / 3) to obtain the segmentation result FCN-8s;
[0018] In the fourth path, the prediction results of the fifth layer are upsampled by 8 times, the prediction results of the fourth layer are upsampled by 4 times, and the prediction results of the third layer are upsampled by 2 times. The three are weightedly fused with the prediction results of the second layer (the weights are all 1 / 4) to obtain the segmentation result FCN-4s;
[0019] In the fifth path, the prediction results of the fifth layer are upsampled by 16 times, the prediction results of the fourth layer are upsampled by 8 times, the prediction results of the third layer are upsampled by 4 times, and the prediction results of the second layer are upsampled by 2 times. The four are weightedly fused with the prediction results of the first layer (the weights are all 1 / 5) to obtain the segmentation result FCN-2s;
[0020] In step 3, a pixel classification method based on the majority voting rule is designed. By comparing the classification labels of pixels in different segmentation results, the ambiguous pixels are reclassified. The method is described as follows:
[0021] For the source image I Pixels in p ,if p In all the different segmentation results R k () are all divided into the same area, then the pixel does not need to be re-divided, and its classification label is. p In different segmentation results, pixels are divided into different areas. p The classification label of is ambiguous, then the pixel p The final classification label will be based on p Its own gray value and segmentation result R k middle p Area m The mean grayscale value of pixels within is determined.
[0022] Ambiguous pixels p The final classification label is , . Where, represents the pixel p The gray value of . Represents pixels p In the segmentation results R k The segmented area to which it belongs, q Represents the pixels within the segmented area, representing the area The number of pixels within.
[0023] The main advantages of the method proposed in the present invention are: (1) a simple linear iterative clustering algorithm is used to perform superpixel segmentation on the source image, thereby improving the operating efficiency of subsequent processing; (2) a fully convolutional neural network based on weighted fusion is constructed, and different image layers are weightedly fused during the upsampling process of different paths, while considering the global features and local detail information of the pixels; (3) a pixel classification method based on the majority voting rule is designed to re-classify pixels with ambiguous class labels, thereby improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 is a flow chart of a brain MR image segmentation method based on superpixels and fully convolutional neural networks of the present invention;
[0025] Figure 2 It is a fully convolutional neural network model based on weighted fusion proposed by the present invention;
[0026] Figure 3 This is a comparison example of the image segmentation results obtained by the brain MR image segmentation method based on superpixels and fully convolutional neural networks of the present invention and other segmentation results. DETAILED DESCRIPTION
[0027] The present invention is described in detail below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that without departing from the core idea of the present invention, those skilled in the art can make some improvements to the present invention, which all belong to the protection scope of the present invention.
[0028] like Figure 1 As shown, the present invention provides a brain MR image segmentation method based on superpixels and a fully convolutional neural network, comprising the following steps:
[0029] Step 1: Use a simple linear iterative clustering algorithm to perform superpixel segmentation on the source image. The algorithm is described as follows:
[0030] Given a source image I , N is the number of all pixels in the source image. initialization K Cluster centers. The distance measurement formula is as follows:
[0031]
[0032] in, Represents pixels p With pixels q The grayscale distance, Represents pixels p With pixels q The spatial distance m and S Represents two parameters for balancing grayscale distance and spatial distance, m The value range is [1, 40].
[0033] Step 2: Construct a fully convolutional neural network based on weighted fusion. During the upsampling process of different paths, weighted fusion is performed on different image layers. The construction method is as follows:
[0034] By constructing 5 different network paths, 5 image segmentation results containing different amounts of information are obtained.
[0035] In the first path, 32 times deconvolution is performed on the pooling layer of the fifth layer to obtain the segmentation result FCN-32s;
[0036] In the second path, the prediction results of the fifth layer are upsampled by 2 times and weighted fused with the prediction results of the fourth layer (the weights are all 1 / 2) to obtain the segmentation result FCN-16s;
[0037] In the third path, the prediction results of the fifth layer are upsampled by 4 times, and the prediction results of the fourth layer are upsampled by 2 times. The two are weighted fused with the prediction results of the third layer (the weights are both 1 / 3) to obtain the segmentation result FCN-8s;
[0038] In the fourth path, the prediction results of the fifth layer are upsampled by 8 times, the prediction results of the fourth layer are upsampled by 4 times, and the prediction results of the third layer are upsampled by 2 times. The three are weightedly fused with the prediction results of the second layer (the weights are all 1 / 4) to obtain the segmentation result FCN-4s;
[0039] In the fifth path, the prediction results of the fifth layer are upsampled by 16 times, the prediction results of the fourth layer are upsampled by 8 times, the prediction results of the third layer are upsampled by 4 times, and the prediction results of the second layer are upsampled by 2 times. The four are weightedly fused with the prediction results of the first layer (the weights are all 1 / 5) to obtain the segmentation result FCN-2s;
[0040] Step 3: Design a pixel classification method based on the majority voting rule. By comparing the classification labels of pixels in different segmentation results, the ambiguous pixels are reclassified. The method is described as follows:
[0041] For the source image I Pixels in p ,if p In all the different segmentation results R k ( ) are all divided into the same area, then the pixel does not need to be re-divided, and its classification label is If the pixel p In different segmentation results, pixels are divided into different areas. p The classification label of is ambiguous, then the pixel p The final classification label will be based on p Its own gray value and segmentation result R k middle p Area m The mean grayscale value of pixels within is determined.
[0042] Ambiguous pixels p The final classification label is , .in, Represents pixels p The gray value of Represents pixels p In the segmentation results R k The segmented area to which it belongs, q Indicates segmentation area The pixels within Indicates area The number of pixels within.
[0043] In this embodiment, Figure 2 It is a fully convolutional neural network model based on weighted fusion proposed in the present invention, whose input is the source image and output is 5 different segmentation results.
[0044] Figure 3 This is a comparison example of the image segmentation results obtained by the brain MR image segmentation method based on superpixels and full convolutional neural network of the present invention and the segmentation results obtained by other segmentation algorithms, including two groups of images ((I) and (II)). Among them, the source image in group (II) is the source image in group (I) with Gaussian noise added N (0, 0.001) obtained. Figure 3 (a) is the brain MR source image, Figure 3(b)-(f) are the segmentation results obtained by different segmentation algorithms. As can be seen from the figure, the segmentation result obtained by the present invention has clear edges, can well retain the structural information and more detail information of the tissue area in the source image, and is better than other segmentation results. In addition, the present invention can effectively avoid the influence of noise.
[0045] Table 1 Figure 3 The accuracy evaluation results of different segmentation algorithms in . Among them, the best evaluation result is marked in bold. It can be seen from the table that the segmentation results obtained by the present invention are better than other segmentation algorithms in terms of accuracy, which verifies the above subjective evaluation from an objective perspective.
[0046] Table 1 Figure 3 Accuracy evaluation of segmentation results
[0047] Test image The present invention DCE HL-IIMT IIMT Otsu (I) 0.9593 0.9156 0.8532 0.8165 0.8021 (II) 0.8447 0.8321 0.7529 0.7345 0.7300
Claims
1. A brain MR image segmentation method based on superpixels and fully convolutional neural networks, characterized in that: This is accomplished by the following steps: Step 1: Use a simple linear iterative clustering algorithm to preprocess the source image to obtain superpixel segmentation results; Given a source image I, N is the number of all pixels in the source image, K cluster centers are initialized with a grid spacing, and the distance measurement formula is calculated as follows: in, represents the grayscale distance between pixel p and pixel q, represents the spatial distance between pixel p and pixel q. m and S represent two parameters used to balance the grayscale distance and spatial distance. The value range of m is [1,40]. Step 2: Construct a fully convolutional neural network based on weighted fusion. During the upsampling process of different paths, weighted fusion is performed on different image layers to obtain different segmentation results. By constructing 5 different network paths, 5 image segmentation results containing different amounts of information are obtained; In the first path, 32 times deconvolution is performed on the pooling layer of the fifth layer to obtain the segmentation result FCN-32s; In the second path, the prediction results of the fifth layer are upsampled by 2 times and weighted fused with the prediction results of the fourth layer, with the weights of 1 / 2 to obtain the segmentation result FCN-16s; In the third path, the prediction results of the fifth layer are upsampled by 4 times, and the prediction results of the fourth layer are upsampled by 2 times. The two are weighted fused with the prediction results of the third layer, and the weights are all 1 / 3 to obtain the segmentation result FCN-8s; In the fourth path, the prediction results of the fifth layer are upsampled by 8 times, the prediction results of the fourth layer are upsampled by 4 times, and the prediction results of the third layer are upsampled by 2 times. The three are weightedly fused with the prediction results of the second layer, and the weights are all 1 / 4 to obtain the segmentation result FCN-4s; In the fifth path, the prediction results of the fifth layer are upsampled 16 times, the prediction results of the fourth layer are upsampled 8 times, the prediction results of the third layer are upsampled 4 times, and the prediction results of the second layer are upsampled 2 times. The four are weightedly fused with the prediction results of the first layer, and the weights are all 1 / 5 to obtain the segmentation result FCN-2s. Step 3: Design a pixel classification method based on the majority voting rule. By comparing the classification labels of pixels in different segmentation results, the ambiguous pixels are reclassified to obtain the final refined segmentation result. For a pixel p in the source image I, if p is k are all divided into the same area, then the pixel does not need to be re-divided, and its classification label is If pixel p is divided into different regions in different segmentation results, that is, the classification label of pixel p is ambiguous, the final classification label of pixel p will be determined based on the grayscale value of p itself and the segmentation result R k The mean grayscale value of pixels in the region m to which p belongs is determined; The final classification label of the ambiguous pixel p is Wherein, g(p) represents the gray value of pixel p. Indicates that pixel p is in the segmentation result R k The segmentation area to which it belongs, q represents the segmentation area The pixels within Indicates area The number of pixels within.