Pancreas segmentation system in CT images based on improved U-shaped network
By introducing contrast enhancement module and reverse attention module in the U-Net network, the U-shaped network is improved for pancreatic CT image segmentation, which solves the problem of unsatisfactory segmentation effect in the prior art and achieves higher segmentation accuracy.
Patent Information
- Application Number
- CN202110161067.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-02-05
- Publication Date
- 2025-04-29
- Estimated Expiration
- 2041-02-05
AI Technical Summary
The existing U-Net network is prone to losing small target information and edge information in pancreatic CT image segmentation, resulting in unsatisfactory segmentation effect.
An improved U-shaped network is adopted, with a contrast enhancement module (CEB) and reverse attention module (RAB) added to extract edge details of the pancreas and mine complementary information of the characteristics.
It improves the segmentation accuracy of the pancreas and effectively solves the problem of easily losing small target information and edge information in the prior art.
Smart Images
Figure CN114119448B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a pancreas segmentation system in a CT image based on an improved U-shaped network, and belongs to the technical field of image processing. Background Art
[0002] The pancreas is the second largest exocrine organ in the human body and an important endocrine gland in the human body. The pancreas can be divided into the head, neck, body, and tail of the pancreas. Pancreatic cancer is one of the common malignant tumors of the digestive system. It is clinically discovered late, has a high degree of malignancy, and has a poor prognosis. The 5-year relative survival rate of pancreatic cancer is less than 10%, which is the most serious among common malignant tumors, and it tends to worsen year by year. Early detection of pancreatic cancer is the key to radical resection of pancreatic cancer and survival. Computed tomography (CT) is the most important imaging method for pancreatic diseases, with advantages such as high-density resolution and repeatability. Automatic segmentation of the pancreas in abdominal CT images can provide more accurate information for the diagnosis and treatment of pancreatic cancer.
[0003] In recent years, convolutional neural networks have rapidly proven to be the most advanced tools for processing various medical images. In particular, the U-Net network has performed well in the task of pixel-level segmentation of medical images. However, since the pancreas occupies a small proportion of the entire image, its size, shape, position and structure vary greatly between different patients, the edges are not clear, and diseases such as pancreatitis and pancreatic cancer can greatly change the appearance of the pancreas. The original U-Net only uses a simple encoder, decoder and jump connection, which is easy to lose small target information and edge information, and the segmentation effect of the pancreas is not ideal. Summary of the invention
[0004] The object of the present invention is to provide a pancreas segmentation system in CT images based on an improved U-shaped network, so as to solve the problems existing in the prior art.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] According to a first aspect, an embodiment of the present invention provides a pancreas segmentation system in a CT image based on an improved U-shaped network, the system comprising a pancreas segmentation network, the pancreas segmentation network comprising: an encoder branch, a decoder branch, a contrast enhancement module CEB, and a reverse attention module RAB;
[0007] The contrast enhancement module CEB is arranged after the encoder branch, and is used to extract edge detail information of the pancreas in the encoder branch feature map;
[0008] The reverse attention module RAB is arranged between the encoder branch and the decoder branch, and utilizes the output features of each layer of the decoder branch to mine the complementary features of the previous layer.
[0009] Optionally, the encoder branch includes four convolution blocks consisting of two 3×3 convolutions, batch normalization, and a relu activation function, a maximum pooling layer is set after each convolution block, and a convolution block is set after the fourth maximum pooling layer to connect the encoder branch and the decoder branch.
[0010] Optionally, the contrast enhancement module CEB is arranged after the second, third and fourth convolution blocks in the encoder branch, and is used to extract edge detail information of the pancreas in the feature map of the encoder branch.
[0011] Optionally, the decoder branch includes four convolution blocks consisting of two 3×3 convolutions, batch normalization, and a relu activation function. A deconvolution layer with a convolution kernel size of 2×2 and a step size of 2 is set before each convolution block. The last convolution block is followed by a 1×1 convolution and a sigmoid activation function to obtain a predicted probability map. Each layer of the encoder branch is connected to the corresponding layer of the decoder branch through a jump connection.
[0012] Optionally, the reverse attention module RAB is arranged between the convolution block of each layer of the encoder branch and the convolution block of each layer of the decoder branch, and the output features of each layer of the decoder branch are used to mine the complementary features between the features of the previous encoder layer.
[0013] Optionally, the output feature map of the contrast enhancement block CEB is:
[0014] Output=(1+G 1 )×Input+(1-G 1 )×(G 2 ×D eem );
[0015] Among them, Deem is the edge detail information feature map, G1 and G2 are gated feature maps.
[0016] Optionally, the output of the reverse attention module RAB is:
[0017] Output=σ 1 (f 1 (Input_1))×W r +Input_1;
[0018] Among them, f1(·) is the convolution operation, σ1(·) is the relu activation function,
[0019] W r =σ 2 (-f 2 (Input_2)), f2(·) is the deconvolution operation, and σ2(·) is the sigmoid activation function.
[0020] Optionally, the loss function of the pancreas segmentation network is a Dice loss function.
[0021] In a second aspect, a pancreas segmentation network training method is provided, the method comprising:
[0022] Obtaining a sample pancreatic CT image;
[0023] The pancreas segmentation network described in the first aspect is trained according to the sample pancreatic CT image.
[0024] In a third aspect, a method for pancreas segmentation in a CT image based on an improved U-shaped network is provided, the method comprising:
[0025] Acquire a target pancreas CT image;
[0026] The target pancreatic CT image is segmented according to the trained pancreatic segmentation network; the trained pancreatic segmentation network is a network trained by the method described in the second aspect.
[0027] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Figure 1 A network structure diagram of a pancreas segmentation network in a CT image based on an improved U-shaped network provided by an embodiment of the present invention;
[0029] Figure 2 A schematic structural diagram of a contrast enhancement module CEB provided in one embodiment of the present invention;
[0030] Figure 3 A schematic diagram of the structure of a reverse attention module RAB provided for one embodiment of the present invention;
[0031] Figure 4 A method flow chart of a pancreas segmentation network training method provided by one embodiment of the present invention;
[0032] Figure 5 A method flow chart of a method for segmenting the pancreas in a CT image using an improved U-shaped network provided by one embodiment of the present invention;
[0033] Figure 6 A comparison diagram of the segmentation results of the pancreas segmentation method provided by an embodiment of the present invention and the existing solution. DETAILED DESCRIPTION
[0034] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.
[0036] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0037] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.
[0038] Please refer to Figure 1 , which shows a network structure diagram of a pancreas segmentation network in a CT image based on an improved U-shaped network provided by an embodiment of the present application, such as Figure 1 As shown, the pancreas segmentation network includes: an encoder branch, a decoder branch, a contrast enhancement block (CEB) and a reverse attention block (RAB);
[0039] The contrast enhancement module CEB is arranged after the encoder branch, and is used to extract edge detail information of the pancreas in the encoder branch feature map;
[0040] The reverse attention module RAB is arranged between the encoder branch and the decoder branch, and utilizes the output features of each layer of the decoder branch to mine the complementary features of the previous layer.
[0041] This application uses U-Net as the basic framework of the pancreas segmentation network. And:
[0042] The encoder branch includes four convolution blocks consisting of two 3×3 convolutions, batch normalization, and a relu activation function. A maximum pooling layer is set after each convolution block. A convolution block is set after the fourth maximum pooling layer as an encoding underlying structural unit for connecting the encoder branch and the decoder branch.
[0043] The decoder branch includes four convolution blocks consisting of two 3×3 convolutions, batch normalization, and a relu activation function. A deconvolution layer with a convolution kernel size of 2×2 and a step size of 2 is set before each convolution block. The last convolution block is followed by a 1×1 convolution and a sigmoid activation function to obtain a predicted probability map. Each layer of the encoder branch is connected to the corresponding layer of the decoder branch through a jump connection.
[0044] Please refer to Figure 2 The contrast enhancement module CEB is arranged after the second, third and fourth convolution blocks in the encoder branch, and is used to extract edge detail information of the pancreas in the feature map of the encoder branch.
[0045] The output of the contrast enhancement module CEB is:
[0046] Output=(1+G 1 )×Input+(1-G 1 )×(G 2 ×D eem )
[0047] Among them, D eem The edge extraction module obtains the edge detail information feature map by subtracting the local average information feature map obtained by average pooling from the input feature map. Input is the input feature map, G 1 and G 2 is a gated feature map. They calculate the input feature map and the obtained edge detail information feature map respectively through 1×1 convolution and sigmoid activation function, representing the deterministic feature map of each position of the input feature map and the edge detail information feature map. The uncertain position of the input feature map can be obtained by (1-G 1 ). For the uncertain position in the input feature map, the deterministic edge information obtained by calculation can supplement and enhance the information of the encoder feature map in detail.
[0048] The reverse attention module RAB is set between the convolution block of each layer of the encoder branch and the convolution block of each layer of the decoder branch, and uses the output features of each layer of the decoder branch to mine the complementary features between the features of the previous encoder layer.
[0049] Please refer to Figure 3 , the input of the reverse attention module RAB is the input Input_2 of the decoder layer and the input Input_1 of the previous decoder layer, and the output of the reverse attention module RAB is:
[0050] Output=σ 1 (f 1 (Input_1))×W r +Input_1;
[0051] Among them, f1(·) is the convolution operation, σ1(·) is the relu activation function, and W r =σ 2 (-f 2 (Input_2)), f2(·) is the deconvolution operation, and σ2(·) is the sigmoid activation function. The rough feature map obtained by deconvolution is inverted and multiplied with the feature map of the previous encoder layer to refine the inaccurate rough estimate into an accurate and complete prediction map.
[0052] Optionally, in the present application, the loss function of the pancreas segmentation network is a Dice loss function. Assuming that the predicted graph Y' and the true value Y are given, the Dice loss is:
[0053]
[0054] In summary, a pancreas segmentation network in CT images based on an improved U-shaped network is provided, and the pancreas segmentation network includes: an encoder branch, a decoder branch, a contrast enhancement module CEB and a reverse attention module RAB; the contrast enhancement module CEB is arranged after the encoder branch, and is used to extract the edge detail information of the pancreas in the feature map of the encoder branch; the reverse attention module RAB is arranged between the encoder branch and the decoder branch, and the output features of each layer of the decoder branch are used to mine the complementary features of the previous layer. It is possible to extract edge detail information and conduct complementary mining of features, which solves the problem of easy loss of small target information and edge information and unsatisfactory segmentation effect of the pancreas in the existing scheme, and achieves the effect of improving the segmentation accuracy of the pancreas.
[0055] Please refer to Figure 4 , which shows a method flow chart of a pancreas segmentation network training method provided by an embodiment of the present application, such as Figure 4 As shown, the method includes:
[0056] Step 401, obtaining a sample pancreatic CT image;
[0057] The data used in the present invention are collected from patients with pancreatic cancer. The data set consists of 106 three-dimensional (3D) CT images with doctor annotations. The image size of each 3D CT image is 512×512×L, where L is the number of slices along the long axis of the body, L∈[174,376]. The present invention uses 63 of them for training.
[0058] Moreover, in actual implementation, before training the pancreatic segmentation network, the sample pancreatic CT images can be preprocessed, and the original intensity value of each sample pancreatic CT image can be truncated to [-100, 240] to enhance the display of pancreatic details. Then, we normalize each original CT data to a mean of 0 and a variance of 1 to reduce the data differences caused by the medical image acquisition process. In order to prevent the pancreatic segmentation network from being seriously affected by the background, the present invention trains the model on slices of the pancreas occupying at least 50 pixels. In order to obtain spatial information in a three-dimensional image without increasing the amount of calculation, three consecutive slices of these three-dimensional images are connected as input.
[0059] Step 402: training the pancreas segmentation network described in the above embodiment according to the sample pancreatic CT image.
[0060] The present invention is trained on a working device equipped with an NVIDIA Tesla K40m GPU with 12G memory. During the training process, the initial learning rate is 10 -4 The Adam optimizer is used to optimize the network weights. The number of sample pancreatic CT images fed into the network each time is set to 3, and the total number of iterations is 20. The pancreatic segmentation network is implemented based on the Tensorflow framework. Since the pancreas is a complete organ, in order to further improve the segmentation accuracy, the three-dimensional maximum connected domain is extracted as the final pancreatic segmentation result.
[0061] In summary, by acquiring a sample pancreatic CT image, the pancreatic segmentation network as described above is trained according to the sample pancreatic CT image. The trained pancreatic segmentation network can extract edge detail information and perform complementary mining on features, thereby solving the problem of easy loss of small target information and edge information and unsatisfactory pancreatic segmentation effect in the existing solution, and achieving the effect of improving the pancreatic segmentation accuracy.
[0062] Please refer to Figure 5 , which shows a method flow chart of a method for pancreas segmentation in CT images based on an improved U-shaped network provided by an embodiment of the present application, such as Figure 5 As shown, the method includes:
[0063] Step 501, obtaining a target pancreatic CT image;
[0064] Step 501, segmenting the target pancreatic CT image according to the trained pancreatic segmentation network; the trained pancreatic segmentation network is a network trained by the method described in the above embodiment.
[0065] In order to quantitatively evaluate the performance of the proposed method, the segmentation results are compared with the gold standard according to the following four indicators: Dice similarity coefficient, intersection over union (IoU), precision and recall. DSC calculates the overlap between the segmentation result and the gold standard and is defined as:
[0066]
[0067] Where TP is the number of true positive segmented pixels, FP is the number of false positive segmented pixels, and FN is the number of false negative segmented pixels. The calculation formulas for the intersection-over-union ratio, precision, and recall are:
[0068]
[0069]
[0070] The present invention compares the segmentation results with the existing methods, as shown in Table 1.
[0071] Table 1 Comparison of the segmentation results of the present invention with those of others
[0072]
[0073] like Figure 6 As shown in Table 1, the segmentation results of different methods are shown in Table 2. The first column is the original CT image. The second column is the corresponding gold standard. The following are the segmentation results of U-Net and Attention U-Net. The last column is the segmentation result of the method of the present invention. Figure 6 , those skilled in the art can clearly know that the segmentation accuracy of the pancreas segmentation network described in this application is higher than that of the existing solutions.
[0074] In summary, a target pancreatic CT image is acquired; the target pancreatic CT image is segmented according to a trained pancreatic segmentation network; the trained pancreatic segmentation network is a network trained by the method described in the above embodiment. The problem that small target information and edge information are easily lost and the pancreatic segmentation effect is not ideal in the existing solution is solved, and the effect of improving the pancreatic segmentation accuracy is achieved.
[0075] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0076] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A pancreas segmentation system in CT images based on an improved U-shaped network, the system comprising a pancreas segmentation network, characterized in that: The pancreas segmentation network includes: an encoder branch, a decoder branch, a contrast enhancement module CEB and a reverse attention module RAB; The contrast enhancement module CEB is arranged after the encoder branch, and is used to extract edge detail information of the pancreas in the encoder branch feature map; The encoder branch includes four convolution blocks consisting of two 3×3 convolutions, batch normalization, and a relu activation function, a maximum pooling layer is set after each convolution block, and a convolution block is set after the fourth maximum pooling layer to connect the encoder branch and the decoder branch; The contrast enhancement module CEB is arranged after the second, third and fourth convolution blocks in the encoder branch, and is used to extract edge detail information of the pancreas in the feature map of the encoder branch; The reverse attention module RAB is arranged between the encoder branch and the decoder branch, and uses the output features of each layer of the decoder branch to mine the complementary features of the previous layer; The decoder branch includes four convolution blocks consisting of two 3×3 convolutions, batch normalization, and a relu activation function. A deconvolution layer with a convolution kernel size of 2×2 and a step size of 2 is set before each convolution block. The last convolution block is followed by a 1×1 convolution and a sigmoid activation function to obtain a prediction probability map. Each layer of the encoder branch is connected to the corresponding layer of the decoder branch through a skip connection; The reverse attention module RAB is set between the convolution block of each layer of the encoder branch and the convolution block of each layer of the decoder branch, and uses the output features of each layer of the decoder branch to mine the complementary features between the features of the previous encoder layer.
2. The pancreas segmentation system according to claim 1, characterized in that: The output characteristic graph of the contrast enhancement module CEB is: Output=(1+G1)×Input+(1-G1)×(G2×D eem ); Among them, D eem It is the edge detail information feature map, Input is the input feature map, G1 and G2 are gated feature maps.
3. The pancreas segmentation system according to claim 1, characterized in that: The output of the reverse attention module RAB is: Output=σ1(f1(Input_1))×W r +Input_1; Among them, f1(·) is the convolution operation, σ1(·) is the relu activation function, W r =σ2(-f2(Input_2)), f2(·) is the deconvolution operation, σ2(·) is the sigmoid activation function, Input_1 is the input of the previous layer of the decoder, and Input_2 is the input of the current layer of the decoder.
4. The pancreas segmentation system according to claim 1, characterized in that: The loss function of the pancreas segmentation network is the Dice loss function.
5. A pancreas segmentation network training method, characterized in that: The method comprises: Obtaining a sample pancreatic CT image; The pancreas segmentation network according to any one of claims 1 to 4 is trained based on the sample pancreatic CT image.
6. A method for pancreas segmentation in CT images based on an improved U-shaped network, characterized in that: The method comprises: Acquire a target pancreas CT image; The target pancreatic CT image is segmented according to the trained pancreatic segmentation network; the trained pancreatic segmentation network is a network trained by the method according to claim 5.