Method, device and medium for automatic segmentation of abdominal three-dimensional CT images

Abstract

The present invention discloses an automatic segmentation method, device and medium for abdominal three-dimensional CT images, including obtaining an abdominal CT image data set for preprocessing, using an improved U-Net network model for coarse segmentation, using an SOA-Net network model for fine segmentation of four regions divided according to the approximate position information of each abdominal organ, and replacing the segmented image into an abdominal 3D scan image by elements to obtain an abdominal segmented image. The abdominal three-dimensional CT image is coarsely segmented by an improved U-Net network model, and then finely segmented by an SOA-Net network model that uses the least parameters and floating-point operations to obtain better segmentation accuracy, to obtain an accurate abdominal organ image, which meets the requirements for accurate and efficient segmentation of abdominal CT images in clinical practice, and solves the problem that the existing abdominal CT image segmentation method cannot accurately and efficiently segment abdominal CT images in clinical practice.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a method, device and medium for automatic segmentation of abdominal three-dimensional CT images. Background Art

[0002] The abdomen contains many important organs of the human body, including the liver, gallbladder, pancreas, spleen, and kidneys. CT scanning is a routine examination for abdominal diseases and an important reference for disease diagnosis. In abdominal CT images, accurate and automatic segmentation of organ regions plays an important role in human structure research, surgical planning, and clinical diagnosis. However, due to the limitations of CT imaging technology, individual differences among patients, and the complex characteristics of abdominal organs, there are still technical challenges in the actual application of multi-organ segmentation of abdominal CT images, such as high noise, complex background, and blurred boundaries between organs.

[0003] Looking back at the research results closely related to abdominal multi-organ segmentation, atlas-based methods were first applied to abdominal multi-organ segmentation. For example, statistical shape models and multi-image label fusion. Both methods are limited by the accuracy and performance of basic image registration algorithms, such as image selection, accurate registration of deformable images, and label fusion, and are often difficult to generalize.

[0004] In recent years, due to the improvement of computer performance and the rapid development of deep learning, deep neural networks have been widely used in tumor segmentation, organ detection and segmentation, and have achieved certain research results. Various deep learning architectures, such as convolutional neural networks (CNN), fully convolutional networks (FCN), U-Net and 3DU-Net have been used for high-precision segmentation of organs such as pancreas and liver. However, the disadvantage of these methods is that they are not effective in segmenting organs with large deformation, small size and blurred tissue edges.

[0005] In order to improve the segmentation effect of small organs, the attention mechanism is introduced. Although the attention mechanism in the existing technology has good performance, these networks are highly complex, the model size is large, and the computing resources are large, which cannot meet the requirements of lightweight and efficient models in clinical practice. Therefore, it is impractical to apply the algorithms in the existing technology to clinical practice. Summary of the invention

[0006] The technical problem to be solved by the present application is that the existing abdominal CT image segmentation methods are inaccurate and inefficient in segmenting multiple abdominal organs in clinical practice, and cannot meet the requirements for accurate and efficient abdominal CT image segmentation in clinical practice. The present application aims to provide an automatic abdominal three-dimensional CT image segmentation method, device and medium. By improving the U-Net network model to perform coarse segmentation of abdominal three-dimensional CT images, the segmentation efficiency of the overall framework is improved and the computational consumption is reduced. Then, the CT images are finely segmented by the SOA-Net network model with higher segmentation accuracy to obtain accurate abdominal organ images, thereby solving the problem that the existing abdominal CT image segmentation methods cannot perform accurate segmentation of abdominal CT images in clinical practice and the segmentation efficiency is low.

[0007] The present invention is achieved through the following technical solutions:

[0008] The present application provides a method for automatically segmenting an abdominal three-dimensional CT image in a first aspect, comprising:

[0009] S1, obtaining an abdominal CT image data set and preprocessing it to obtain an abdominal 3D scan image;

[0010] S2. Use the improved U-Net network model to roughly segment the 3D scanned image to obtain the approximate location information of each abdominal organ;

[0011] S3, dividing the abdominal organs into four regions according to their approximate location information, and using the SOA-Net network model to perform fine segmentation on the four regions to obtain fine organ images;

[0012] S4. Replace the fine organ image into the abdominal 3D scan image element by element to obtain an abdominal segmentation image.

[0013] In the above technical solution, the improved U-Net network model is used to roughly segment the abdominal three-dimensional CT image to obtain the approximate location information of each abdominal organ. According to the approximate location information of the abdominal organs, the four abdominal organs of liver, kidney, spleen and pancreas are divided into four regions, each corresponding to an abdominal organ. The use of the improved U-Net network model to roughly segment the abdominal three-dimensional CT image improves the segmentation efficiency of the overall framework and reduces the computational consumption.

[0014] Then, the SOA-Net network model is used to segment the abdominal organs in the four regions to obtain a fine abdominal segmentation image. Then, the fine organ image is replaced element by element into the original abdominal 3D scan image to obtain an abdominal segmentation image.

[0015] Since the SOA-Net network model is a self-regulating organ-aware deep network, it can achieve higher image segmentation accuracy, making the abdominal three-dimensional CT images more accurate when performing fine segmentation, meeting the requirements for accurate abdominal CT image segmentation in clinical practice.

[0016] In an optional embodiment, the method of finely segmenting the four regions using the SOA-Net network model is as follows:

[0017] The encoder of the SOA-Net network model introduces the MBFA module, which includes four encoding branches, each of which processes the image of the abdominal organs in a region to obtain the abdominal image features;

[0018] The decoder of the SOA-Net network model introduces the FAA module, which includes two decoding branches. The abdominal image features are decoded through the two decoding branches to obtain a fine organ image.

[0019] In an optional embodiment, the method in which each encoding branch processes an abdominal organ image in a region includes:

[0020] According to the image features of an abdominal organ, a coding branch is constructed by using multiple depth convolutions and multiple point convolutions;

[0021] The features of the corresponding abdominal organ image are extracted through an encoding branch to obtain the corresponding abdominal image features.

[0022] In an optional embodiment, each encoding branch processes an image of an abdominal organ in a region, and after obtaining the abdominal image features, the following further includes:

[0023] Introducing a coding adjustment attention module at the output end of the MBFA module, the coding adjustment attention module receives the abdominal image features output by four coding branches;

[0024] The encoding adjustment attention module adaptively adjusts the size of the receiving field according to the multi-scale features of the abdominal image and segments the target organs of different scales within the receiving field to obtain a final abdominal image feature map.

[0025] In an optional embodiment, the method for the encoding adjustment attention module to segment target organs of different scales within the receiving field is as follows:

[0026] U=U1+U2+U3+U4

[0027]

[0028] S=F FC =δ(B(WZ))

[0029]

[0030]

[0031]

[0032]

[0033] V=U1a+U2b+U3c+U4d

[0034] Among them, U1, U2, U3, and U4 represent the outputs of the four encoding branches; F GB represents global average pooling; Z∈R C ,Z C represents the Cth element of Z; δ is the reLU activation function; B represents batch normalization; W∈R d×C And S∈R d ; A,B,C,D∈R C×d ; A C represents the Cth row of A; a, b, c, d represent the soft attention of the four outputs respectively, and a C +b C +c C +d C =1; a C represents the Cth element of a; V represents the final abdominal image feature map obtained by adding each element one by one.

[0035] In an optional embodiment, after decoding the abdomen image features through two decoding branches, the following steps are further included:

[0036] A decoding adjustment attention module is introduced at the output end of the FAA module, and the decoding adjustment attention module receives feature information output by two decoding branches;

[0037] The decoding adjustment attention module integrates the feature information from the two decoding branches to obtain a feature map of the fine organ image.

[0038] In an optional embodiment, the method in which the decoding adjustment attention module integrates feature information from two decoding branches is as follows:

[0039] U 1 =U1+U2

[0040]

[0041] S 1 =F FC =δ(B(WZ 1 ))

[0042]

[0043]

[0044] V 1 =U1a 1 +U2b 1

[0045] Among them, U1 and U2 represent the outputs of the two decoding branches. Represents Z 1 The Cth element of C×d ,A C represents the Cth row of A; a and b represent the soft attention of two outputs respectively; Indicates a 1 The Cth element in ; V 1 Feature maps representing fine-scale organ images.

[0046] In an optional embodiment, before using the SOA-Net network model to finely segment the four regions, the method further includes:

[0047] The SOA-Net network model introduces a residual structure to deepen the network hierarchy;

[0048] The feature pyramid structure is used to enhance the SOA-Net network model.

[0049] A second aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an automatic abdominal three-dimensional CT image segmentation method is implemented.

[0050] A third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, a method for automatically segmenting abdominal three-dimensional CT images is implemented.

[0051] Compared with the prior art, this application has the following advantages and beneficial effects:

[0052] By improving the U-Net network model, the abdominal three-dimensional CT images are roughly segmented, and then the SOA-Net network model with the least parameters and floating-point operations is used to finely segment the CT images to obtain accurate abdominal organ images. This meets the requirements for accurate and efficient segmentation of abdominal CT images in clinical practice and solves the problem that the existing abdominal CT image segmentation methods cannot accurately and efficiently segment abdominal CT images in clinical practice.

[0053] Compared with the existing abdominal organ segmentation networks, the main contributions of our SOA-Net network model are as follows:

[0054] 1. In order to improve the feature representation ability of abdominal multi-organ segmentation, a new network SOA-Net with a self-adjusting attention mechanism is proposed.

[0055] 2. A four-branch MBFA module is designed to segment the four abdominal organs (liver, kidney, spleen, and pancreas) as the encoding module of the U-Net model. At the same time, a two-branch FAA module is used as the decoding module in U-Net. These two modules have multiple branches with different kernel sizes, which can obtain feature information of different scales according to the multi-scale of the target organ.

[0056] 3. The effectiveness and efficiency of the proposed 3D SOA-Net framework are demonstrated on a challenging abdominal multi-organ CT dataset, where we achieve state-of-the-art methods with relatively low time cost, fewer parameters and failures. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0058] Figure 1 A flowchart of a method for automatic segmentation of abdominal three-dimensional CT images provided in one embodiment of the present application;

[0059] Figure 2 A schematic diagram of the structure of an MBFA module provided in one embodiment of the present application;

[0060] Figure 3 A structural schematic diagram of an FAA module provided in one embodiment of the present application;

[0061] Figure 4 A comparison chart of segmentation performance of different methods on the AbdomenCT-1K dataset provided in an embodiment of the present application;

[0062] Figure 5 An example diagram of abdominal multi-organ segmentation results of different methods on the AbdomenCT-1K dataset provided in an embodiment of the present application;

[0063] Figure 6 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0065] Example

[0066] Figure 1 A method for automatically segmenting an abdominal three-dimensional CT image is provided in Example 1 of the present invention, such as Figure 1 As shown, the following steps are included:

[0067] S1. Obtain an abdominal CT image dataset and preprocess it to obtain an abdominal 3D scan image.

[0068] The abdominal CT image data set is sampled from the CT scan volume and preprocessed to obtain an abdominal 3D scan image for subsequent segmentation of the abdominal 3D scan image. The preprocessing method may adopt any preprocessing method in the prior art.

[0069] S2. Use the improved U-Net network model to roughly segment the 3D scanned image to obtain the approximate location information of each abdominal organ.

[0070] Among them, the abdominal 3D scan image is input into the improved U-Net network model. The encoder in the improved U-Net network model is a feature extractor, which is used to learn increasingly advanced representations; the decoder in the improved U-Net network model decomposes the compressed features into increasingly fine resolutions for predicting the segmentation of each target.

[0071] Furthermore, each layer of the encoder and decoder has two residual blocks, each of which consists of a 3D convolutional layer with instance normalization (IN) and rectified linear unit (ReLU) activation function. The input resolution of each layer is [160, 80, 40, 20, 10].

[0072] Furthermore, the addition of the residual block occurs before the ReLU activation. Specifically, the encoding stage includes 5 convolutions and 4 downsampling operations. In each residual module, a convolution layer with a kernel size of 3×3×3 and a stride of 1×1×1 is used to extract the input feature map. Among them, the feature information extracted by the previous layer is used as the input of the next layer to help complete residual learning. Correspondingly, the decoding stage includes four convolutions and four upsampling operations, using a convolution kernel size of 3×3×3 and a stride of 1×1×1.

[0073] Furthermore, in addition to the left and right encoding and decoding paths, the improved U-Net network model uses an improved ReduceChannel+Upsample+Add operation to replace the original concat operation, which is beneficial to reducing operation parameters and improving computational efficiency.

[0074] In particular, we use 1×1×1 convolution to reduce the number of features by one half, 3D bilinear upsampling to double the spatial dimension, and then add the encoder outputs of the corresponding spatial horizontal layers. In the decoding stage, this operation is performed on the results of each group of convolutional layers to achieve multi-scale feature fusion and optimize the coarse segmentation results.

[0075] S3. The abdominal organs are divided into four regions according to their approximate location information, and the four regions are finely segmented using the SOA-Net network model to obtain fine organ images.

[0076] Among them, the SOA-Net network model is a network model that performs fine segmentation of the abdominal organs in the approximate location information of each abdominal organ obtained by the rough segmentation of the improved U-Net network model. Different from the coarse segmentation model, the SOA-Net segmentation model uses a larger resolution [192,96,48,24,12] to obtain more accurate segmentation results.

[0077] Furthermore, the motivation for designing the architecture using the SOA-Net network model is that a larger network width and resolution can improve segmentation accuracy, but at the same time, the introduction of the SOA-Net network model increases the number of network parameters and computational cost. Therefore, in our SOA-Net network model, we introduced the MBFA module in the encoder and the FAA module in the decoder, which restored the larger parameters and computational cost to a certain extent. The specific configuration of the SOA-Net network model is shown in the following table:

[0078]

[0079] In the above table, the MBFA module is introduced in the encoder and the FAA module is introduced in the decoder. Each module has multiple branches, and each branch is composed of different convolutions. For example, branch Branch1 is composed of ConvINReLU(1-1), DWConvIN(3-2-1-1) and ConvINReLU(1-1) in sequence. The specific convolution structures of each branch of the MBFA model and FAA model are shown in the above table.

[0080] Compared with the encoding path, without significantly affecting the segmentation accuracy, we performed a series of simplification operations on SOA-Net in the decoding path, reducing the complexity of the network and the number of computational parameters, and finally obtained an asymmetric U-Net network model.

[0081] The network can adaptively adjust the size of the receptive field according to the multi-scale input information to achieve efficient segmentation of multiple abdominal organs.

[0082] The SOA-Net network model also introduces a residual structure to deepen the network layer, so that more feature representations can be learned.

[0083] In an optional embodiment, in order to adapt to the segmentation task of multiple abdominal organs (liver, kidney, spleen, and pancreas), we introduce the MBFA module in the encoder of the SOA-Net network model. The MBFA module includes four encoding branches, each of which processes the image of the abdominal organs in a region to obtain the abdominal image features.

[0084] Among them, MBFA modules such as Figure 2 As shown in the figure, the MBFA module is designed following the concept of InceptionNets, which consists of four encoding branches, each with a different number and parameter of convolution kernels. In this way, each encoding branch has a receptive field of different sizes, which helps self-regulation and organ perception to cope with the segmentation of four organs of different sizes in the abdomen.

[0085] Furthermore, the convolution kernel uses depth-wise separable convolution instead of the standard convolution used in the prior art.

[0086] Among them, the depth-wise separable convolution is divided into depth-wise convolution and point-wise convolution. The depth-wise convolution is a lightweight filter that uses a single convolution filter on each input channel; the point-wise convolution uses 1×1×1 convolution, which mainly establishes new features by calculating the linear combination of the input channels. The MBFA module has four encoding branches in total. The first encoding branch is composed of two ConvINReLU (1×1×1)s=2 point convolutions plus SWConvIN (3×3×3)s=2 depth-wise convolutions. It first uses ConvINReLU (1×1×1)s=2 point convolutions to process the input data, and then uses SWConvIN (3×3×3)s=2 depth-wise convolutions to process the features extracted by the previous layer of ConvINReLU (1×1×1)s=2 point convolutions, and finally uses ConvINReLU (1×1×1)s=2 point convolutions to process the features, thereby identifying a specific abdominal organ. The specific structure of the MBFA model is as follows: Figure 2 shown.

[0087] Research shows that the performance of depthwise separable convolution is almost the same as that of traditional convolution, but the amount of computation and the number of parameters are greatly reduced. Depthwise separable convolution has the same receptive field as traditional convolution, but the former consumes half the computation and space than the latter. In addition, each branch has an inverted residual module. In particular, the feature dimension is expanded by point convolution, then the features are extracted by depthwise convolution, and finally the output dimension is reduced by pointwise convolution. The advantage of this method is that depthwise convolution can extract higher-dimensional features and improve feature extraction capabilities.

[0088] In an optional embodiment, a coding adjustment attention module is introduced at the output end of the MBFA module, and the coding adjustment attention module receives feature information output by four coding branches; the coding adjustment attention module adaptively adjusts the size of the receiving field according to the multi-scale feature information and segments the target organs of different scales within the receiving field. The coding adjustment attention module is an effective and lightweight attention mechanism. The mechanism can adaptively adjust the size of the receiving field according to the multi-scale input information, thereby achieving the segmentation of target organs of different scales.

[0089] Furthermore, the segmentation method for segmenting target organs of different scales is as follows:

[0090] First, we need to integrate the information from the four branches. This is done using an element-wise sum merge operation:

[0091] U=U1+U2+U3+U4

[0092] Among them, U1, U2, U3, and U4 represent the outputs of these four branches.

[0093] Then, through the global average pooling operation, the information of all points in the space is averaged into one value to obtain the global semantic information:

[0094]

[0095] Among them, F GB represents global average pooling, Z∈R C ,Z C Represents the Cth element of Z.

[0096] Next, we pass two fully connected layers to find the ratio of each channel. The channels of the first fully connected layer are reduced to improve efficiency, and then the second fully connected layer restores the original channels:

[0097] S=F FC =δ(B(WZ))

[0098] Where δ is the reLU activation function and B represents batch normalization. d×C And S∈R d .

[0099] Under the guidance of S obtained in the previous step, cross-channel soft attention is used to adaptively select information of different spatial scales. Specifically, the softmax operator is applied to the channel-level numbers:

[0100]

[0101]

[0102]

[0103]

[0104] Among them, A,B,C,D∈R C×d ,A C represents the Cth row of A. a, b, c, d represent the soft attention of the four outputs respectively. a C represents the Cth element of a. The outputs of the four branches are weighted by four weight matrices A, B, C, and D, and a C +b C +c C +d C = 1. Then, we add each element together to get the final feature map V:

[0105] V=U1a+U2b+U3c+U4d

[0106] In an optional embodiment, a FAA module is introduced into the decoder of the SOA-Net network model, and the FAA module includes two decoding branches. The abdominal image features are decoded by the two decoding branches to obtain a fine organ image.

[0107] Among them, the FAA module consists of a residual convolution module. The design of the FAA module is roughly the same as that of the MBFA module. It follows the concept of InceptionNets and uses depth-separable convolution for each branch, which reduces the amount of convolution calculations and parameters and achieves high module efficiency. However, unlike the MBFA module, its network structure is simpler, such as Figure 3 First, it only contains two branches of convolution kernels; second, each branch no longer uses the inverted residual module structure, and the number of input channels and output channels of the module is consistent, which greatly reduces the network complexity and computational complexity.

[0108] Furthermore, a decoding adjustment attention module is introduced at the output end of the FAA module, and the decoding adjustment attention module receives feature information output by two decoding branches; the decoding adjustment attention module integrates the feature information from the two decoding branches to obtain a feature map of the fine organ image.

[0109] The FAA module’s attention adjustment mechanism is the same as the MBFA module’s attention adjustment mechanism, except that the number of branches is changed from 4 to 2. Therefore, the FAA module’s attention adjustment module needs to integrate information from two branches. The algorithm is explained as follows.

[0110] U 1 =U1+U2

[0111]

[0112] S 1 =F FC =δ(B(WZ 1 ))

[0113]

[0114]

[0115] V 1 =U1a 1 +U2b 1

[0116] Among them, U1 and U2 represent the outputs of the two branches. Represents Z 1 The Cth element of A,B∈R C×d ,A C represents the Cth row of A. a and b represent the soft attention of two outputs respectively. Indicates a 1 The Cth element in . Use two weight matrices A and B to weight the outputs of the two branches, and then add them element by element to get the final feature map V 1 .

[0117] S4. Replace the fine organ image into the abdominal 3D scan image element by element to obtain an abdominal segmentation image.

[0118] In an optional embodiment, the SOA-Net network model introduces a residual structure for deepening the network hierarchy and uses a feature pyramid structure to enhance the SOA-Net network model, thereby improving the accuracy of abdominal three-dimensional CT image segmentation.

[0119] In order to prove the superiority of the algorithm in this technical solution, we compared our algorithm with several cutting-edge segmentation algorithms. The quantitative segmentation results are shown in the following table:

[0120]

[0121]

[0122] The methods compared in this embodiment include 3D U-Net, Attention U-Net, Nonlocal U-Net, ECAU-Net, EfficientSegNet and CBAM U-Net. All compared methods were tested on the data set under the same experimental conditions. It can be seen from the experimental results in the above table that all networks can obtain high DSC and NSD scores in the segmentation of liver, kidney and spleen organs. However, due to the changes in the volume and shape of the pancreas, the DSC and NSD scores show that the performance of the pancreas is not ideal. Our network achieved the best DSC and NSD values ​​for the segmentation of the liver, kidney, spleen and pancreas, with an average DSC of 94.7% and an NSD of 86.8%. Especially in terms of pancreatic DSC and NSD, our method is superior to the previous method (CBAM U-Net) by nearly 3%, which is a large improvement. Finally, for all networks, we observed a significant gap between DSC and NSD, which indicates that the current network cannot obtain accurate boundary segmentation and needs further improvement.

[0123] Figure 4 The average DSC score (Average_DSC) and the average NSD score (Average_NSD) of the four organs in the test dataset are shown. Figure 4 To demonstrate the improvement of our method in small organ segmentation, we also show the average DSC score of the pancreas (Pancreas_DSC) and the average NSD score of the pancreas (Pancreas_NSD).

[0124] from Figure 4 It can be seen that the automatic segmentation method of abdominal three-dimensional CT images of the present application has a very significant improvement in average segmentation performance over 3DU-Net, Attention U-Net, CBAM U-Net and Nonlocal U-Net. Compared with ECA U-Net and EfficientSegNet, the results of the automatic segmentation method of abdominal three-dimensional CT images of the present application are more concentrated in the upper and middle reaches. This means that our method has lower DSC and NSD standard deviations, and our method is the most stable and robust across all different CT cases. In addition, for the segmentation of small and irregular pancreas, current methods often do not work well due to large organ deformation, small volume and blurred tissue edges. But our method also achieved better segmentation performance than other advanced methods. The DSC and NSD values ​​obtained by the algorithm are the best among all tested methods.

[0125] In order to evaluate the segmentation efficiency, we list the average inference time (s), parameter amount (MB), and floating point operation amount (GB) of each volume of different networks on the abdomen dataset as shown in the following table:

[0126]

[0127] As can be seen from the table above, all the networks compared have increased inference time compared to 3D U-Net. They all take more time to segment abdominal CT images, and our network takes an extra half of the time during inference. For parameter amount and floating-point operation amount, 3D U-Net has 2.4MB parameters and 508GB floating-point operation amount, while our network has only 1.1MB parameters and 83GB floating-point operation amount. These values ​​show that our network has the least parameters and floating-point operation amount than all other networks. All parameter amount and floating-point operation amount values ​​in the above table are divided by 10 and 1000 respectively, and the histogram obtained by normalization to 0 to 1 shows that other networks generally require more parameters and floating-point operations than 3D U-Net to obtain improved segmentation accuracy, while our network can obtain better segmentation accuracy with the least parameters and floating-point operations.

[0128] We also conduct ablation studies to show the effectiveness of the proposed module for abdominal multi-organ segmentation. To evaluate the segmentation performance, we introduce residual blocks in U-Net to obtain a base U-Net. In the base U-Net, each encoder and decoder layer has two residual modules. Each residual block contains a 3D convolution layer, plus instance normalization (IN) and rectified linear unit activation function (ReLU). In the encoder module, a convolution layer with a 3×3×3 filter and a stride of 2 is used; in the decoder module, a convolution layer with a 3×3×3 filter and a stride of 1 is used. The ablation analysis results are shown in Table 5. We can see that the base U-Net achieves an average DSC of 94.4% and NSD of 85.6% with 0.14s inference time, 4.6MB parameters, and 246.5GB floating-point operations. When we replace the encoder and decoder modules in U-Net with our MBFA module and FAA module, the inference time increases, but the network improves the segmentation performance with reduced parameters and floating-point operations. In particular, our SOA-Net achieves an average of 94.7% DSC and 86.8% NSD at 0.29s inference time, 1.1MB parameters, and 82.7GB floating-point operations. Although this method has fewer parameters and floating-point operations, it has better segmentation results and is an ideal method for abdominal multi-organ segmentation.

[0129] The abdominal multi-organ segmentation results of different methods on the AbdomenCT-1K dataset are shown in the figure. Figure 5 As shown in the figure, the red, yellow, green, and blue areas represent the liver, pancreas, kidney, and spleen, respectively. We list three CT images containing the liver, kidney, spleen, and pancreas to show the segmentation results more intuitively. These test images are randomly selected from our dataset. Figure 5 It can be seen that there are many over-segmentations and under-segmentations in the compared methods. For example, the left image is mainly under-segmented on the pancreas and liver, and the middle image is over-segmented on the pancreas. In addition, the performance gap between these methods is mainly reflected in the pancreas segmentation. The compared methods perform well on large organs (spleen, liver, kidney) but poorly on small organs. In the abdominal multi-organ segmentation, this method achieves better segmentation performance than other methods.

[0130] Compared with the prior art, this application has the following breakthroughs:

[0131] In order to cope with the challenges of abdominal multi-organ segmentation, an improved abdominal multi-organ segmentation network SOA-Net was proposed based on the clinical practice and the requirements of segmentation accuracy and efficiency.

[0132] First, our method follows a holistic coarse-to-fine framework to fully utilize the 3D information in abdominal CT scans. The coarse model is designed to obtain the approximate location of the target organ from the entire CT volume. Our SOA-Net is a refinement network used to improve the results obtained from the coarse model.

[0133] In addition, the model introduces a residual structure to deepen the network layer so that more feature representations can be learned. In order to effectively deal with the multi-scale changes and long-range information dependence of multiple organs and improve the segmentation accuracy of smaller organs, we use a feature pyramid structure (FPN) to enhance our refinement network.

[0134] Finally, we improved the SK module in SK-Net and obtained the Multi-branch Feature Attention (MBFA) module and Feature Attention Aggregation (FAA) module in our SOA-Net to replace the encoding and decoding modules of the U-Net network. This network is a three-dimensional self-adjusting organ-aware deep network that can adaptively adjust the size of the receptive field according to the multi-scale input information to achieve efficient segmentation of multiple abdominal organs.

[0135] Compared with the existing abdominal organ segmentation networks, the main contributions of our SOA-Net network model are as follows:

[0136] 1. In order to improve the feature representation ability of abdominal multi-organ segmentation, a new network SOA-Net with a self-adjusting attention mechanism is proposed.

[0137] 2. A four-branch MBFA module is designed to segment the four abdominal organs (liver, kidney, spleen, and pancreas) as the encoding module of the U-Net model. At the same time, a two-branch FAA module is used as the decoding module in U-Net. These two modules have multiple branches with different kernel sizes, which can obtain feature information of different scales according to the multi-scale of the target organ.

[0138] 3. The effectiveness and efficiency of the proposed 3D SOA-Net framework are demonstrated on a challenging abdominal multi-organ CT dataset, where we achieve state-of-the-art methods with relatively low time cost, fewer parameters and floating-point numbers.

[0139] Example 2

[0140] Figure 6 A schematic diagram of the structure of an electronic device provided in Embodiment 2 of the present invention, such as Figure 6 As shown, the electronic device includes a processor 21, a memory 22, an input device 23 and an output device 24; the number of processors 21 in the computer device can be one or more. Figure 6 A processor 21 is taken as an example; the processor 21, the memory 22, the input device 23 and the output device 24 in the electronic device can be connected by a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0141] The memory 22 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules. The processor 21 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 22, that is, to implement an automatic abdominal three-dimensional CT image segmentation method of Example 1.

[0142] The memory 22 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0143] In some optional embodiments, the memory 22 may further include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the electronic device via a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0144] The input device 23 can be used to receive the ID and password input by the user, etc. The output device 24 is used to output the network configuration page.

[0145] Example 3

[0146] Embodiment 3 of the present invention further provides a computer-readable storage medium, wherein the computer-executable instructions are used to implement an automatic abdominal three-dimensional CT image segmentation method as provided in Embodiment 1 when executed by a computer processor.

[0147] The storage medium containing computer executable instructions provided in an embodiment of the present invention is not limited to the method operations provided in Example 1, and can also execute related operations in an abdominal three-dimensional CT image automatic segmentation method provided in any embodiment of the present invention.

[0148] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for automatic segmentation of abdominal three-dimensional CT images, characterized in that: The steps include: S1, obtaining an abdominal CT image data set and preprocessing it to obtain an abdominal 3D scan image; S2. Use the improved U-Net network model to roughly segment the 3D scanned image to obtain the approximate location information of each abdominal organ; S3, dividing the abdominal organs into four regions according to their approximate location information, and using the SOA-Net network model to perform fine segmentation on the four regions to obtain fine organ images; S4, replacing the fine organ image into the abdominal 3D scan image element by element to obtain an abdominal segmentation image; Among them, the method of using the SOA-Net network model to finely segment the four areas is as follows: The encoder of the SOA-Net network model introduces the MBFA module, which includes four encoding branches, each of which processes the abdominal organ image in a region to obtain the abdominal image features; The decoder of the SOA-Net network model introduces the FAA module, which includes two decoding branches. The abdominal image features are decoded through the two decoding branches to obtain a fine organ image. Among them, a coding adjustment attention module is introduced at the output end of the MBFA module, and the coding adjustment attention module performs a method of segmenting target organs of different scales within the receiving field as follows: U=U1+U2+U3+U4 S=F FC =δ(B(WZ)) V=U1a+U2b+U3c+U4d Among them, U1, U2, U3, and U4 represent the outputs of the four encoding branches; F GB represents global average pooling; Z∈R C ,Z C represents the Cth element of Z; δ is the reLU activation function; B represents batch normalization; W∈R d×C And S∈R d ; A,B,C,D∈R C×d ; A C represents the Cth row of A; a, b, c, d represent the soft attention of the four outputs respectively, and a C +b C +c C +d C =1; a C represents the Cth element of a; V represents the final abdominal image feature map obtained by adding each element one by one; After decoding the features of the abdomen image through two decoding branches, the following steps are also included: A decoding adjustment attention module is introduced at the output end of the FAA module, and the decoding adjustment attention module receives feature information output by two decoding branches; The decoding adjustment attention module integrates the feature information from the two decoding branches to obtain a feature map of the fine organ image.

2. The method for automatic segmentation of abdominal three-dimensional CT images according to claim 1, characterized in that: The method for processing an abdominal organ image in a region corresponding to each encoding branch includes: According to the image features of an abdominal organ, a coding branch is constructed by using multiple depth convolutions and multiple point convolutions; The image of the abdominal organ is subjected to feature extraction through the encoding branch to obtain corresponding abdominal image features.

3. The method for automatic segmentation of abdominal three-dimensional CT images according to claim 1, characterized in that: The method in which the decoding adjustment attention module integrates the feature information from the two decoding branches is as follows: U 1 =U1+U2 S 1 =F FC =δ(B(WZ 1 )) In 1 =U1a 1 +U2b 1 Among them, U1 and U2 represent the outputs of two decoding branches; Represents Z 1 The Cth element of C×d ,A C represents the Cth row of A; a and b represent the soft attention of two outputs respectively; Indicates a 1 The Cth element in ; V 1 Feature maps representing fine-scale organ images.

4. The method for automatic segmentation of abdominal three-dimensional CT images according to claim 1, characterized in that: Before using the SOA-Net network model to finely segment the four areas, it also includes: The SOA-Net network model introduces a residual structure to deepen the network hierarchy; The feature pyramid structure is used to enhance the SOA-Net network model.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for automatic segmentation of abdominal three-dimensional CT images as described in any one of claims 1 to 4 is implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for automatic segmentation of abdominal three-dimensional CT images as claimed in any one of claims 1 to 4 is implemented.

Images