A liver tumor segmentation method in CT images based on 3DA-U-Nets architecture
By introducing a three-dimensional attention mechanism and network cascade structure into the U-Net network, the problem of insufficient accuracy of liver tumor segmentation is solved, more efficient and accurate liver tumor segmentation is achieved, and the ability of computer automation segmentation is improved.
Patent Information
- Application Number
- CN202210672085.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-15
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2042-06-15
AI Technical Summary
Liver tumor segmentation has the problem of insufficient accuracy in medical images, especially because the liver image has low contrast and is closely related to other tissues and organs, which makes it difficult to artificial segmentation, and computer automated segmentation is difficult to improve segmentation efficiency and accuracy.
Using the CT image liver tumor segmentation method based on the 3DA-U-Nets architecture, the network's attention to features is enhanced by introducing a three-dimensional attention mechanism into the U-Net network, and the primary and secondary network cascade structures are used to generate liver Mask and deep feature extraction are performed to improve the accuracy of liver tumor segmentation.
It significantly improves the accuracy and accuracy of liver tumor segmentation, and has better segmentation effect than traditional U-Net model, reducing the impact on other organs similar to morphology, and reliable segmentation results.
Smart Images

Figure CN115131369B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a CT image liver tumor segmentation method based on a 3DA-U-Nets architecture, and belongs to the technical field of medical image processing. Background Art
[0002] The liver is an important digestive organ in the human body, and liver cancer is also a malignant disease with a high mortality rate. Therefore, early detection of liver lesions and tumors during abdominal examination is of great significance for diagnosis, treatment and cure. However, liver tumor segmentation has always been a difficult problem in medical image segmentation. Liver images have low contrast and are closely and complexly connected with other tissues and organs. Even manual segmentation of liver tumors is difficult. Therefore, in order to maximize the segmentation efficiency and avoid the influence of human factors on liver segmentation, it is still necessary to rely on computers to automatically segment liver tumors. Effectively improving the segmentation accuracy of liver tumors is one of the key factors to improve the efficiency of clinical diagnosis and treatment. Summary of the invention
[0003] In order to solve the problem of insufficient accuracy of liver tumor segmentation in the prior art, the present invention proposes a CT image liver tumor segmentation method based on the 3DA-U-Nets architecture. This method attempts to integrate a new hybrid domain attention mechanism into the network on the basis of U-Net, thereby increasing the network's attention to target features from multiple aspects such as channels and space, and increasing the network's ability to extract features, so that the network can obtain more effective features of the research target on the basis of the original convolution module, thereby improving the accuracy of liver tumor segmentation.
[0004] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:
[0005] In a first aspect, a method for segmenting liver tumors in a CT image is provided, comprising:
[0006] Obtain a CT image that includes the liver tumor;
[0007] The liver region is segmented from the CT image using a first-level liver segmentation network based on 3DA-U-Net, and the liver region is converted into a binary liver mask according to a preset threshold.
[0008] performing a mask operation on the CT image and the liver mask to obtain a liver region of interest;
[0009] Using a secondary tumor segmentation network based on 3DA-U-Net to extract features of the liver region of interest, and obtain deep features of the CT image;
[0010] According to the deep features of the CT image, a liver tumor segmentation result is output.
[0011] In some embodiments, the method for constructing the 3DA-U-Net of the primary liver segmentation network and the secondary tumor segmentation network includes:
[0012] Get the U-Net network and 3D attention mechanism;
[0013] Among them, the contraction path of the U-Net network includes five network modules connected in sequence. The first, second, third, and fourth network modules all include 2 convolutional layers and 1 pooling layer, and the fifth network module includes 2 convolutional layers;
[0014] The three-dimensional attention mechanism is recorded as 3DA; after adding the 3DA module to each convolution module in the U-Net network, a 3DA-U-Net network is formed.
[0015] The U-Net network structure specifically includes:
[0016] Input port input, convolution layer Conv 1-1, convolution layer Conv 1-2, pooling layer Maxpool-1, convolution layer Conv 2-1, convolution layer Conv 2-1, pooling layer Maxpool-2, convolution layer Conv 3-1, convolution layer Conv 3-2, pooling layer Maxpool-3, convolution layer Conv 4-1, convolution layer Conv 4-2, pooling layer Maxpool-4, convolution layer Conv 5-1, convolution layer Conv 5-2, upper convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, upper convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output;
[0017] The 3DA-U-Nets network structure specifically includes:
[0018] First-level liver segmentation network: input port input, convolution layer Conv 1-1, convolution layer Conv 1-2, 3DA layer 1, pooling layer Maxpool-1, convolution layer Conv 2-1, convolution layer Conv 2-1, 3DA layer 2, pooling layer Maxpool-2, convolution layer Conv 3-1, convolution layer Conv 3-2, 3DA layer 3, pooling layer Maxpool-3, convolution layer Conv 4-1, convolution layer Conv 4-2, 3DA layer 4, pooling layer Maxpool-4, convolution layer Conv 5-1, convolution layer Conv 5-2, 3DA layer 5, upper convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, up convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output;
[0019] Secondary tumor segmentation network: input port input, convolution layer Conv 1-1, convolution layer Conv 1-2, 3DA layer 1, pooling layer Maxpool-1, convolution layer Conv 2-1, convolution layer Conv 2-1, 3DA layer 2, pooling layer Maxpool-2, convolution layer Conv 3-1, convolution layer Conv 3-2, 3DA layer 3, pooling layer Maxpool-3, convolution layer Conv 4-1, convolution layer Conv 4-2, 3DA layer 4, pooling layer Maxpool-4, convolution layer Conv 5-1, convolution layer Conv 5-2, 3DA layer 5, upper convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, up convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output.
[0020] The structures of the primary liver segmentation network and the secondary tumor segmentation network are consistent. The specific parameters of the 3DA-U-Net network are shown in Table 1, including the convolution kernel size, convolution step size, number of channels, and feature map size.
[0021] In some embodiments, the processing flow of the three-dimensional attention mechanism 3DA in 3DA-U-Net includes:
[0022] Input convolutional layer processed features Figure X , the number of channels is C, the size is W×H, and the 3DA structure features Figure X Make three copies and perform global average pooling operations along three different directions to obtain one-dimensional features of three different dimensions, namely W×1×1, 1×H×1, and 1×1×C tensors;
[0023] The one-dimensional features of three different dimensions are respectively sent to two fully connected layers. The number of channels of the first fully connected layer is C / r, where r is the reduction rate, and the activation function ReLU is passed; the output is then input into the second fully connected layer. The number of channels of this layer is C, and the activation function sigmoid is used for nonlinear expression to obtain new W×1×1, 1×H×1 and 1×1×C tensors, which contain the weight information between each element of the feature map and the target feature;
[0024] The new W×1×1, 1×H×1, and 1×1×C tensors containing weight information and the initial features Figure X Multiply them together to get the new features recalibrated by 3DA Figure X '.
[0025] Each element in the new feature map can obtain long-range correlation with other elements while retaining precise location information, so that the target is enhanced both in channel and space, allowing the network to focus on all targets of interest in the world.
[0026] In some embodiments, the training process of the primary liver segmentation network based on 3DA-U-Net includes:
[0027] Preprocess the CT images containing liver tumors to obtain a training sample set;
[0028] Initialize the network parameters of 3DA-U-Net and input the training samples into 3DA-U-Net in sequence;
[0029] After each 2 iterations, the network loss function is calculated according to the updated liver segmentation image; the network parameters of the 3DA-U-Net are iteratively updated according to the loss function to obtain an updated 3DA-U-Net; and an updated liver segmentation image is obtained according to the updated 3DA-U-Net;
[0030] Convergence is judged based on the network loss function. When the network loss function converges, the iteration ends and the trained first-level liver segmentation network based on 3DA-U-Net is output.
[0031] In some embodiments, the training process of the secondary tumor segmentation network based on 3DA-U-Net includes:
[0032] The liver region of interest containing liver tumors is used as the training sample set; the network parameters of the first-level liver segmentation network based on 3DA-U-Net are loaded as the initialization of the second-level network parameters, and the liver region of interest training sample set is input into the 3DA-U-Net second-level network in sequence to fine-tune the network parameters of the second-level network; after each 2 iterations, the network loss function is calculated according to the updated tumor segmentation image; convergence judgment is made according to the network loss function, and when the network loss function converges, the iteration is terminated and the trained second-level tumor segmentation network based on 3DA-U-Net is output.
[0033] In some embodiments, the network parameters during the training process of the 3DA-U-Net network include a learning rate and a batch size, the learning rate is initialized to 0.0001, and the batch size is initialized to 5.
[0034] In some embodiments, during the training process of the two-stage 3DA-U-Net network, the network parameters include a learning rate and a batch size, the learning rate is initialized to 0.0001, and the batch size is initialized to 5.
[0035] In some embodiments, the loss function is a Dice coefficient, and the loss function is calculated as follows:
[0036]
[0037] Where A is the sample predicted by the model, and B is the gold standard in the dataset; for two given images A and B, their overlapping area is |A∩B|, and |A|+|B| is the sum of the areas of the two images; the DiceLoss(A, B) loss function changes continuously during training to provide direction for the iteration of the network, and its value range is [0, 1].
[0038] In a second aspect, the present invention provides a CT image liver tumor segmentation device, including a processor and a storage medium;
[0039] The storage medium is used to store instructions;
[0040] The processor is used to operate according to the instructions to execute the steps of the method according to the first aspect.
[0041] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in the first aspect.
[0042] The following advantages can be obtained by using the above technical means:
[0043] The present invention proposes a CT image liver tumor segmentation method based on a 3DA-U-Nets architecture. The method of the present invention proposes a 3DA module based on a traditional attention mechanism. The module strengthens the attention to the target area and characteristics by scanning each dimension of the feature map, and fuses the module with a U-Net network to obtain a 3DA-U-Net liver segmentation network. At the same time, network cascading is used, a first-stage 3DA-U-Net generates a liver mask, and a second-stage 3DA-U-Net realizes the segmentation of liver tumors. By shielding other organs, the target attention is increased, so as to achieve the purpose of improving the tumor segmentation accuracy. The liver tumor segmentation accuracy of the method of the present invention is high, the segmentation result is accurate and reliable, and the liver tumor segmentation effect is much higher than the segmentation effect of the U-Net model. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 A flowchart of the steps of a method for segmenting liver tumors in CT images based on a 3DA-U-Nets architecture of the present invention;
[0045] Figure 2 This is a diagram of the 3DA-U-Nets network structure in an embodiment of the present invention;
[0046] Figure 3 This is a calculation flow chart of a three-dimensional attention module in an embodiment of the present invention;
[0047] Figure 4 This is a schematic diagram of the structure of the original U-Net network in an embodiment of the present invention;
[0048] Figure 5 3DA-U-Net network structure diagram in an embodiment of the present invention;
[0049] Figure 6 Schematic diagram of the effect of the preprocessing window technology on an image in an embodiment of the present invention;
[0050] Figure 7 A schematic diagram of a CT image in an embodiment of the present invention;
[0051] Figure 8 Schematic diagram of the liver tumor segmentation gold standard in an embodiment of the present invention;
[0052] Fig. 9 This is a liver tumor segmentation probability map obtained by using the CFCNs network processing in an embodiment of the present invention;
[0053] Fig.10 The liver tumor segmentation probability map obtained by using the H-DenseUNet segmentation network in an embodiment of the present invention;
[0054] Fig.11The liver tumor segmentation probability map obtained by using SE-U-Nets network processing in an embodiment of the present invention;
[0055] Fig.12 This is a liver tumor segmentation probability map obtained by processing using the method of the present invention in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] The technical solution of the present invention is further described below in conjunction with the accompanying drawings:
[0057] In the description of the present invention, "several" means more than one, "many" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.
[0058] In the description of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.
[0059] The present invention proposes a CT image liver tumor segmentation method based on the 3DA-U-Nets architecture. Starting from the perspective of improving the network feature extraction capability, the method integrates the 3DA structure into the U-Net network to achieve the effect of improving the target weight, allowing the network to pay more attention to the target information and encode the input data more efficiently, while obtaining global channel attention and global spatial attention; the cascade network structure can prevent the segmentation network from being affected by other organ tissues with similar morphology, effectively reducing the situation of incorrect tumor segmentation, thereby improving the segmentation accuracy of liver tumors.
[0060] Example 1
[0061] like Figure 1 , 2 As shown, a method for segmenting liver tumors in CT images includes:
[0062] Obtain a CT image that includes the liver tumor;
[0063] The liver region is segmented from the CT image using a first-level liver segmentation network based on 3DA-U-Net, and the liver region is converted into a binary liver mask according to a preset threshold.
[0064] performing a mask operation on the CT image and the liver mask to obtain a liver region of interest;
[0065] Using a secondary tumor segmentation network based on 3DA-U-Net to extract features of the liver region of interest, and obtain deep features of the CT image;
[0066] According to the deep features of the CT image, a liver tumor segmentation result is output.
[0067] The calculation process of the three-dimensional attention mechanism 3DA is as follows:
[0068] Input convolutional layer processed features Figure X , the number of channels is C, the size is W×H, and the 3DA structure features Figure X Make three copies and perform global average pooling operations in three different directions. Obtain one-dimensional features of three different dimensions, namely W×1×1, 1×H×1 and 1×1×C tensors. Send them to two fully connected layers respectively. The number of channels of the first fully connected layer is C / r (r is the reduction rate), and pass through the activation function ReLU. The output is then input into the second fully connected layer. The number of channels of this layer is C. It is expressed nonlinearly through the activation function sigmoid to obtain new W×1×1, 1×H×1 and 1×1×C tensors, which contain the weight information between each element of the feature map and the target feature. Finally, multiply the new weights and the initial feature map to obtain the new features recalibrated by 3DA. Figure X '. The calculation process is as follows Figure 3 shown.
[0069] 3DA-U-Net is formed by integrating three-dimensional attention into the U-Net network. That is, a three-dimensional attention module is added after each convolution module in the U-Net network encoding process, and the feature map obtained by the convolution module is processed to obtain the attention weights in three dimensions. By weighting each pixel point of the feature map, more attention can be focused on the target features in the encoding stage, allowing the network to converge as quickly as possible and improving the segmentation accuracy of liver tumors.
[0070] The structures of the primary liver segmentation network and the secondary tumor segmentation network are consistent. The specific parameters of the 3DA-U-Net network are shown in Table 1, including the convolution kernel size, convolution step size, number of channels, and feature map size.
[0071] Table 1 Specific parameters of 3DA-U-Net network
[0072]
[0073]
[0074] The 3DA-U-Nets liver tumor segmentation method is an image segmentation network composed of a two-level network cascade. This is because the abdominal CT image contains many organs, which is a disadvantage for liver tumor segmentation. If the segmentation is performed directly, it is easy to be affected by other organs and tissues with similar morphology. In order to avoid this situation, this network uses a primary network to segment the organs, obtains a mask image of the liver, and then calculates with the image itself to obtain a set of images of a pure liver area. The data set obtained by the previous network is input into the secondary network for liver tumor segmentation. In this way, reducing the area processed by the network can also greatly improve the network accuracy and effectively reduce the situation of incorrect tumor segmentation. At the same time, when segmenting the liver, the network has already learned the feature knowledge of the abdominal CT image, and training the network for segmenting the liver on this basis will converge better. The network design is divided into the following parts.
[0075] (1) The first-level liver segmentation network uses abdominal CT images as a data set. After the CT images are processed through window technology and BM3D denoising technology, they are input into the first-level liver segmentation network 3DA-U-Net. The input data in the network passes through each convolution module to obtain a feature map W×H×C, where the number of channels C is determined by the number of convolution kernels in the convolution layer. Then, a new feature map focusing on the research target in three dimensions is obtained through the three-dimensional attention module. Finally, the image size is reduced through the maximum pooling layer to complete the calculation of this layer of the network. Then, the image is input into the next convolution module for learning until the encoding is completed. The network contains 5 convolution modules. The size of the experimental image is downsampled from 512×512 to 32×32. After upsampling and skip connection, the size is restored to 512×512 to obtain the output result of the first-level network, which is then used as the input of the second-level tumor segmentation network.
[0076] (2) The secondary tumor segmentation network uses the liver mask and CT image obtained by the upper network as input to the second-level tumor segmentation network. The network uses the mask to obtain the liver region of interest and shields other organ tissues, so that the network can focus on the segmentation of liver tumors, and the final segmentation result is obtained using the 3DA-U-Net network. Since the secondary tumor segmentation network in the present invention has the same structure as the primary liver segmentation network, the parameters obtained after the training of the primary network are used as the initialization parameters of the secondary network, and then the tumor segmentation training is performed, which can accelerate the convergence of the network.
[0077] The network parameters in the training process of the CT image liver tumor segmentation network based on the 3DA-U-Nets architecture include the learning rate and batch size. The learning rate is initialized to 0.0001, the batch size is initialized to 5, and the network loss function is calculated as follows:
[0078]
[0079] Where A is the sample predicted by the model, and B is the gold standard in the dataset. For two given images A and B, their overlapping area is |A∩B|, and |A|+|B| is the sum of the areas of the two images. The DiceLoss(A, B) loss function changes continuously during training, providing direction for the iteration of the network, and its value range is [0, 1].
[0080] The specific training process is as follows:
[0081] (1) First, train the 3DA-U-Net liver segmentation network.
[0082] Preprocess the CT images containing liver tumors to obtain a training sample set;
[0083] Initialize the network parameters of 3DA-U-Net and input the training samples into 3DA-U-Net in sequence;
[0084] After each 2 iterations, the network loss function is calculated according to the updated liver segmentation image; the network parameters of the 3DA-U-Net are iteratively updated according to the loss function to obtain an updated 3DA-U-Net; and an updated liver segmentation image is obtained according to the updated 3DA-U-Net;
[0085] Convergence is judged based on the network loss function. When the network loss function converges, the iteration ends and the trained first-level liver segmentation network based on 3DA-U-Net is output.
[0086] (2) Training the liver tumor segmentation network
[0087] The liver region of interest containing liver tumors is used as the training sample set; the network parameters of the first-level liver segmentation network based on 3DA-U-Net are loaded as the initialization of the second-level network parameters, and the liver region of interest training sample set is input into the 3DA-U-Net second-level network in sequence to fine-tune the network parameters of the second-level network; after each 2 iterations, the network loss function is calculated according to the updated tumor segmentation image; convergence judgment is made according to the network loss function, and when the network loss function converges, the iteration is terminated and the trained second-level tumor segmentation network based on 3DA-U-Net is output.
[0088] After completing the training of the CT image liver tumor segmentation network based on the 3DA-U-Nets architecture, the present invention first uses the CT image of the test set as the input of the 3DA-U-Nets liver tumor segmentation network, and uses the first-level 3DA-U-Net to output the segmented liver area; then the CT image and the liver mask are operated to retain the liver area on the original CT image, while shielding other non-related organs to obtain the liver region of interest; then the trained second-level 3DA-U-Net is used to extract the features of the liver area to obtain the liver tumor area.
[0089] In order to verify the effect of the method of the present invention, several groups of comparative experiments are given in the examples of the present invention:
[0090] The hardware and software simulation environment in the comparative experiment is shown in Table 2:
[0091] Table 2
[0092] CPU Core i7-6700 GPU Nvidia GeForce RTX 2080Ti Memory 8G operating system Windows 10 Development Environment Tensorflow programming language Python 3.6 Cuda Cuda10.0with cudnn6.0
[0093] The commonly used and annotated public liver tumor segmentation dataset 3DIRCADB is used as the dataset. CFCNs network, SEU-Net network and H-DenseUNet segmentation network are used as comparison algorithms respectively.
[0094] In this experiment, the DICE correlation coefficient is used as the main indicator to evaluate the performance of liver tumor segmentation, and the volume overlap error (VOE), relative volume difference (RVD), precision and recall are selected for evaluation. The comparison algorithm and the method of the present invention are used to process the test set under the simulation environment of Table 2 to obtain the liver tumor segmentation results, and the various evaluation indicators of the network segmentation are recorded.
[0095] Figure 7 This is the CT image in the comparison experiment. Figure 8 yes Figure 7 Schematic diagram of the corresponding liver tumor segmentation gold standard. Fig. 9 , 10 11 and 12 are respectively FCNs network, U-Net network, SEU-Net network and H-DenseUNet segmentation network and the processing method of the present invention Fig. 9 The liver tumor segmentation probability map obtained later shows that the present invention can locate the position of the tumor more accurately, and the obtained liver tumor segmentation probability map is closer to the gold standard. The edges of large tumors are clearer, and the edge accuracy of small tumors is higher. Although there is a slight loss, it will not lead to the neglect of the lesion, and the liver tumor segmentation result is more accurate and reliable.
[0096] The average indicators of various evaluation indicators obtained by each segmentation method through the test set are shown in Table 2:
[0097] Table 2
[0098]
[0099] In Table 2, we can see that by comparing CFCNs, SEU-Nets, and 3DA-U-Net, it can be proved that whether it is pure channel attention or mixed attention, the addition of the attention mechanism can effectively improve the overall segmentation accuracy. Compared with the SEU-Nets liver tumor segmentation algorithm, the segmentation algorithm of 3DA-U-Nets in this paper improves the liver tumor segmentation accuracy by 1.03%, especially for the edge segmentation effect, with high accuracy. It can be proved that the three-dimensional attention mechanism learns features from more aspects than the channel attention mechanism, which effectively improves the performance of the network. The experiments prove the superiority and accuracy of the 3DA-U-Net liver tumor segmentation algorithm proposed in this paper.
[0100] Example 2
[0101] In a second aspect, this embodiment provides a CT image liver tumor segmentation device, including a processor and a storage medium;
[0102] The storage medium is used to store instructions;
[0103] The processor is used to operate according to the instructions to execute the steps of the method according to embodiment 1.
[0104] Example 3
[0105] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.
[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0110] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for segmenting liver tumors in CT images. It is characterized in that include: Obtain a CT image that includes the liver tumor; The liver region is segmented from the CT image using a first-level liver segmentation network based on 3DA-U-Net, and the liver region is converted into a binary liver mask according to a preset threshold. performing a mask operation on the CT image and the liver mask to obtain a liver region of interest; Using a secondary tumor segmentation network based on 3DA-U-Net to extract features of the liver region of interest, and obtain deep features of the CT image; Outputting a liver tumor segmentation result according to the deep features of the CT image; The network structure of 3DA-U-Nets includes: First-level liver segmentation network: input port input, convolution layer Conv 1-1, convolution layer Conv 1-2, 3DA layer 1, pooling layer Maxpool-1, convolution layer Conv 2-1, convolution layer Conv 2-1, 3DA layer 2, pooling layer Maxpool-2, convolution layer Conv3-1, convolution layer Conv 3-2, 3DA layer 3, pooling layer Maxpool-3, convolution layer Conv 4-1, convolution layer Conv 4-2, 3DA layer 4, pooling layer Maxpool-4, convolution layer Conv 5-1, convolution layer Conv 5-2, 3DA layer 5, upper convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, up convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output; Secondary tumor segmentation network: input port input, convolution layer Conv 1-1, convolution layer Conv 1-2, 3DA layer 1, pooling layer Maxpool-1, convolution layer Conv 2-1, convolution layer Conv 2-1, 3DA layer 2, pooling layer Maxpool-2, convolution layer Conv3-1, convolution layer Conv 3-2, 3DA layer 3, pooling layer Maxpool-3, convolution layer Conv 4-1, convolution layer Conv 4-2, 3DA layer 4, pooling layer Maxpool-4, convolution layer Conv 5-1, convolution layer Conv 5-2, 3DA layer 5, upper convolution layer Upconv-1, convolution layer Conv 6_1, convolution layer Conv 6-2, upper convolution layer Upconv-2, convolution layer Conv 7-1, convolution layer Conv 7-2, upper convolution layer Upconv-3, convolution layer Conv 8-1, convolution layer Conv 8-2, up convolution layer Upconv-4, convolution layer Conv 9-1, convolution layer Conv 9-2, convolution layer Conv 10-1 and output port output.
2. The method for segmenting liver tumors in CT images according to claim 1, It is characterized in that The method for constructing the 3DA-U-Net of the primary liver segmentation network and the secondary tumor segmentation network includes: Get the U-Net network and 3D attention mechanism; Among them, the contraction path of the U-Net network includes five network modules connected in sequence. The first, second, third, and fourth network modules all include 2 convolutional layers and 1 pooling layer, and the fifth network module includes 2 convolutional layers; The three-dimensional attention mechanism is recorded as 3DA; after adding the 3DA module to each convolution module in the U-Net network, a 3DA-U-Net network is formed.
3. The method for segmenting liver tumors in CT images according to claim 2, It is characterized in that The processing flow of the three-dimensional attention mechanism 3DA in 3DA-U-Net includes: The input convolutional layer processes the feature map X, which has C channels and W×H size. The 3DA structure copies the feature map X three times and performs global average pooling operations along three different directions to obtain one-dimensional features of three different dimensions, which are W×1×1, 1×H×1, and 1×1×C tensors respectively. The one-dimensional features of three different dimensions are respectively sent to two fully connected layers. The number of channels of the first fully connected layer is C / r, where r is the reduction rate, and the activation function ReLU is passed; the output is then input into the second fully connected layer. The number of channels of this layer is C, and the activation function sigmoid is used for nonlinear expression to obtain new W×1×1, 1×H×1 and 1×1×C tensors, which contain the weight information between each element of the feature map and the target feature; Multiply the new W×1×1, 1×H×1, and 1×1×C tensors containing weight information with the initial feature map X to obtain a new feature map X' recalibrated by 3DA.
4. The method for segmenting liver tumors in CT images according to claim 1, It is characterized in that The training process of the first-level liver segmentation network based on 3DA-U-Net includes: Preprocess the CT images containing liver tumors to obtain a training sample set; Initialize the network parameters of 3DA-U-Net and input the training samples into 3DA-U-Net in sequence; After each 2 iterations, the network loss function is calculated according to the updated liver segmentation image; the network parameters of the 3DA-U-Net are iteratively updated according to the loss function to obtain an updated 3DA-U-Net; and an updated liver segmentation image is obtained according to the updated 3DA-U-Net; Convergence is judged based on the network loss function. When the network loss function converges, the iteration ends and the trained first-level liver segmentation network based on 3DA-U-Net is output.
5. The method for segmenting liver tumors in CT images according to claim 1, It is characterized in that The training process of the secondary tumor segmentation network based on 3DA-U-Net includes: The liver region of interest containing liver tumors is used as the training sample set; the network parameters of the first-level liver segmentation network based on 3DA-U-Net are loaded as the initialization of the second-level network parameters, and the liver region of interest training sample set is input into the 3DA-U-Net second-level network in sequence to fine-tune the network parameters of the second-level network; after each 2 iterations, the network loss function is calculated according to the updated tumor segmentation image; convergence judgment is made according to the network loss function, and when the network loss function converges, the iteration is terminated and the trained second-level tumor segmentation network based on 3DA-U-Net is output.
6. The method for segmenting liver tumors in CT images according to claim 1, It is characterized in that During the training process of the 3DA-U-Net network, the network parameters include learning rate and batch size. The learning rate is initialized to 0.0001 and the batch size is initialized to 5.
7. The method for segmenting liver tumors in CT images according to claim 4 or 5, It is characterized in that The loss function is the Dice coefficient, and the calculation formula of the loss function is as follows: Where A is the sample predicted by the model, and B is the gold standard in the dataset; for two given images A and B, their overlapping area is |A∩B|, and |A|+|B| is the sum of the areas of the two images; the DiceLoss(A, B) loss function changes continuously during training to provide direction for the iteration of the network, and its value range is [0, 1].
8. A CT image liver tumor segmentation device, It is characterized in that including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 7.
9. A storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
