A 3D deep learning-based automatic tumor segmentation method and system

Abstract

The application discloses a kind of based on 3D deep learning tumor automatic segmentation method and system, method includes: obtaining the multiple types CT image to be segmented, the multiple types CT image to be segmented includes enhanced CT arterial portal phase image, enhanced CT hepatic arterial phase image and low-dose computed tomography image;Introduce double U-Net architecture, construct three-dimensional convolution image segmentation network;Based on three-dimensional convolution image segmentation network, the multiple types CT image to be segmented is segmented and handled, and network image segmentation result is obtained.The embodiment of the application can improve the segmentation accuracy of liver, lung and tumor, and reduce the error index of dosimetry.The present application can be widely applied in the field of medical image segmentation technology.

Description

Technical Field

[0001] This application relates to the field of medical image segmentation technology, and in particular to an automatic tumor segmentation method and system based on 3D deep learning. Background Technology

[0002] Y-90 microsphere radioembolization (RE) is an effective treatment for inoperable liver cancer and metastatic tumors, selectively destroying nearby tumor cells using beta rays. Patient selection and the corresponding injection activity (IA) for Y-90RE depend on parameters determined by pre-treatment Tc-99m-MAA SPECT or planar imaging, such as lung shunt fraction (LSF) and tumor-to-normal liver ratio (TNR), while delineating the contours of the lung, liver, and tumor is essential. In current clinical practice, the subdivision of these target volumes (VOIs) requires significant effort from radiologists.

[0003] Currently, deep learning (DL) algorithms, especially convolutional neural networks (CNNs), can be used for automatic segmentation of different organs in medical imaging. However, related technical solutions only use single-modal images for segmentation. Due to the limited information carried by single-modal images, the segmentation accuracy is poor, making it difficult to obtain accurate segmentation results that meet actual clinical requirements. Therefore, the results of deep learning-based medical image segmentation are not ideal.

[0004] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0005] The main objective of this application is to propose an automatic tumor segmentation method and system based on 3D deep learning, which can improve the segmentation accuracy of liver, lung and tumor by reducing image dosimetry error.

[0006] To achieve the above objectives, one aspect of this application proposes an automatic tumor segmentation method based on 3D deep learning, the method comprising:

[0007] Acquire multiple types of CT images to be segmented, including enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images;

[0008] A dual U-Net architecture is introduced to construct a three-dimensional convolutional image segmentation network;

[0009] The three-dimensional convolutional image segmentation network is used to segment the various types of CT images to be segmented, and the network image segmentation results are obtained.

[0010] In some embodiments, acquiring the multi-type CT images to be segmented includes:

[0011] The low-dose computed tomography (CT) images were obtained by injecting a radionuclide tracer into the patient and performing CT scan imaging.

[0012] The enhanced CT arterial portal vein angiography image was obtained by injecting contrast agent into the patient's superior mesenteric artery and performing CT scan imaging.

[0013] The enhanced CT hepatic artery angiography image is obtained by injecting contrast agent into the patient's common hepatic artery and performing CT scan imaging.

[0014] The enhanced CT portal vein angiography image, the enhanced CT hepatic artery angiography image, and the low-dose computed tomography image are combined to construct the multi-type CT image to be segmented.

[0015] In some embodiments, the three-dimensional convolutional image segmentation network includes a first branch U-Net network, a second branch U-Net network, and a convolutional activation module, wherein:

[0016] The first branch U-Net network includes a first branch downsampling module and a first branch upsampling module, which are connected by a skip connection. The first branch downsampling module includes a first branch first downsampling layer, a first branch second downsampling layer, a first branch third downsampling layer, a first branch fourth downsampling layer, and a first branch fifth downsampling layer. The first branch upsampling module includes a first branch first upsampling layer, a first branch second upsampling layer, a first branch third upsampling layer, a first branch fourth upsampling layer, a first branch fifth upsampling layer, and a first branch sixth upsampling layer.

[0017] The second branch U-Net network includes a second branch downsampling module and a second branch upsampling module. The second branch downsampling module and the second branch upsampling module are connected by a jump connection. The output of the first branch downsampling module is bridged with the output of the second branch downsampling module. The second branch downsampling module includes a second branch first downsampling layer, a second branch second downsampling layer, a second branch third downsampling layer, a second branch fourth downsampling layer, and a second branch fifth downsampling layer. The second branch upsampling module includes a second branch first upsampling layer, a second branch second upsampling layer, a second branch third upsampling layer, a second branch fourth upsampling layer, a second branch fifth upsampling layer, and a second branch sixth upsampling layer.

[0018] Both the second branch downsampling module and the second branch upsampling module include a three-dimensional convolution module and a convolution kernel module. The three-dimensional convolution module includes a three-dimensional depth convolution block, a group normalization layer, a first three-dimensional convolution block, a Gaussian error linear unit, and a second three-dimensional convolution block.

[0019] In some embodiments, the expression for the loss function of the three-dimensional convolutional image segmentation network is as follows:

[0020]

[0021] In the above formula, Loss represents the loss function of the 3D convolutional image segmentation network, A represents the total number of classes, N represents the total number of voxels, T represents the one-hot encoding of ground reality segmentation, I represents the image size, P represents the predicted value of each class, a represents the class element, and i represents the voxel element.

[0022] In some embodiments, the segmentation process of the multi-type CT images to be segmented based on the three-dimensional convolutional image segmentation network to obtain the network image segmentation result includes:

[0023] The multi-type CT images to be segmented are input into the three-dimensional convolutional image segmentation network;

[0024] Based on the first branch of the three-dimensional convolutional image segmentation network, the low-dose computed tomography image is identified and segmented to obtain a lung segmentation image and a first liver segmentation image.

[0025] Based on the second branch of the three-dimensional convolutional image segmentation network, the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are identified and segmented to obtain a liver tumor segmentation image and a second liver segmentation image.

[0026] Based on the convolutional activation module of the three-dimensional convolutional image segmentation network, the first liver segmentation image and the second liver segmentation image are subjected to convolutional mapping processing to obtain a normal liver segmentation image.

[0027] The network image segmentation result is obtained by integrating the lung segmentation image, the liver tumor segmentation image, and the normal liver segmentation image.

[0028] In some embodiments, the U-Net network, based on the second branch of the three-dimensional convolutional image segmentation network, performs recognition and segmentation processing on the enhanced CT portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain a liver tumor segmentation image and a second liver segmentation image, including:

[0029] The enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are input into the second branch of the three-dimensional convolutional image segmentation network, U-Net network;

[0030] Based on the three-dimensional convolutional module of the second branch U-Net network, three-dimensional feature extraction processing is performed on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain three-dimensional liver tumor segmentation features;

[0031] Based on the convolution kernel module of the second branch U-Net network, global feature extraction processing is performed on the enhanced CT arterial portal vein angiography image, the enhanced CT hepatic artery angiography image, and the output of the first branch downsampling module of the first branch U-Net network to obtain global liver tumor segmentation features;

[0032] The three-dimensional liver tumor segmentation features and the global liver tumor segmentation features are convolutionally mapped to obtain the liver tumor segmentation image and the second liver segmentation image.

[0033] In some embodiments, the three-dimensional convolutional module based on the second branch U-Net network performs three-dimensional feature extraction processing on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain three-dimensional liver tumor segmentation features, including:

[0034] The enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are input into the three-dimensional convolution module of the second branch U-Net network;

[0035] Based on the three-dimensional depth convolution block of the three-dimensional convolution module, three-dimensional depth feature extraction processing is performed on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain deep three-dimensional liver tumor segmentation features.

[0036] Based on the group normalization layer of the three-dimensional convolutional module, the deep three-dimensional liver tumor segmentation features are normalized to obtain the normalized deep three-dimensional liver tumor segmentation features.

[0037] Based on the first three-dimensional convolution block of the three-dimensional convolution module, three-dimensional feature extraction processing is performed on the normalized deep three-dimensional liver tumor segmentation features to obtain preliminary three-dimensional liver tumor segmentation features.

[0038] Based on the Gaussian error linear unit of the three-dimensional convolution module, the preliminary three-dimensional liver tumor segmentation features are linearized to obtain the linearized three-dimensional liver tumor segmentation features.

[0039] Based on the second three-dimensional convolutional block of the three-dimensional convolutional module, feature extraction processing is performed on the linearized three-dimensional liver tumor segmentation features to obtain the three-dimensional liver tumor segmentation features.

[0040] In some embodiments, the method further includes preprocessing the multi-type CT images to be segmented to construct a standard image segmentation result, wherein:

[0041] The enhanced CT arterial portal vein angiography image and the low-dose computed tomography image are manually aligned to obtain aligned enhanced CT arterial portal vein angiography image and aligned low-dose computed tomography image.

[0042] The aligned enhanced CT arterial portal vein angiography image and the aligned low-dose computed tomography image are transformed by rigid + b-spline transformation to obtain the transformation matrix;

[0043] The enhanced CT hepatic artery angiography image is aligned based on the transformation matrix to obtain an aligned enhanced CT hepatic artery angiography image.

[0044] The aligned enhanced CT portal vein angiography image and the aligned enhanced CT hepatic artery angiography image are subjected to tumor segmentation processing to obtain a tumor liver image and a normal liver image.

[0045] A lung delineation map is obtained by processing the aligned low-dose computed tomography image into a lung map.

[0046] The standard image segmentation result is constructed by integrating the tumor liver image, the normal liver image, and the lung depiction image.

[0047] In some embodiments, the method further includes performing physical and dosimetric analysis on the standard image segmentation result and the network image segmentation result to obtain dosimetric analysis results, wherein:

[0048] The standard image segmentation result and the network image segmentation result are subjected to segmentation performance evaluation processing to obtain a performance segmentation evaluation result. The segmentation performance evaluation includes the Dice index and the Hausdorff distance index. The Dice index is used to evaluate the proportion of the overlapping part volume to the volume of the network image segmentation result and the volume of the standard image segmentation result. The Hausdorff distance index is used to evaluate the distance between the network image segmentation result and the standard image segmentation result.

[0049] The standard image segmentation results and the network image segmentation results are subjected to dosimetric evaluation processing to obtain dosimetric evaluation results, which include the ratio of tumor to normal liver, lung shunt coefficient, absorbed dose and Y-90 injection dose.

[0050] The dosimetric index analysis results are obtained by combining the performance segmentation evaluation results with the dosimetric evaluation results.

[0051] To achieve the above objectives, another aspect of this application proposes an automatic tumor segmentation system based on 3D deep learning, the system comprising:

[0052] The acquisition module is used to acquire multiple types of CT images to be segmented, including enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images.

[0053] The building block is used to introduce the dual U-Net architecture and construct a 3D convolutional image segmentation network;

[0054] The segmentation module is used to segment the multi-type CT images to be segmented based on the three-dimensional convolutional image segmentation network to obtain the network image segmentation result.

[0055] The embodiments of this application include at least the following beneficial effects: This application provides a method and system for automatic tumor segmentation based on 3D deep learning. This scheme constructs multimodal CT images of multiple types to be segmented by acquiring enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images. It performs joint segmentation of the liver, lungs, and tumors. Furthermore, based on a dual U-Net architecture, a three-dimensional convolutional image segmentation network is constructed. The three-dimensional convolutional image segmentation network is used to segment the multiple types of CT images to be segmented, which can improve the segmentation accuracy of the liver, lungs, and tumors by reducing image dosimetric errors. Attached Figure Description

[0056] Figure 1 This is a flowchart of an automatic tumor segmentation method based on 3D deep learning provided in an embodiment of this application;

[0057] Figure 2 This is a schematic diagram of the structure of an automatic tumor segmentation system based on 3D deep learning provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the multi-type CT images of the first patient to be segmented, provided in an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the multi-type CT images of the second patient to be segmented, provided in an embodiment of this application;

[0060] Figure 5 This is a schematic diagram of the network image segmentation results provided in the embodiments of this application;

[0061] Figure 6 This is a schematic diagram of a liver tumor segmentation image provided in an embodiment of this application;

[0062] Figure 7 This is a schematic diagram of the structure of the three-dimensional convolutional image segmentation network provided in the embodiments of this application;

[0063] Figure 8 This is a schematic diagram of the structure of the three-dimensional convolution module provided in the embodiments of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of systems and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0065] It is understood that the terms “first,” “second,” etc., used in this application may be used herein to describe various concepts, but unless otherwise stated, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the words “if,” “when,” or “in response to a determination” as used herein may be interpreted as “when…” or “when…” or “in response to a determination.”

[0066] As used in this application, the terms "at least one", "multiple", "each", "any", etc., "at least one" includes one, two or more, "multiple" includes two or more, "each" refers to each of the corresponding multiples, and "any" refers to any one of the multiples.

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0068] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0069] Y-90 radioembolization (RE) is an effective treatment for inoperable liver tumors. Pre-treatment imaging using Tc-99m macroalbumin (MAA) SPECT / CT is performed, followed by pre-treatment planning that requires segmentation of the lungs, liver, and tumor. CT arterial portal angiography (CTAP) and CT hepatic arteriography (CTHA) provide structural and blood supply information for segmenting liver tumors and normal liver tissue, while low-dose computed tomography images from Tc-99m SPECT / CT can be used for lung segmentation.

[0070] Reference Figure 1 , Figure 1 A flowchart illustrating an automatic tumor segmentation method based on 3D deep learning, provided in an embodiment of the present invention, is shown below. Figure 1 The method includes the following steps:

[0071] S100. Acquire multiple types of CT images to be segmented, including enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images.

[0072] It should be noted that in some embodiments, step S100 may include: S110, obtaining the low-dose computed tomography image by injecting a radionuclide tracer into the patient and performing CT scanning imaging; S120, obtaining the enhanced CT arterial portal vein angiography image by injecting a contrast agent into the patient's superior mesenteric artery and performing CT scanning imaging; S130, obtaining the enhanced CT hepatic artery angiography image by injecting a contrast agent into the patient's common hepatic artery and performing CT scanning imaging; S140, constructing the multi-type CT image to be segmented by combining the enhanced CT arterial portal vein angiography image, the enhanced CT hepatic artery angiography image, and the low-dose computed tomography image.

[0073] In some specific embodiments, data from 57 patients were collected, including Tc-99m-MAA SPECT / CT, CTAP, and CTHA scans for each patient. Seven patients underwent contrast enhancement in both the left hepatic artery (LHA) and right hepatic artery (RHA), resulting in two sets of CTHA images. Therefore, the analysis included data from 64 Tc-99m-MAA SPECT / CT, CTAP, and CTHA cases, with two representative cases as follows: Figure 3 and Figure 4 As shown. The characteristics of T patients are listed in Table 1. Among them, such as Figure 3 and Figure 4As shown, the radiologist manually sampled low-dose computed tomography (CT), SPECT / CT, CTAP, and CTHA images for two patients, along with corresponding images of normal liver (red), lung (green), and tumor (blue). Normal liver is enhanced on CTAP, and tumor is enhanced on CTHA. Figure 3 The tumor in the patient showed enhancement only on CTHA, while the normal liver showed enhancement only on CTAP. Figure 4 Patients whose cysts did not enhance on both CTHA and CTAP (yellow arrows).

[0074] The SPECT image voxel size is 4.42×4.42×4.42mm. 3 The matrix size is 128×128×128. The corresponding low-dose computed tomography (CT) image voxel size is 0.98×0.98×3.75mm. 3 The matrix size was 512×512×108. To obtain CECT scans, arterial access was established by placing one catheter in the superior mesenteric artery for CTAP and another catheter in the common hepatic artery for CTHA. In CTAP, Ultravist 370 was injected at a rate of 2.0–2.5 mL / s, and a CT scan was performed 40 seconds after injection. In CTHA, the same contrast agent was injected at a rate of 0.5–1.2 mL / s, and a CT scan was performed 8 seconds after injection. Reconstruction of CTAP and CTHA scans performed at 120 kVp and 515 mA yielded a matrix size of 512×512×(29–53) and a voxel size of (0.54–0.79)×(0.54–0.79)×5.00 mm. 3 .

[0075] Table 1 Patient demographics in this application embodiment

[0076]

[0077] In summary, three types of images were acquired. The first type is SPECT / CT, or low-dose computed tomography (LTCT) images. These were obtained on a SPECT / CT scanner after the patient received a pre-injection of Tc-99m MAA. The second type is CTAP, or enhanced CT portal vein angiography images. Enhanced CT images were obtained using a CT scanner, with Ultravisist 370 injected into the superior mesenteric artery and imaged 40 seconds later. The third type is CTHA, or enhanced CT hepatic arteriography images. Enhanced CT images were obtained using a CT scanner, with Ultravisist 370 injected into the common hepatic artery and imaged 8 seconds later.

[0078] S200 introduces a dual U-Net architecture to construct a three-dimensional convolutional image segmentation network;

[0079] It should be noted that in some embodiments, the 3D convolutional image segmentation network includes a first branch U-Net network, a second branch U-Net network, and a convolutional activation module. The first branch U-Net network includes a first branch downsampling module and a first branch upsampling module, which are connected via skip connections. The first branch downsampling module includes a first branch first downsampling layer, a first branch second downsampling layer, a first branch third downsampling layer, a first branch fourth downsampling layer, and a first branch fifth downsampling layer. The first branch upsampling module includes a first branch first upsampling layer, a first branch second upsampling layer, a first branch third upsampling layer, a first branch fourth upsampling layer, a first branch fifth upsampling layer, and a first branch sixth upsampling layer. The second branch U-Net network includes a second branch downsampling module and a second branch upsampling module. The second branch downsampling module and the second branch upsampling module are connected via a jump connection. The output of the first branch downsampling module is bridged with the output of the second branch downsampling module. The second branch downsampling module includes a second branch first downsampling layer, a second branch second downsampling layer, a second branch third downsampling layer, a second branch fourth downsampling layer, and a second branch fifth downsampling layer. The second branch upsampling module includes a second branch first upsampling layer, a second branch second upsampling layer, a second branch third upsampling layer, a second branch fourth upsampling layer, a second branch fifth upsampling layer, and a second branch sixth upsampling layer. Both the second branch downsampling module and the second branch upsampling module include a three-dimensional convolution module and a convolution kernel module. The three-dimensional convolution module includes a three-dimensional depth convolution block, a group normalization layer, a first three-dimensional convolution block, a Gaussian error linear unit, and a second three-dimensional convolution block.

[0080] In some specific embodiments, such as Figure 7As shown, the first U-Net branch uses the nnU-Net network for automatic configuration, taking low-dose computed tomography images as input to segment the lungs and normal liver. The second U-Net branch, enhanced by the 3DConvNeXt module, uses the first convolutional module in each upsampling layer of the second branch, taking co-registered CTAP and CTHA as input, to generate segments of the normal liver and tumor. The 3D ConvNeXt module consists of a DW convolutional layer using 3×3×3 kernels and 1 stride, followed by a group normalization, then a 1×1×1 convolutional layer that quadruples the number of channels to the input channels, combined with a Gaussian error linear unit (GELU) activation function. Finally, another 1×1×1 convolutional layer maps the number of channels back to the original input channel number and connects the input and output residuals. Both branches integrate 1×1×1 convolutional layers and SoftMax activations to achieve depth supervision in the decoder. Skip connections in the first U-Net branch are connected to corresponding skip connections in the second U-Net branch to share liver and lung information. Therefore, normal liver segmentation results are provided jointly by the two branches.

[0081] S300. Based on the three-dimensional convolutional image segmentation network, the multi-type CT images to be segmented are segmented to obtain the network image segmentation result;

[0082] It should be noted that in some embodiments, step S300 may include: S310, inputting the multi-type CT images to be segmented into the three-dimensional convolutional image segmentation network; S320, performing recognition and segmentation processing on the low-dose computed tomography images based on the first branch U-Net network of the three-dimensional convolutional image segmentation network to obtain lung segmentation images and first liver segmentation images; S330, performing recognition and segmentation processing on the enhanced CT arterial portal vein angiography images and the enhanced CT hepatic artery angiography images based on the second branch U-Net network of the three-dimensional convolutional image segmentation network to obtain liver tumor segmentation images and second liver segmentation images; S340, performing convolution mapping processing on the first liver segmentation images and the second liver segmentation images based on the convolution activation module of the three-dimensional convolutional image segmentation network to obtain normal liver segmentation images; S350, integrating the lung segmentation images, the liver tumor segmentation images, and the normal liver segmentation images to obtain the network image segmentation result.

[0083] It should be noted that the first U-Net outputs a whole liver image, while the second U-Net, combined with a convolutional activation module, outputs a normal liver image.

[0084] In some embodiments, step S330 may include: S331, inputting the enhanced CT portal vein angiography image and the enhanced CT hepatic artery angiography image into the second branch U-Net network of the three-dimensional convolutional image segmentation network; S332, performing three-dimensional feature extraction processing on the enhanced CT portal vein angiography image and the enhanced CT hepatic artery angiography image based on the three-dimensional convolution module of the second branch U-Net network to obtain three-dimensional liver tumor segmentation features; S333, performing global feature extraction processing on the enhanced CT portal vein angiography image and the enhanced CT hepatic artery angiography image and the output of the first branch downsampling module of the first branch U-Net network based on the convolution kernel module of the second branch U-Net network to obtain global liver tumor segmentation features; S334, performing convolution mapping processing on the three-dimensional liver tumor segmentation features and the global liver tumor segmentation features to obtain the liver tumor segmentation image.

[0085] In some embodiments, step S332 may include: S3321, inputting the enhanced CT portal vein angiography image and the enhanced CT hepatic artery angiography image into the three-dimensional convolutional module of the second branch U-Net network; S3322, performing three-dimensional depth feature extraction processing on the enhanced CT portal vein angiography image and the enhanced CT hepatic artery angiography image based on the three-dimensional depth convolutional block of the three-dimensional convolutional module to obtain deep three-dimensional liver tumor segmentation features; S3323, performing normalization processing on the deep three-dimensional liver tumor segmentation features based on the group normalization layer of the three-dimensional convolutional module to obtain normalized... S3324. Based on the first three-dimensional convolutional block of the three-dimensional convolutional module, perform three-dimensional feature extraction processing on the normalized deep three-dimensional liver tumor segmentation features to obtain preliminary three-dimensional liver tumor segmentation features; S3325. Based on the Gaussian error linear unit of the three-dimensional convolutional module, perform linearization processing on the preliminary three-dimensional liver tumor segmentation features to obtain linearized three-dimensional liver tumor segmentation features; S3326. Based on the second three-dimensional convolutional block of the three-dimensional convolutional module, perform feature extraction processing on the linearized three-dimensional liver tumor segmentation features to obtain the three-dimensional liver tumor segmentation features.

[0086] In some specific embodiments, the first is a standard 3D U-Net, consisting of two parts: six upsampling layers and five downsampling layers. Each layer includes two convolutional blocks. The first convolutional block uses a 3×3×3 convolutional kernel with a stride of 1, followed by an instance normalization layer and a Rectified Linear Unit (ReLU). The second convolutional block has the same structure as the first, except that the stride of the first convolutional kernel is (1, 2, 2) or (2, 2, 2). Furthermore, skip connections between the upsampling and downsampling layers bridge to skip connections in each corresponding layer of the second U-Net network. These bridging skip connections allow information about lung segmentation to be shared with the second U-Net.

[0087] In the second U-net network, the first convolutional block of each downsampling layer is replaced with a 3DConvNeXt block, as shown below. Figure 8 As shown, 3D ConvNeXt consists of a depthwise separable convolutional layer using a 3×3×3 kernel and a stride of 1, followed by group normalization, then a 1×1×1 convolutional layer that quadruples the number of channels compared to the input, followed by a Gaussian error linear unit activation function. Finally, another 1×1×1 convolutional layer maps the number of channels back to the original input number, and residuals are used to connect the input and output of this convolutional block. The 3D ConvNeXt block is used to handle bridging skip connections, and the depthwise separable convolutional layer is considered to have global awareness, enabling better handling of mixed information.

[0088] The network output is compared to the gold standard graph, and the sum of the dice roll and cross-entropy is used as the training loss function, which is expressed as:

[0089]

[0090] In the above formula, Loss represents the loss function of the 3D convolutional image segmentation network, A represents the total number of classes, N represents the total number of voxels, T represents the one-hot encoding of ground reality segmentation, I represents the image size, P represents the predicted value of each class, a represents the class element, and i represents the voxel element.

[0091] This invention also includes preprocessing the multi-type CT images to be segmented to construct standard image segmentation results;

[0092] It should be noted that in some embodiments, the enhanced CT portal vein angiography image and the low-dose computed tomography (CT) image are manually aligned to obtain aligned enhanced CT portal vein angiography image and aligned low-dose CT image; the aligned enhanced CT portal vein angiography image and the aligned low-dose CT image are transformed by rigid + b-spline transformation to obtain a transformation matrix; the enhanced CT hepatic artery angiography image is aligned based on the transformation matrix to obtain aligned enhanced CT hepatic artery angiography image; tumor segmentation is performed on the aligned enhanced CT portal vein angiography image and the aligned enhanced CT hepatic artery angiography image to obtain a whole liver segmentation image, which is then split into a tumor liver image and a normal liver image; lung image rendering is performed on the aligned low-dose CT image to obtain a lung depiction image; the tumor liver image, the normal liver image, and the lung depiction image are integrated to construct the standard image segmentation result.

[0093] In some specific embodiments, CTHA images are aligned with CTAP images using a rigid +b spline transform. Low-dose computed tomography (LDCT) images and CTAP are initially manually aligned, then a rigid +b spline transform is performed using coarse liver masks in both modes to register CTAP to the LDCT images. The transform matrix is ​​then applied to CTHA to ensure alignment of the three CT images. All registrations are performed using Elastix. Tumor atlases and normal liver maps are segmented on co-registered CTAP and CTHA images by experienced radiologists, excluding tumors throughout the liver map, as the gold standard, i.e., the standard image segmentation result. A total of 355 tumors were segmented from 57 patients. Lung maps are depicted using a -400 HU threshold and manually fine-tuned on the LDCT images as the gold standard. Other preprocessing, including resampling, normalization, and data splitting, is performed within the nnU-Net pipeline.

[0094] This invention also includes performing dosimetric analysis on the standard image segmentation results and the network image segmentation results to obtain dosimetric analysis results;

[0095] It should be noted that in some embodiments, the standard image segmentation result and the network image segmentation result are subjected to segmentation performance evaluation processing to obtain performance segmentation evaluation results. The segmentation performance evaluation includes the Dice index and the Hausdorff distance index. The Dice index is used to evaluate the proportion of the overlapping part volume to the volume of the network image segmentation result and the volume of the standard image segmentation result. The Hausdorff distance index is used to evaluate the distance between the network image segmentation result and the standard image segmentation result. The volume similarity coefficient is used to evaluate the volume difference between the network image segmentation result and the standard image segmentation result. The standard image segmentation result and the network image segmentation result are subjected to dosimetric evaluation processing to obtain dosimetric evaluation results. The dosimetric evaluation includes the tumor to normal liver ratio, lung shunt coefficient, absorbed dose, and Y-90 injection dose. The dosimetric index analysis results are obtained by combining the performance segmentation evaluation results and the dosimetric evaluation results.

[0096] In some specific embodiments, the segmentation performance of different networks on target VOIs (normal liver, lung, and tumor) was first compared using Dice (Dice coefficient) and Hausdorff distance (Hausdorff distance exponent HD) at 95%. The Dice coefficient measures the volume overlap between the segmentation result and the ground truth, and the Dice exponent is expressed as:

[0097]

[0098] Among them, V network output V represents the segmented quantities generated by different network outputs. ground truth It is a segmentation of the ground condition value.

[0099] The physical distance between the 95% measurement segmentation result and the actual ground condition, expressed as the Hausdorf distance index HD, is:

[0100] HD95% = max 95％ [d(X,Y),d(Y,X)]

[0101] Where X represents different network outputs, Y represents the ground truth, d(X,Y)max represents the minimum distance between surfaces X and Y, and d(X,Y)max represents the maximum value. 95％ The 95th percentile.

[0102] Y-90RE treatment planning metrics, including LSF (lung shunt coefficient), TNR (tumor-to-normal liver ratio), absorbed dose (AD) of the target VOI, and IA (injection activity), are also compared with the gold standard fractionation based on the segmentation volume of different networks to assess their clinical relevance.

[0103] LSF is used to estimate microspheres shunted to the lungs by Y-90RE. The expression for the lung shunt coefficient is:

[0104]

[0105] Among them, A lungs and A liver These are images of lung activity and the entire liver.

[0106] TNR measures the relative concentration or activity of tumor tissue compared to surrounding normal liver tissue. The expression for the ratio of tumor to normal liver tissue is:

[0107]

[0108] Among them, M tumors and M normalliver These refer to the quality of the tumor and the normal liver, respectively.

[0109] The AD for each VOI is calculated using the MIRD model, and the expression for the target volume absorbed dose is:

[0110]

[0111] 49.67 (J / GBq) is a constant used to convert the activity of Y-90 to AD.

[0112] The calculation method for IA (effective dose base) is as follows:

[0113]

[0114] 70 Gy and 30 Gy are the upper limits of AD for normal liver and lung, respectively.

[0115] Finally, ablation experiments were conducted on the 3D convolutional image segmentation network constructed in the embodiments of this application. Two U-Net structures, one sharing skip connections and the other connecting to the normal liver, improved the average Dice coefficient of the normal liver from 0.923 to 0.927, and the other improving the average Dice coefficient of the tumor from 0.734 to 0.738 or 0.741, respectively. Combining the shared skip connections and the normal liver connections further improved the segmentation performance for both the normal liver (average Dice coefficient = 0.928) and the tumor (average Dice coefficient = 0.739). The 3D ConvNeXt block achieved the best segmentation performance across all metrics for both the normal liver and the tumor, followed by the ResNet block. All segmentation results were similar in lung segmentation. The comparative results demonstrate that the proposed components—shared skip connections, normal liver connections, and the 3D ConvNeXt block—contribute to improving the model performance for this segmentation task, as shown in Table 2.

[0116] Table 2. Segmentation results of the ablation study

[0117]

[0118] Further segmentation performance studies were conducted, and the segmentation performance of different networks is shown in Table 3. The results for the sample patients are as follows: Figure 5 As shown. For normal liver segmentation, the proposed ConvNeXt-2U outperforms other products in Dice coefficient, followed by MedNeXt. MedNeXt performs best in HD95 coefficient, followed by ConvNeXt-2U. U-Net and Attention U-Net rank lower than the proposed ConvNeXt-2U network and MedNeXt, but better than Swin-UNETR. The network results are compared with... Figure 5 The average Dice coefficient > 0.9 is closely related to the segmentation results.

[0119] Table 3. Segmentation performance data of different networks trained on low-dose computed tomography images, CTAP, and CTHA.

[0120]

[0121]

[0122] For lung segmentation, all networks achieved segmentation performance with an average Dice coefficient > 0.99, such as... Figure 5 As shown, the proposed network outperforms other networks across all metrics.

[0123] For tumor segmentation, all networks performed worse than those for normal liver and lung, with an average Dice coefficient <0.8. ConvNeXt-2U outperformed the others, with an average Dice coefficient of 0.774 and an average HD95 coefficient of 19.730. MedNeXt had an average Dice coefficient of 0.746, and Attention U-Net ranked second with an average HD95 coefficient of 20.574. Swin-UNETR performed the worst across all indices. The tumor segmentation results for this patient are as follows: Figure 5 As shown.

[0124] Table 3 also shows the segmentation performance of the network for tumors >2 cm in diameter for each patient. ConvNeXt-2U also showed the best performance, with an average Dice coefficient of 0.833 and an average HD95 coefficient of 21.474. MedNeXt ranked second with an average Dice coefficient of 0.793, and U-Net ranked second with an average HD95 coefficient of 22.881, consistent with the results for all tumor cases. U-Net and Attention U-Net also achieved better performance than Swin-UNETR. Figure 6As shown, all networks exhibit better performance with larger tumors and poorer performance with smaller tumors. The training execution times for U-Net, Attention U-Net, MedNeXt, Swin-UNETR, and ConvNeXt-2U are 43h, 46h, 55h, 61h, and 53h, respectively, and the time required to predict a single patient is 61s, 65s, 73s, 85s, and 71s, respectively.

[0125] Finally, dosimetric assessments were performed, and Table 4 lists the dosimetric errors for different fractionation errors in the Y-90RE treatment plan. All IAs were calculated based on the upper limit of the absorbable dose in a normal liver. The proposed ConvNeXt-2U demonstrated the best performance among all methods. U-Net ranked second in lung shunt coefficient, with a mean absolute error of 2.56% for LSF and AD. lungs The mean absolute error was 1.98%. MedNeXt had a mean absolute error of 27.67% in TNR and AD. normal liver The mean absolute error is 12.06%, and the mean absolute AD is... tumors With an error rate of 16.57%, and an IA error rate of 9.25%, it ranked second. Swin-UNETR performed the worst.

[0126] Table 4. Absolute error data of different networks for Y-90RE treatment plan dosimetry.

[0127]

[0128] In summary, this invention proposes a novel ConvNeXt-2U network and provides a comprehensive analysis of its segmentation performance on normal liver, tumors, and lungs. It is also used in conjunction with various other deep learning networks for Y-90RE treatment planning. All networks are implemented based on the nnU-Net framework. U-Net and Attention U-Net demonstrate performance similar to those reported in the LiTS dataset for normal liver and tumor segmentation tasks. However, the performance of Attention U-Net is somewhat limited within the nnU-Net framework. Swin-UNETR, a VisionTransformer-based method, exhibits the worst performance, similar to results observed in another lung and normal liver segmentation study. Vision Transformers achieve excellent performance on large datasets, such as the BraTS dataset of 2000 patients used for brain tumor segmentation. However, for smaller datasets, the performance of Transformer-based networks tends to decline. ConvNeXt-based architectures, such as MedNeXt, outperform convolutional-based networks and transformers on smaller datasets, consistent with our research. Compared to single-branch networks, dual-branch networks can integrate multimodal image information. Therefore, our proposed ConvNeXt-2U exhibits the best performance on both physical and clinical segmentation metrics.

[0129] For segmentation of normal liver and lung, all networks achieved average Dice coefficients >0.9 and >0.99, respectively, consistent with existing literature. The lung has lower density and clearer boundaries with other tissues, making it relatively easy to segment on CT images. Figure 5 As shown, normal liver segmentation is limited by challenging tumor segmentation. No segmentation errors of surrounding organs or soft tissues were observed in any patients, likely because CTAP and CTHA are enhanced images of the liver.

[0130] For tumor segmentation, the proposed ConvNeXt-2U achieved an optimal average Dice coefficient of only 0.774. In an MRI-based segmentation study, the average Dice coefficients for inter-segmentation and intra-segmentation of liver tumors were 0.79 and 0.85, respectively. In the LiTS dataset, only low-density regions in enhanced CT were considered tumors, and the state-of-the-art segmentation method achieved a Dice coefficient of 0.76. Compared to the LiTS dataset, patients participating in Y-90RE treatment required higher tumor segmentation accuracy. Therefore, in this study, the combined set of non-enhanced regions in CTAP and enhanced regions in CTHA was considered tumors. The baseline nnU-Net achieved a Dice coefficient of 0.734.

[0131] Ablation studies demonstrate the effectiveness of each component: shared skip connections, normal liver connections, and the 3DConvNeXt component. The dual-branch architecture enhances the segmentation of the normal liver, tumors, and lungs by generating task-specific outputs in each branch. We hypothesize that the first branch, responsible for segmenting the entire liver and lungs, can utilize locational information about the normal liver and tumors obtained from the second branch. Furthermore, the ConvNeXt module has the potential to improve normal liver and tumor segmentation compared to ResNet and traditional convolutional modules because it utilizes DW convolutions, considered equivalent to self-attention, and capable of capturing global information. Therefore, it is applied to the first block of the decoder in the second branch to effectively capture global information from the cascaded skip connections.

[0132] For dosimetric metrics, our proposed ConvNeXt also demonstrated the best performance across all dosimetric metrics. Regarding its use in excluding patients with potential radiation pneumonitis, in lung segmentation, the proposed network was consistent with the results of manual segmentation by physicians, with all networks exhibiting >23% error in TNR and >12% error in AD.

[0133] Please see Figure 2 This application also provides an automatic tumor segmentation system based on 3D deep learning, which can implement the above-mentioned automatic tumor segmentation method based on 3D deep learning. The system includes:

[0134] The acquisition module is used to acquire multiple types of CT images to be segmented, including enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images.

[0135] The building block is used to introduce the dual U-Net architecture and construct a 3D convolutional image segmentation network;

[0136] The segmentation module is used to segment the multi-type CT images to be segmented based on the three-dimensional convolutional image segmentation network to obtain the network image segmentation result.

[0137] It is understood that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0138] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for automatic tumor segmentation based on 3D deep learning, characterized in that, The method includes the following steps: Acquire multiple types of CT images to be segmented, including enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images; A dual U-Net architecture is introduced to construct a three-dimensional convolutional image segmentation network; The three-dimensional convolutional image segmentation network is used to segment the multi-type CT images to be segmented, and the network image segmentation result is obtained. The specific expression for the loss function of the 3D convolutional image segmentation network is as follows: In the above formula, This represents the loss function of a 3D convolutional image segmentation network. Represents the total number of classes. This represents the total number of voxels. This represents a single-hot code representing ground reality segmentation. Indicates image size, This represents the predicted value for each class. Represents class elements, Represents voxel elements; The segmentation process of the multi-type CT images to be segmented based on the three-dimensional convolutional image segmentation network to obtain the network image segmentation result includes: The multi-type CT images to be segmented are input into the three-dimensional convolutional image segmentation network; Based on the first branch of the three-dimensional convolutional image segmentation network, the low-dose computed tomography image is identified and segmented to obtain a lung segmentation image and a first liver segmentation image. Based on the second branch of the three-dimensional convolutional image segmentation network, the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are identified and segmented to obtain a liver tumor segmentation image and a second liver segmentation image. Based on the convolutional activation module of the three-dimensional convolutional image segmentation network, the first liver segmentation image and the second liver segmentation image are subjected to convolutional mapping processing to obtain a normal liver segmentation image. The lung segmentation image, the liver tumor segmentation image, and the normal liver segmentation image are integrated to obtain the network image segmentation result; The second branch of the U-Net network based on the three-dimensional convolutional image segmentation network performs recognition and segmentation processing on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain a liver tumor segmentation image and a second liver segmentation image, including: The enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are input into the second branch of the three-dimensional convolutional image segmentation network, U-Net network; Based on the three-dimensional convolutional module of the second branch U-Net network, three-dimensional feature extraction processing is performed on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain three-dimensional liver tumor segmentation features; Based on the convolution kernel module of the second branch U-Net network, global feature extraction processing is performed on the enhanced CT arterial portal vein angiography image, the enhanced CT hepatic artery angiography image, and the output of the first branch downsampling module of the first branch U-Net network to obtain global liver tumor segmentation features; The three-dimensional liver tumor segmentation features and the global liver tumor segmentation features are convolutionally mapped to obtain the liver tumor segmentation image and the second liver segmentation image.

2. The method according to claim 1, characterized in that, The acquisition of the multi-type CT images to be segmented includes: The low-dose computed tomography (CT) images were obtained by injecting a radionuclide tracer into the patient and performing CT scan imaging. The enhanced CT arterial portal vein angiography image was obtained by injecting contrast agent into the patient's superior mesenteric artery and performing CT scan imaging. The enhanced CT hepatic artery angiography image is obtained by injecting contrast agent into the patient's common hepatic artery and performing CT scan imaging. The enhanced CT portal vein angiography image, the enhanced CT hepatic artery angiography image, and the low-dose computed tomography image are combined to construct the multi-type CT image to be segmented.

3. The method according to claim 1, characterized in that, The three-dimensional convolutional image segmentation network includes a first branch U-Net network, a second branch U-Net network, and a convolutional activation module, wherein: The first branch U-Net network includes a first branch downsampling module and a first branch upsampling module, which are connected by a skip connection. The first branch downsampling module includes a first branch first downsampling layer, a first branch second downsampling layer, a first branch third downsampling layer, a first branch fourth downsampling layer, and a first branch fifth downsampling layer. The first branch upsampling module includes a first branch first upsampling layer, a first branch second upsampling layer, a first branch third upsampling layer, a first branch fourth upsampling layer, a first branch fifth upsampling layer, and a first branch sixth upsampling layer. The second branch U-Net network includes a second branch downsampling module and a second branch upsampling module. The second branch downsampling module and the second branch upsampling module are connected by a jump connection. The output of the first branch downsampling module is bridged with the output of the second branch downsampling module. The second branch downsampling module includes a second branch first downsampling layer, a second branch second downsampling layer, a second branch third downsampling layer, a second branch fourth downsampling layer, and a second branch fifth downsampling layer. The second branch upsampling module includes a second branch first upsampling layer, a second branch second upsampling layer, a second branch third upsampling layer, a second branch fourth upsampling layer, a second branch fifth upsampling layer, and a second branch sixth upsampling layer. Both the second branch downsampling module and the second branch upsampling module include a three-dimensional convolution module and a convolution kernel module. The three-dimensional convolution module includes a three-dimensional depth convolution block, a group normalization layer, a first three-dimensional convolution block, a Gaussian error linear unit, and a second three-dimensional convolution block.

4. The method according to claim 1, characterized in that, The three-dimensional convolutional module based on the second branch U-Net network performs three-dimensional feature extraction processing on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain three-dimensional liver tumor segmentation features, including: The enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are input into the three-dimensional convolution module of the second branch U-Net network; Based on the three-dimensional depth convolution block of the three-dimensional convolution module, three-dimensional depth feature extraction processing is performed on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain deep three-dimensional liver tumor segmentation features. Based on the group normalization layer of the three-dimensional convolutional module, the deep three-dimensional liver tumor segmentation features are normalized to obtain the normalized deep three-dimensional liver tumor segmentation features. Based on the first three-dimensional convolution block of the three-dimensional convolution module, three-dimensional feature extraction processing is performed on the normalized deep three-dimensional liver tumor segmentation features to obtain preliminary three-dimensional liver tumor segmentation features. Based on the Gaussian error linear unit of the three-dimensional convolution module, the preliminary three-dimensional liver tumor segmentation features are linearized to obtain the linearized three-dimensional liver tumor segmentation features. Based on the second three-dimensional convolutional block of the three-dimensional convolutional module, feature extraction processing is performed on the linearized three-dimensional liver tumor segmentation features to obtain the three-dimensional liver tumor segmentation features.

5. The method according to claim 1, characterized in that, It also includes preprocessing the multi-type CT images to be segmented to construct standard image segmentation results, wherein: The enhanced CT arterial portal vein angiography image and the low-dose computed tomography image are manually aligned to obtain aligned enhanced CT arterial portal vein angiography image and aligned low-dose computed tomography image. The aligned enhanced CT arterial portal vein angiography image and the aligned low-dose computed tomography image are transformed by rigid + b-spline transformation to obtain the transformation matrix; The enhanced CT hepatic artery angiography image is aligned based on the transformation matrix to obtain an aligned enhanced CT hepatic artery angiography image. The aligned enhanced CT portal vein angiography image and the aligned enhanced CT hepatic artery angiography image are subjected to tumor segmentation processing to obtain a tumor liver image and a normal liver image. A lung delineation map is obtained by processing the aligned low-dose computed tomography image into a lung map. The standard image segmentation result is constructed by integrating the tumor liver image, the normal liver image, and the lung depiction image.

6. The method according to claim 5, characterized in that, It also includes performing physical and dosimetric analysis on the standard image segmentation results and the network image segmentation results to obtain dosimetric analysis results, wherein: The standard image segmentation result and the network image segmentation result are subjected to segmentation performance evaluation processing to obtain a performance segmentation evaluation result. The segmentation performance evaluation includes the Dice index and the Hausdorff distance index. The Dice index is used to evaluate the proportion of the overlapping part volume to the volume of the network image segmentation result and the volume of the standard image segmentation result. The Hausdorff distance index is used to evaluate the distance between the network image segmentation result and the standard image segmentation result. The standard image segmentation results and the network image segmentation results are subjected to dosimetric evaluation processing to obtain dosimetric evaluation results, which include the ratio of tumor to normal liver, lung shunt coefficient, absorbed dose and Y-90 injection dose. The dosimetric index analysis results are obtained by combining the performance segmentation evaluation results with the dosimetric evaluation results.

7. An automatic tumor segmentation system based on 3D deep learning, characterized in that, The system includes: The acquisition module is used to acquire multiple types of CT images to be segmented, including enhanced CT arterial portal vein angiography images, enhanced CT hepatic artery angiography images, and low-dose computed tomography images. The building module is used to introduce the dual U-Net architecture and construct a 3D convolutional image segmentation network. The specific expression of the loss function of the 3D convolutional image segmentation network is as follows: In the above formula, This represents the loss function of a 3D convolutional image segmentation network. Represents the total number of classes. This represents the total number of voxels. This represents a single-hot code representing ground reality segmentation. Indicates image size, This represents the predicted value for each class. Represents class elements, Represents voxel elements; The segmentation module is used to segment the multi-type CT images to be segmented based on the three-dimensional convolutional image segmentation network to obtain network image segmentation results. The segmentation of the multi-type CT images to be segmented based on the three-dimensional convolutional image segmentation network to obtain network image segmentation results includes: The multi-type CT images to be segmented are input into the three-dimensional convolutional image segmentation network; Based on the first branch of the three-dimensional convolutional image segmentation network, the low-dose computed tomography image is identified and segmented to obtain a lung segmentation image and a first liver segmentation image. Based on the second branch of the three-dimensional convolutional image segmentation network, the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are identified and segmented to obtain a liver tumor segmentation image and a second liver segmentation image. Based on the convolutional activation module of the three-dimensional convolutional image segmentation network, the first liver segmentation image and the second liver segmentation image are subjected to convolutional mapping processing to obtain a normal liver segmentation image. The lung segmentation image, the liver tumor segmentation image, and the normal liver segmentation image are integrated to obtain the network image segmentation result; The second branch of the U-Net network based on the three-dimensional convolutional image segmentation network performs recognition and segmentation processing on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain a liver tumor segmentation image and a second liver segmentation image, including: The enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image are input into the second branch of the three-dimensional convolutional image segmentation network, U-Net network; Based on the three-dimensional convolutional module of the second branch U-Net network, three-dimensional feature extraction processing is performed on the enhanced CT arterial portal vein angiography image and the enhanced CT hepatic artery angiography image to obtain three-dimensional liver tumor segmentation features; Based on the convolution kernel module of the second branch U-Net network, global feature extraction processing is performed on the enhanced CT arterial portal vein angiography image, the enhanced CT hepatic artery angiography image, and the output of the first branch downsampling module of the first branch U-Net network to obtain global liver tumor segmentation features; The three-dimensional liver tumor segmentation features and the global liver tumor segmentation features are convolutionally mapped to obtain the liver tumor segmentation image and the second liver segmentation image.

Images