Segmented intelligent recognition model and recognition method for liver CT images

Abstract

The present invention discloses a method for intelligent segmentation and recognition of liver CT images, comprising the following steps: S1. Using a liver segmentation model to segment the liver parenchyma region of an abdominal CT image to obtain a liver mask; S2. Multiplying the liver mask with the original input abdominal CT image to obtain a liver localization image; S3. Inputting the liver localization image into the liver segmentation model, which then performs Couinaud segmentation on the liver region. The present invention also discloses an intelligent segmentation and recognition model for liver CT images. The present invention can achieve more accurate liver segmentation recognition results.

Description

Technical Field

[0001] The present invention belongs to the technical field of CT image recognition, and in particular relates to a segmented intelligent recognition model and a recognition method for liver CT images. Background Art

[0002] The planning and navigation of liver surgery require the support of medical imaging data. Computed tomography (CT) is an important medical imaging data source in current clinical practice. To improve the accuracy of surgical planning, the technology of three-dimensional reconstruction based on multi-slice continuous liver scan images is becoming increasingly sophisticated. However, current liver surgery navigation systems require clinicians to demarcate the surgical area layer by layer, which seriously lacks intelligent operation. The accuracy of liver lesion annotation results also depends on the skill and experience of the medical experts performing the operation, and lack repeatability and reproducibility.

[0003] Currently, most automated liver segmentation methods are based on vascular segmentation. They first segment the liver, then define the vascular structure within it, and then perform Couinaud segmentation based on vascular localization. Vascular segmentation methods are primarily categorized as vessel enhancement, active contours, and tracking. These methods require a basic understanding of vascular physics or medical knowledge of vascular segmentation. Couinaud segmentation methods also require a physician to annotate voxel localization points, requiring a high level of physician experience.

[0004] With the development of deep learning, some studies have proposed using deep learning to achieve liver segmentation. However, the current application of deep learning in the field of liver segmentation is not ideal. For example, medical CT images of the liver usually include abdominal CT images of multiple organs such as the liver, gallbladder, spleen, and pancreas. Relatively speaking, the target liver segmentation accounts for a relatively small proportion in the CT image data. Existing solutions segment the liver in CT images directly based on CT images, and the results are very rough, which affects doctors' judgment and formulation of surgical plans. Summary of the Invention

[0005] The purpose of the present invention is to address the above-mentioned problems and provide a segmented intelligent recognition model and recognition method for liver CT images.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for intelligent segmentation recognition of liver CT images, comprising the following steps:

[0008] S1. Liver mask obtained by segmenting the liver parenchyma region of the abdominal CT image using the liver segmentation model;

[0009] S2. multiplying the liver mask with the original input abdominal CT image to obtain a liver positioning image;

[0010] S3. Input the liver localization image into the liver segmentation model, and the liver segmentation model performs Couinaud segmentation on the liver region.

[0011] In the above-mentioned intelligent recognition method for liver CT image segmentation, the liver segmentation model described in step S1 is a trained 3D-UNet network, including a decoder and an encoder, the encoder includes a downsampling module composed of multiple layers of encoding modules, and the decoder includes an upsampling module composed of multiple layers of decoding modules.

[0012] In the above-mentioned intelligent recognition method for liver CT image segmentation, the liver segmentation model described in step S2 is a trained adapted 3D-UNet network.

[0013] In the above-mentioned intelligent recognition method for liver CT image segmentation, the adapted 3D-UNet network includes a downsampling residual network composed of multiple layers of residual modules and an upsampling residual network composed of multiple layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module.

[0014] In the above-mentioned intelligent recognition method for liver CT image segmentation, the upsampling residual network and the downsampling residual network of the adapted 3D-UNet network each include a four-layer residual module;

[0015] The output of the first-layer residual module of the downsampling residual network is processed by an attention residual module and then summed with the output of the third-layer residual network of the upsampling residual network and input into the fourth-layer residual module;

[0016] The output of the second-layer residual module of the downsampling residual network is processed by an attention residual module, and then summed with the output of the second-layer residual module of the upsampling residual network and input into the third-layer residual module;

[0017] The output of the third-layer residual module of the downsampling residual network is processed by an attention residual module and then summed with the output of the first-layer residual module of the upsampling residual network before being input into the second-layer residual module.

[0018] In the above-mentioned intelligent recognition method for liver CT image segmentation, each residual module includes two consecutive convolution modules that use a normalization function and a ReLU function. The second convolution module takes the output of the first convolution module as input, and the output of the second convolution module is added to the original input of the residual module as the output of the residual module.

[0019] In the above-mentioned intelligent recognition method for liver CT image segmentation, the attention residual module includes two residual modules and a trunk branch and a soft mask branch located between the two residual modules. The input of the attention residual module is processed by the first residual module and then input into the trunk branch and the soft mask branch respectively. The output of the trunk branch and the output of the soft mask branch are multiplied element by element and then added element by element to the output of the trunk branch as the input of the second residual module.

[0020] In the above-mentioned intelligent recognition method for liver CT image segmentation, the main branch includes two residual modules connected front and back;

[0021] The soft mask branch includes an automatic encoder composed of an encoding module and a decoding module, and the output of the encoding module is respectively processed by a residual module and bilinear interpolation upsampling and then element-by-element summed as the input of the decoding module.

[0022] A segmented intelligent recognition model for liver CT images includes a cascaded liver segmentation model and a liver segmentation model. The liver segmentation model includes a 3D-UNet network for segmenting the liver parenchyma area of ​​abdominal CT images. The liver segmentation model includes a residual network composed of several residual modules and an attention mechanism module mixed with residuals for performing Couinaud segmentation on the liver area segmented by the liver segmentation model.

[0023] In the above-mentioned intelligent recognition model for liver CT image segmentation, the residual network includes a downsampling residual network composed of multiple layers of residual modules and an upsampling residual network composed of multiple layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module;

[0024] The residual module includes two consecutive convolution modules that use a normalization function and a ReLU function. The second convolution module takes the output of the first convolution module as input, and the output of the second convolution module is added to the original input of the residual module as the output of the residual module.

[0025] The attention residual module includes two residual modules and a trunk branch and a soft mask branch located between the two residual modules. The input of the attention residual module is processed by the first residual module and then input into the trunk branch and the soft mask branch respectively. The output of the trunk branch and the output of the soft mask branch are multiplied and added to the output of the trunk branch as the input of the second residual module.

[0026] The advantages of the present invention are:

[0027] 1. A cascade model is proposed to first coarsely locate the liver part and then finely segment the located liver part, which can obtain more accurate liver segmentation recognition results;

[0028] 2. The two cascaded models use a conventional 3D-UNet network and an adapted 3D-UNet network, respectively. The 3D-UNet network currently used for liver segmentation is used for liver parenchymal region segmentation, and the improved 3D-UNet network is then used to perform segmented recognition on the segmented liver, which can obtain more accurate recognition results.

[0029] 3. The improved 3D-UNet network further uses a mixed residual attention mechanism module at the skip connection level, which can effectively improve the recognition function of the Couinaud segmented area of ​​the liver on multiple scanning levels and further improve the recognition ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a model architecture diagram of the liver CT image segmentation intelligent recognition model of the present invention;

[0031] Figure 2 This is a flow chart of the recognition method for the liver CT image segmentation intelligent recognition model of the present invention;

[0032] Figure 3 This is a model architecture diagram of the liver segmentation model of the present invention;

[0033] Figure 4 This is a schematic diagram of the structure of the residual module used in the 3D-UNet network adapted by the present invention;

[0034] Figure 5 The structure diagram of the attention residual module that mixes residual and attention mechanisms in the liver segmentation model;

[0035] Figure 6 Schematic diagram of the structure of the soft mask branch in the attention residual module. DETAILED DESCRIPTION

[0036] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] like Figure 1 As shown, this solution addresses the problem that existing technologies cannot accurately and quickly segment the liver in abdominal CT images by Couinaud segmentation. It proposes an intelligent recognition model for liver CT image segmentation, including a cascaded liver segmentation model and a liver segmentation model. Abdominal CT images with liver areas labeled with segmentation results are used to train the liver segmentation model, and liver areas labeled with segmentation results are used to train the liver segmentation model so that the entire network has the ability to perform Couinaud segmentation on the liver in CT images.

[0038] Use the well-trained model to perform intelligent segmentation recognition of liver CT images, such as Figure 2 As shown, the specific identification method includes the following steps:

[0039] S1. Liver mask obtained by segmenting the liver parenchyma region of the abdominal CT image using the liver segmentation model;

[0040] S2. multiplying the liver mask by the original input abdominal CT image to obtain a liver localization image;

[0041] S3. Input the liver localization image into the liver segmentation model, and the liver segmentation model performs Couinaud segmentation on the liver region.

[0042] Specifically, the liver segmentation model uses a 3D-UNet network to segment the liver parenchyma region from abdominal CT images. The model comprises a decoder and an encoder. The encoder comprises a downsampling module composed of multiple layers of encoding modules, while the decoder comprises an upsampling module composed of multiple layers of decoding modules. CT data exists in three-dimensional form, and three-dimensional convolution can better mine data correlations in high-dimensional space. 3D-UNet has demonstrated excellent results for liver segmentation, and the first phase of this solution directly uses this highly effective 3D-UNet for the liver segmentation model.

[0043] Each layer of the encoding module consists of a 3*3*3 convolution kernel, a rectified linear unit (ReLU), and a maximum pooling (max pool) with a stride of 2 connected in sequence. The 2 next to the arrow in the figure indicates a stride of 2.

[0044] The decoding module of each layer includes a 2*2*2 upper convolution kernel with a stride of 2, a 3*3*3 convolution kernel and a rectified linear unit (ReLU). The skip connection in the same layer provides the high-resolution features on the encoding path to the decoding path. In the last layer, a 1*1*1 convolution is used to reduce the number of output channels.

[0045] like Figure 3 As shown in the figure, the liver segmentation model is a trained adapted 3D-UNet network. The adapted 3D-UNet network includes a downsampling residual network composed of multiple layers of residual modules and an upsampling residual network composed of multiple layers of residual modules. Each layer of residual modules can have one residual module, or two or more residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through the attention residual module.

[0046] Specifically, if Figure 3, the upsampling residual network and downsampling residual network of the adapted 3D-UNet network include four layers of residual modules respectively, and the output of the first layer residual module of the downsampling residual network is processed by an attention residual module and then summed with the output of the third layer residual network of the upsampling residual network and input into the fourth layer residual module;

[0047] The output of the second-layer residual module of the downsampling residual network is processed by an attention residual module, and then summed with the output of the second-layer residual module of the upsampling residual network and input into the third-layer residual module;

[0048] The output of the third residual module of the downsampling residual network is processed by an attention residual module, summed with the output of the first residual module of the upsampling residual network, and then input into the second residual module. The nth layer of both the upsampling residual network and the downsampling residual network is defined from left to right.

[0049] This solution further uses a mixed residual attention mechanism module on the skip connection level of the improved 3D-UNet network, which can effectively improve the recognition function of the Couinaud segmented areas of the liver on multiple scanning levels and further enhance the recognition ability of the model.

[0050] Specifically, if Figure 4 As shown in the figure, the residual module consists of two consecutive convolutional modules that use the normalization function Group Norm and the ReLU function. The second convolutional module takes the output of the first convolutional module as input. The output of the second convolutional module is added to the original input of the residual module and used as the output of the residual module. In this solution, the residual module uses Group Normalization (GN) for normalization, which performs better than Batch Normalization (BN) when the batch size is small.

[0051] Further, if Figure 5 As shown in the figure, the attention residual module consists of two residual modules and a trunk branch and a softmask branch located between the two residual modules. The input of the attention module is processed by the first residual module and then input into the trunk branch and the softmask branch respectively. The output of the trunk branch is multiplied element-wise with the output of the softmask branch and then added element-wise to the output of the trunk branch as the input of the second residual module. The second residual module processes the input and outputs it. The trunk branch is used to process the original image features, and the softmask branch is used to construct the identity mapping.

[0052] Specifically, the main branch includes two residual modules connected front and back. Figure 6As shown, the soft mask branch includes an autoencoder consisting of an encoding module and a decoding module. The output of the encoding module is processed through a residual module and bilinear interpolation upsampling, and then element-by-element summation is used as the input of the decoding module. The encoding module and decoding module can adopt a general structure and are not limited here. In addition, this embodiment also preferably connects two convolutional layers and a sigmoid layer to the output of the encoding module to normalize the output.

[0053] Furthermore, the loss function used in model training of this solution is preferably the DiceFocal loss function, which is a combination of the Dice Loss and Focal Loss loss functions. The formula of the DiceFocal loss function is as follows:

[0054] L=λ1L Dice +λ2L Focal (1)

[0055]

[0056]

[0057] In formula (1), λ1 and λ2 are the L Dice and L Focal In this experiment, we set λ1 = 1 and λ2 = 1. In formulas (2) and (3), b is the sample index of the batch, c is the category index of the sample, i is the row index of the pixel in the sample image, j is the column index of the pixel, and k is the height index of the pixel. γ is the modulating factor of FocalLoss, which is used to reduce the weight of easy-to-classify samples. Setting γ to 0 is the standard cross-entropy loss. Because the category imbalance problem of liver vascular task data is more prominent, setting γ to 2 can reduce the weight of easy-to-classify (background) pixels to a certain extent.

[0058] To illustrate the effectiveness of this solution, this example uses CT data from approximately 200 patients provided by the Medical Image Decathlon dataset for comparative experiments. The experimental subjects are the 3D U-Net + Improved 3D U-Net, 3D U-Net + 3D U-Net, and Level 1 3D U-Net of this solution. Models I to VIII represent models with different training levels / conditions. The experimental results are shown in the following table:

[0059] Model 3D-UNet 3D-UNet+3D-UNet 3D-UNet+Improved 3D-UNet Ⅰ 0.7411 0.7840 0.7966 Ⅱ 0.7783 0.8230 0.8239 Ⅲ 0.6715 0.7396 0.7554 Ⅳ 0.7710 0.8213 0.8212 Ⅴ 0.7411 0.7840 0.7966 Ⅵ 0.7332 0.7582 0.8364 Ⅶ 0.8231 0.8218 0.8154 Ⅷ 0.7816 0.7972 0.8153 Mean 0.7549 0.7867 0.8176

[0060] Experimental results show that the algorithm designed in this paper can achieve an approximately 16% improvement in the Dice coefficient for liver segmentation. Experiments show that this solution uses a cascaded network combining a modified residual module and a hybrid residual attention mechanism module to improve the recognition of liver Couinaud segmented regions across multiple scan slices, achieving superior segmentation recognition capabilities compared to conventional recognition models.

[0061] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Persons skilled in the art may make various modifications, additions, or substitutions to the described specific embodiments without departing from the spirit of the present invention or exceeding the scope of the appended claims.

Claims

1. A method for intelligent segmentation recognition of liver CT images, characterized in that: The following steps are involved: S1. Using the liver segmentation model, the liver parenchyma region of the abdominal CT image is segmented to obtain a liver mask. S2. multiplying the liver mask with the original input abdominal CT image to obtain a liver positioning image; S3. Inputting the liver localization image into the liver segmentation model, and performing Couinaud segmentation on the liver region by the liver segmentation model; The liver segmentation model is a trained adapted 3D-UNet network; The adapted 3D-UNet network includes a downsampling residual network composed of multiple layers of residual modules and an upsampling residual network composed of multiple layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module; The upsampling residual network and downsampling residual network of the adapted 3D-UNet network each include a four-layer residual module; The output of the first-layer residual module of the downsampling residual network is processed by an attention residual module and then summed with the output of the third-layer residual network of the upsampling residual network and input into the fourth-layer residual module; The output of the second-layer residual module of the downsampling residual network is processed by an attention residual module, and then summed with the output of the second-layer residual module of the upsampling residual network and input into the third-layer residual module; The output of the third-layer residual module of the downsampling residual network is processed by an attention residual module and then summed with the output of the first-layer residual module of the upsampling residual network before being input into the second-layer residual module.

2. The method for intelligent segmentation recognition of liver CT images according to claim 1, characterized in that: The liver segmentation model described in step S1 is a trained 3D-UNet network, including a decoder and an encoder, the encoder includes a downsampling module composed of multiple layers of encoding modules, and the decoder includes an upsampling module composed of multiple layers of decoding modules.

3. The method for intelligent segmentation recognition of liver CT images according to claim 1, characterized in that: Each residual module consists of two consecutive convolution modules that use normalization function and ReLU function. The second convolution module takes the output of the first convolution module as input. The output of the second convolution module is added to the original input of this residual module as the output of this residual module.

4. The method for intelligent segmentation recognition of liver CT images according to claim 1, characterized in that: The attention residual module includes two residual modules and a trunk branch and a soft mask branch located between the two residual modules. The input of the attention residual module is processed by the first residual module and then input into the trunk branch and the soft mask branch respectively. The output of the trunk branch and the output of the soft mask branch are multiplied element by element and then added element by element to the output of the trunk branch as the input of the second residual module.

5. The method for intelligent segmentation recognition of liver CT images according to claim 4, characterized in that: The trunk branch includes two residual modules connected front and back; The soft mask branch includes an automatic encoder composed of an encoding module and a decoding module, and the output of the encoding module is respectively processed by a residual module and bilinear interpolation upsampling and then element-by-element summed as the input of the decoding module.

6. An intelligent recognition system for liver CT image segmentation, characterized in that: The invention comprises a cascaded liver segmentation model and a liver segmentation model, wherein the liver segmentation model comprises a 3D-UNet network for segmenting the liver parenchyma region of abdominal CT images, and the liver segmentation model is a trained adapted 3D-UNet network, comprising a residual network composed of several residual modules and an attention mechanism module mixed with residual modules, for performing Couinaud segmentation on the liver region segmented by the liver segmentation model; The residual network includes a downsampling residual network composed of multiple layers of residual modules and an upsampling residual network composed of multiple layers of residual modules, and the residual module of the upsampling residual network is jump-connected to the residual module of the downsampling residual network through an attention residual module; The upsampling residual network and the downsampling residual network each include a four-layer residual module; The output of the first-layer residual module of the downsampling residual network is processed by an attention residual module and then summed with the output of the third-layer residual network of the upsampling residual network and input into the fourth-layer residual module; The output of the second-layer residual module of the downsampling residual network is processed by an attention residual module, and then summed with the output of the second-layer residual module of the upsampling residual network and input into the third-layer residual module; The output of the third-layer residual module of the downsampling residual network is processed by an attention residual module, and then summed with the output of the first-layer residual module of the upsampling residual network and input into the second-layer residual module; The residual module includes two consecutive convolution modules that use a normalization function and a ReLU function. The second convolution module takes the output of the first convolution module as input, and the output of the second convolution module is added to the original input of the residual module as the output of the residual module. The attention residual module includes two residual modules and a trunk branch and a soft mask branch located between the two residual modules. The input of the attention residual module is processed by the first residual module and then input into the trunk branch and the soft mask branch respectively. The output of the trunk branch and the output of the soft mask branch are multiplied and added to the output of the trunk branch as the input of the second residual module.

Images