A deep learning-based automatic liver tumor CT image segmentation method

Abstract

The present application relates to the technical field of digital medical treatment, in particular to a liver tumor CT image automatic segmentation method based on deep learning, comprising the following steps: extracting a CT image to be segmented; using a multi-scale feature recognition network to extract multi-scale features of the CT image to be segmented; using an attention mechanism network to identify a lesion area in the CT image to be segmented through the multi-scale features; and segmenting a liver tumor image in the CT image to be segmented according to the identification result. The present application solves the problems of manual labeling, low segmentation accuracy and efficiency of traditional methods based on deep learning technology, and has higher automation, more accurate segmentation results and higher segmentation efficiency compared with the prior art.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of digital medical technology, and in particular to a liver tumor CT image automatic segmentation method based on deep learning. BACKGROUND

[0002] Liver cancer is the most common and highest mortality tumor in the world, which seriously threatens people's life and health. With the progress of science and technology, computer application technology and medical informatization level have been greatly improved, and medical image facilities have also been more perfect.

[0003] Computed Tomography (CT) has the characteristics of fast scanning time, high image resolution, and is the current common diagnosis method for liver lesions. At present, the segmentation of liver tumors in clinical practice is usually manually annotated by experienced doctors, which not only consumes time and effort, but also may cause different results in the annotation of the same patient's liver CT affected tumor area by different doctors, and the accuracy of the annotation results is seriously dependent on the experience and skills of doctors.

[0004] Traditional medical image segmentation methods include threshold, level set, region growing, etc. Due to the characteristics of low contrast, blurred boundary, and size, shape, location, and number of liver tumors in CT images are not fixed, traditional segmentation methods require manual intervention, and it is difficult to effectively adapt to the complexity and diversity of liver tumors, the segmentation accuracy of the target is poor, the efficiency is low, and automatic segmentation of the tumor area cannot be realized.

[0005] In recent years, deep learning technology has developed rapidly and has been widely applied in the field of medical image segmentation. Ronneberger et al. proposed a U-shaped convolutional network, which first introduced a skip connection into the convolutional network, and the network realized end-to-end semantic segmentation of images through encoding-decoding operation: the encoder down-samples the extracted features to capture the context information of the image, and the decoder up-samples the extracted features to accurately locate the segmentation area. Li et al. proposed a Bottleneck Supervised Unet (BS-net) model, which is a hybrid tight connection structure that can segment by fully utilizing the information between each layer of the network. Schlemper et al. incorporated attention mechanism into the Unet network and proposed an A-Unet (Attention Unet) model. This model can automatically learn the region features related to the segmentation task and suppress irrelevant features.

[0006] Although these networks are widely used in liver tumor segmentation field, there are still some problems such as inaccurate boundary segmentation, difficulty in detecting small tumors, and difficulty in tumor segmentation caused by unbalanced data categories. At the same time, the locality of the receptive field in the convolution layer of the existing network still limits its learning ability to a relatively small area, which will seriously affect the segmentation performance. SUMMARY

[0007] The purpose of the present application is to provide a deep learning-based automatic liver tumor CT image segmentation method, which aims to solve the technical problems of inaccurate boundary segmentation and difficulty in detecting small tumors in the deep learning-based liver tumor segmentation method, and to realize more accurate positioning and identification of the lesion area.

[0008] To achieve the above purpose, the present application provides a deep learning-based automatic liver tumor CT image segmentation method, comprising the following steps:

[0009] extracting the CT image to be segmented;

[0010] extracting the multi-scale features of the CT image to be segmented using a multi-scale feature recognition network;

[0011] using an attention mechanism network to identify the lesion area in the CT image to be segmented through the multi-scale features;

[0012] According to the identification result, the liver tumor image is segmented in the CT image to be segmented.

[0013] Optionally, the CT image to be segmented is a CT image containing only the liver region.

[0014] Optionally, the process of extracting the CT image to be segmented comprises the following steps:

[0015] obtaining the original CT data set;

[0016] adjusting the window width and window level of any CT image in the original CT data set, and labeling the liver label in the CT image;

[0017] According to the marking result, the CT image without liver label in the original CT data set is removed to obtain the target CT data set;

[0018] pre-segmenting any CT image in the target CT data set to obtain the CT image data set to be segmented.

[0019] Optionally, the multi-scale feature recognition network comprises two or more convolution modules, and adjacent two convolution modules are connected through a pooling module.

[0020] Optionally, each convolution module comprises three convolution layers, a first adder and a normalization layer;

[0021] the dilation rate of any convolutional layer is greater than or equal to 1;

[0022] the input of any convolutional layer is connected to a pooling module or an import layer;

[0023] the three convolutional layers are parallel, and the outputs of the three convolutional layers are connected to the first adder;

[0024] the first adder is connected to the normalization layer through an attention mechanism channel;

[0025] the normalization layer is connected to another pooling module.

[0026] Optionally, each of the pooling modules comprises a standard convolutional layer, a pooling layer, and a subtracter, the input of the standard convolutional layer and the pooling layer is connected to the same convolutional module, the input of the standard convolutional layer and the pooling layer is connected to the subtracter, and the subtracter is connected to another convolutional module.

[0027] Optionally, the number of input channels and the number of output channels of the pooling module satisfy the following rules: wherein C out represents the number of output channels of the pooling module, C in represents the number of input channels of the pooling module, and ΔC represents the number of compensation channels of the pooling module.

[0028] Optionally, the attention mechanism network comprises two or more up-sampling modules, and adjacent two up-sampling modules are connected through a residual module, and the number of the up-sampling modules is the same as the number of the residual modules.

[0029] Optionally, the residual module comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, a CA attention mechanism layer, and a second adder;

[0030] the input of the first convolutional layer and the input of the third convolutional layer are respectively connected to an up-sampling module;

[0031] the second convolutional layer is connected in series with the first convolutional layer, and the CA attention mechanism layer is connected in series with the third convolutional layer;

[0032] the input of the third convolutional layer and the input of the CA attention mechanism layer are both connected to the second adder;

[0033] the second adder is connected to another up-sampling module or an output network.

[0034] The application provides a liver tumor CT image automatic segmentation method based on deep learning, comprising the following steps: extracting a CT image to be segmented; using a multi-scale feature recognition network to extract multi-scale features of the CT image to be segmented; using an attention mechanism network to identify a lesion area in the CT image to be segmented through the multi-scale features; and segmenting a liver tumor image in the CT image to be segmented according to the identification result. The application solves the problems of manual labeling, low segmentation accuracy and efficiency of traditional methods based on deep learning technology, and has higher automation, more accurate segmentation results and higher segmentation efficiency compared with the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort on the basis of these drawings.

[0036] Figure 1 is a flowchart of a liver tumor CT image automatic segmentation method based on deep learning of the present application.

[0037] Figure 2 is a CT image to be segmented provided by a specific embodiment of the present application.

[0038] Figure 3 is an implementation flowchart of the process of extracting a CT image to be segmented of the present application.

[0039] Figure 4 is a structural diagram of a multi-scale feature recognition network of the present application.

[0040] Figure 5 is a structural diagram of a convolution module of the present application.

[0041] Figure 6 is a structural diagram of a pooling module of the present application.

[0042] Figure 7 is a structural diagram of an attention mechanism network of the present application.

[0043] Figure 8 is a structural diagram of a residual module of the present application.

[0044] Figure 9 is a flowchart of the process of inputting and outputting a CT image in a specific embodiment of the present application. DETAILED DESCRIPTION

[0045] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0046] Please see Figure 1 This invention provides a method for automatic segmentation of liver tumor CT images based on deep learning, comprising the following steps:

[0047] S1: Extract the CT image to be segmented;

[0048] S2: Utilize a multi-scale feature recognition network to extract multi-scale features from the CT image to be segmented;

[0049] S3: Using an attention mechanism network, identify the lesion region in the CT image to be segmented through the multi-scale features;

[0050] S4: Based on the recognition results, the liver tumor image is segmented from the CT image to be segmented.

[0051] The following provides further explanation with reference to specific embodiments and implementation steps:

[0052] S1. Extract the CT image to be segmented;

[0053] CT images are typically presented as two-dimensional slices, each containing information about a cross-section of the human body.

[0054] The CT images to be segmented are typically generated by medical equipment (such as a CT scanner) and stored digitally in a computer system. Furthermore, the CT images to be segmented can be one or multiple images, and any single CT image may contain images of the liver as well as images of other organs and tissues of the body.

[0055] Figure 2 The CT image to be segmented is provided in an embodiment of the present invention. Figure 2 As shown, in Figure 2 The area selected in the middle box represents the liver. Besides the liver, the CT image to be segmented mainly contains images of other organs and tissues. The liver image occupies a small proportion of the CT image to be segmented; furthermore, liver tumors growing on the liver occupy an even smaller proportion. Therefore, it is difficult to intuitively extract and analyze this information before segmentation.

[0056] S2. Using a multi-scale feature recognition network, extract the multi-scale features of the CT image to be segmented.

[0057] The multi-scale feature recognition network is a convolutional network based on deep learning, which is used to extract multi-scale feature representations from the CT image to be segmented.

[0058] The multi-scale feature recognition network is generally composed of multiple parallel branches, each branch is responsible for processing input images of different scales, and each branch contains convolutional layers, pooling layers and other feature extraction layers to effectively capture image information at different scales.

[0059] Specifically, the multi-scale feature recognition network can be any one of U-Net, FCN (Fully Convolutional Network), DeepLab, or a combination of multiple deep learning networks.

[0060] The multi-scale features are image feature representations with different scales and levels of abstraction extracted from the CT image to be segmented by the multi-scale feature recognition network.

[0061] In this embodiment, the multi-scale features specifically include low-level features and high-level features, wherein the low-level features contain edge, texture and other detailed information, and have good capture ability for small-scale structures; the high-level features contain information such as the overall shape of the organ and the distribution pattern of the tumor, and have better representation ability for larger-scale structures and semantic information.

[0062] Further, the multi-scale features are usually represented by a tensor composed of multiple feature maps of different scales. Each feature map corresponds to a branch or convolutional layer in the multi-scale feature recognition network, which encodes the feature information of the image at different spatial resolutions and channel numbers.

[0063] The embodiment of the application extracts multi-scale features using a multi-scale feature recognition network, overcomes the limitations of single-scale features, and improves the accuracy and robustness of the lesion area such as liver tumors by integrating information of multiple scales.

[0064] S3, using an attention mechanism network, recognizing the lesion area in the CT image to be segmented by the multi-scale features.

[0065] The attention mechanism network is also a deep learning network, which mainly realizes the recognition of target features by enhancing the attention and weight distribution of the network to the target features in the input data.

[0066] An attention mechanism network generally consists of two main components: an encoder and a decoder, where the encoder is responsible for transforming the input data (such as a CT image to be segmented) into a series of feature representations, and the decoder uses these feature representations to generate attention weights or re-weights the input data in order to focus on the target region.

[0067] It should be understood that the attention mechanism network described in the present application only uses its decoder part, i.e., through the decoding performance of the attention mechanism network, the decoding analysis of the multi-scale feature extracted by the multi-scale feature recognition network is realized, and finally the lesion region in the CT image to be segmented is recognized. The function of the encoder part in the embodiment of the present application has been realized in step S2.

[0068] Using the attention mechanism network to weight and process the multi-scale features, by learning the relevance and importance of different regions in the image, attention weights are assigned to target features with higher influence, improving the segmentation effect of the lesion region.

[0069] S4, according to the recognition result, segmenting a liver tumor image in the CT image to be segmented.

[0070] The recognition result obtained in step S3 contains the positioning and boundary information of the liver tumor region. Step S4 can separate and extract the liver tumor region from other tissues and structures according to these recognition results combined with image processing techniques.

[0071] Further, since the liver tumor occupies a small proportion of the CT image to be segmented with poor contrast and blurred boundary, in an optional embodiment, the process of extracting the CT image to be segmented in step S1 of the present application is further set. Please refer to Figure 3 , Figure 3 The flowchart for implementing step S1 in the embodiment of the present application is shown in Figure 3 As shown in the figure, the extraction of the CT image to be segmented in step S1 includes the following steps:

[0072] S11, obtaining an original CT data set.

[0073] When a patient undergoes CT scanning, the original CT data set is usually a three-dimensional data set containing multiple images, where each image represents a tomographic image of the body at different cross sections (usually transverse sections).

[0074] The original CT data set may contain dozens or even hundreds of images, and the specific number depends on the scanning range and layer thickness. Further, in the original CT data set, not every image necessarily contains the liver. The liver may be completely visible in some images, while in other images it may only be partially visible or not visible at all.

[0075] S12, adjusting the window width and window level of any CT image in the original CT data set, and labeling the liver label in the CT image.

[0076] Window width and window level are used to adjust the display method of medical images, commonly used in CT (computed tomography) image visualization, where window width refers to the gray scale range of image display, indicating the width of the gray level visible in the image; window level indicates the position of the central gray value of the image, which determines the brightness level of the image display. By adjusting the window width and window level, the contrast and display range of the image can be changed.

[0077] A wider window width will increase the contrast of the image, making the differences between different tissues and structures more obvious; a narrower window width can highlight the details of a specific area. In this embodiment, the window width and window level of any CT image in the original CT data set are set to 200 Hu and 60 Hu respectively, at which time the liver and other organs can be easily distinguished.

[0078] Further, step S12 can use medical image software such as RadiAnt DICOM Viewer, 3D Slicer, ITK-SNAP to manually label the CT image with liver image; or step S12 can use computer-aided methods to achieve automatic labeling. It can be understood that the specific labeling method depends on the specific circumstances and available resources.

[0079] S13, according to the labeling result, removing the CT images in the original CT data set that do not contain liver labels, obtaining a target CT data set.

[0080] It should be understood that any CT image in the target CT data set obtained in step S13 contains a liver image, and any liver image may or may not contain a liver tumor image.

[0081] S14, pre-cutting any CT image in the target CT data set to obtain a CT image data set to be segmented.

[0082] Although there is a difference in density between liver tumors and normal liver tissue, the density of liver tumors is similar to that of other organs in the abdomen, so it is difficult to obtain ideal results by directly using the CT images in the target CT data set for segmentation.

[0083] Step S14 performs preliminary cutting on the CT images in the target CT data set, so that the cut images only contain CT images of the liver region, effectively avoiding the interference of other organs in the abdomen on liver tumor segmentation.

[0084] Specifically, the pre-cut implementation methods include but are not limited to threshold segmentation method, region growing algorithm, Canny edge detection method, etc. The specific method is selected according to the specific situation and available resources. When selecting and implementing the segmentation method, factors such as image quality, liver shape change, lesion characteristics, etc. need to be considered, and appropriate parameter adjustment and verification are needed.

[0085] In this embodiment, a trained UNet deep learning network model is selected to segment the liver region in the CT image, and the liver region is taken as the CT image to be segmented. The pixels of any CT image to be segmented are 512x512.

[0086] By extracting the target CT image, adjusting the window width and window level, and performing preliminary cutting operation, the interference of non-liver regions can be reduced, so that the liver and its tumors are more clear and accurate in the segmentation process, and the quality of the segmentation result is improved. At the same time, through preprocessing and preliminary cutting, the number of pixels that need to be processed in the segmentation task can be reduced, and the range of the segmentation region can be reduced, so as to improve the speed and efficiency of the segmentation.

[0087] In specific embodiments, to achieve multi-scale feature extraction in the CT image to be segmented, the present application provides a multi-scale feature recognition network, please refer to Figure 4 As shown in Figure 4 , the multi-scale feature recognition network includes two or more convolution modules, and adjacent two convolution modules are connected through a pooling module.

[0088] The multi-scale feature recognition network realizes multi-scale feature extraction and fusion through the combination of multiple convolution modules and pooling modules, so that the network can capture the details, textures and semantic information of the image at the same time. At the same time, the connection of the pooling module promotes the information transmission and integration between different scale features, improves the expression ability and performance of the network. And through multi-level feature representation, the network can better understand the structure and content of the image, and realize accurate image segmentation.

[0089] Specifically, Figure 5 is a structural diagram of the convolution module of the present application. As shown in Figure 4 and Figure 5 , in the multi-scale feature recognition network for strengthening the receptive field of the segmentation network feature map, any convolution module includes three convolution layers, a first adder and a normalization layer; the dilation rate of any convolution layer is greater than or equal to 1; the input end of any convolution layer is connected with a pooling module or an import layer; the three convolution layers are parallel, and the three output ends of the three convolution layers are connected to the first adder; the first adder is connected with the normalization layer through an attention mechanism channel; and the normalization layer is connected with another pooling module.

[0090] In the embodiment, the convolution kernel of each convolution layer in each convolution module is 3x3, the hole rates of the three parallel convolution layers are 1, 2 and 4 respectively, and the receptive fields of the three parallel convolution layers are 3, 5 and 7 respectively. In the embodiment, the multi-scale feature recognition network based on the enhanced segmentation network feature map receptive field is used to realize accurate and efficient multi-scale feature extraction by setting three groups of parallel convolution layers and attention mechanism channel connection normalization layers. This design can capture image information of different scales, enhance the expression ability and diversity of features, and effectively fuse important feature information. At the same time, the connection with the pooling module realizes down-sampling and scale unification.

[0091] Further, Figure 6 The structure diagram of the pooling module of the application is shown in Figure 1. Figure 4 And Figure 6 As shown in the multi-scale feature recognition network based on the enhanced segmentation network feature map receptive field provided in the embodiment, any of the pooling modules comprises a standard convolution layer, a pooling layer and a subtractor; the input ends of the standard convolution layer and the pooling layer are connected to the same convolution module; the input ends of the standard convolution layer and the pooling layer are connected to the subtractor; and the subtractor is connected to another convolution module.

[0092] Through such a setting, the input channel number and the output channel number of any of the pooling modules satisfy the following rules: Wherein, C out represents the output channel number of the pooling module, C in represents the input channel number of the pooling module, and ΔC represents the compensation channel number of the pooling module.

[0093] Further, ΔC = |C in -C out |, that is, assuming that the feature map input into the pooling module has C in channels, and the output feature map has c out channels: if C in <C out , the pooling layer will generate a feature map containing C in channels, and the standard convolution layer will generate a feature map containing the remaining C out -C in channels; otherwise, the pooling layer will be ignored, and only the standard convolution layer will generate a feature map containing C out channels. In the embodiment, the convolution kernel of the standard convolution layer in each pooling module is 3x3, the step is 2, and the hole rate is 2; and the pooling layer window is 2x2.

[0094] The pooling module is usually referred to as a down-sampling module, which can help the multi-scale feature recognition network to capture multi-scale features and reduce the amount of calculation. The pooling module can increase the size of the receptive field by reducing the resolution of the feature map, so that the network can model the context information in a larger range. The multi-scale feature recognition network provided in the embodiment realizes multi-scale feature extraction through a specific pooling module structure, reduces the amount of calculation, and improves the receptive field. At the same time, the module simplifies the network connectivity, reduces the number of parameters and the computational complexity.

[0095] In view of the problem that small tumor boundary segmentation is not accurate and difficult to detect, a multi-scale feature recognition network capable of accurately identifying small tumor features is proposed, please refer to Figure 4 、 Figure 5 and Figure 6 The multi-scale feature recognition network provided in the embodiment includes two or more convolution modules, and adjacent two convolution modules are connected through a pooling module; any convolution module uses the convolution module structure proposed in the above embodiment; any pooling module uses the pooling module structure proposed in the above embodiment.

[0096] In the embodiment, the multi-scale feature recognition network provided includes 5 convolution modules and 4 pooling modules. Specifically, the 512x512 CT image to be segmented (RGB, 3 channels) stored in the import layer is input, as shown in Figure 4 After passing through the 5 convolution modules, the channel number of the feature map is 64, 128, 256, 512, and 1024, respectively; after passing through the 4 convolution modules, the channel number of the feature map is 128, 256, 512, and 1024, respectively.

[0097] To better utilize multi-scale features to realize the recognition of dry tumors, in the decoder part, the application provides an attention mechanism network, please refer to Figure 7 and Figure 8 , Figure 7 is a structural diagram of the attention mechanism network provided in the embodiment of the application, Figure 8 is a structural diagram of the residual module provided in the embodiment of the application. As shown in Figure 7 , it includes two or more up-sampling modules, adjacent two up-sampling modules are connected through a residual module, and the number of the up-sampling modules is the same as that of the residual module.

[0098] In the embodiment, any of the residual modules comprises a first convolutional layer, a second convolutional layer, a third convolutional layer, a CA attention mechanism layer, and a second adder; the input ends of the first convolutional layer and the third convolutional layer are connected with the up-sampling module respectively; the second convolutional layer is connected in series with the first convolutional layer, and the CA attention mechanism layer is connected in series with the third convolutional layer; the input ends of the third convolutional layer and the CA attention mechanism layer are connected with the second adder; and the second adder is connected with the up-sampling module or the output network. The CA attention mechanism is an attention mechanism considering the channel dimension and the spatial dimension at the same time, and can learn adaptive channel weights, so that the model pays more attention to useful channel information. Specifically, the attention mechanism is a method for improving the neural network in recent years, which has achieved good results in the field of image segmentation. The role of the attention mechanism is to make the model pay attention to more useful semantic information and ignore useless information, so as to obtain more global context information, greatly enriching the representation ability of the neural network. The channel attention mechanism GCT_B0 and the coordinate attention mechanism (Coordinate Attention) are introduced to improve the segmentation performance of the network on medical images.

[0099] In the embodiment, the convolution kernel of the first convolutional layer and the second convolutional layer is 3*3, and the hole rate is 1; the convolution kernel of the third convolutional layer is 1*1, and the hole rate is 1. For the feature map input into the residual module provided in the embodiment, 3*3 ordinary convolution operation is performed twice on the convolutional layer in series, 1*1 convolution operation is performed on the coordinate attention layer first, the generated feature map is subjected to CA attention mechanism, and finally the feature maps of the two modules are added to form a new feature map. The residual module accelerates the convergence of the training network, reduces the model degradation, and thus effectively avoids the gradient vanishing problem.

[0100] The specific execution step diagram is shown in Figure 9 The CT image to be segmented is input into the multi-scale feature recognition network comprising the convolution module (SPCB) and the pooling module (SPPB) for feature extraction, and then into the attention mechanism network comprising the up-sampling module (UP sample) and the residual module (RA-Block). According to the recognition and positioning results of the multi-scale feature recognition network and the attention mechanism network, the liver tumor image is cut out from the CT image to be segmented.

[0101] Further, the advantages of the present application are described as follows in combination with the technical features:

[0102] 1. U-shaped structure: The main feature of SPA-UNet is its unique U-shaped structure, which consists of symmetrical downsampling (encoder) and upsampling (decoder) paths. The downsampling path gradually reduces the size of the input image through convolution and pooling operations, extracting features at different levels. The upsampling path combines the features from the encoder and decoder through deconvolution and skip connections to restore resolution and detail information. This U-shaped structure effectively captures features at different scales, enabling accurate localization and segmentation of target regions in medical image segmentation.

[0103] 2. Skip Connections: SPA-UNet uses skip connections to link features at different levels between the encoder and decoder. These skip connections help information propagate quickly from lower levels to the decoder, solving the information loss problem in traditional convolutional neural networks and preserving more detail and contextual information. In medical image segmentation, this plays a crucial role in accurately locating and segmenting important structures such as tumors and organs.

[0104] 3. Data Augmentation and Regularization: Medical image data typically has a small sample size and significant imbalance. SPA-UNet employs a series of data augmentation techniques during training, such as random cropping, mirroring, and rotation, to expand the diversity of training samples and alleviate the problem of insufficient data. Furthermore, SPA-UNet uses common regularization techniques, such as Dropout and Batch Normalization, which help improve the model's generalization ability and stability.

[0105] 4. Multi-scale feature fusion: SPA-UNet combines and integrates features from different resolutions by using feature fusion strategies at different levels. This multi-scale feature fusion enables the model to accurately detect and segment the boundaries of target objects, and is particularly suitable for problems with complex structures and large scale variations in medical images.

[0106] The deep learning network proposed in this embodiment of the invention has a loss function that satisfies the following model: Where Loss represents the loss value, α represents the adjustable hyperparameter, N represents the total number of samples, and y i p is the category to which the i-th sample belongs. i This indicates that the i-th sample belongs to category y. i The loss function, by comparing the predicted probability with the true label and calculating crossover destruction, enables the network to learn to predict the type of sample more accurately. Simultaneously, this loss function also improves the stability and robustness of the learning network by calculating the difference between the predicted rate and the combined persistence and marginal persistence of the true label.

[0107] Further, in order to prove the segmentation performance of the proposed model, the SPA-UNet is compared with UNet, Topformer, Segformer_B0, BiSeNetv2, OCRNet, Unet++, Unet_attention, Espnet and other advanced models. The performance of three evaluation indexes of different networks in liver segmentation is shown in Table 1.

[0108] Table 1. Performance comparison of quantitative indexes of different networks in liver segmentation

[0109] Model Iou Precision Recall UNet 0.943 0.961 0.966 Topformer 0.903 0.943 0.956 Segformer_B0 0.943 0.969 0.973 BiSeNetV2 0.932 0.963 0.967 OCRNet 0.874 0.921 0.944 UNet++ 0.942 0.966 0.971 UNet_attention 0.952 0.971 0.977 ESPNet 0.925 0.961 0.961 Ours (the invention) 0.953 0.974 0.978

[0110] As can be seen from Table 1, compared with the original UNet model, the proposed model has better performance, and its Iou, Precision and Recall indexes are improved by 1.0%, 1.3% and 1.2% respectively compared with the original UNet, and are improved by 7.9%, 5.3% and 3.4% respectively compared with OCRNet.

[0111] The performance of three evaluation indexes of different networks in liver tumor segmentation is shown in Table 2.

[0112] Table 2. Performance comparison of quantitative indexes of different networks in liver tumor segmentation

[0113] Model Iou Precision Recall UNet 0.806 0.885 0.889 Topformer 0.662 0.805 0.787 Segformer_B0 0.792 0.884 0.883 BiSeNetV2 0.751 0.875 0.841 OCRNet 0.732 0.859 0.832 UNet++ 0.796 0.883 0.890 UNet_attention 0.821 0.908 0.896 ESPNet 0.774 0.874 0.820 Ours (the invention) 0.826 0.902 0.907

[0114] As can be seen from Table 2, the proposed model improves the Iou, Precision and Recall indexes by 2.0%, 1.7% and 1.8% respectively compared with the original UNet, and improves the indexes by 14.4%, 8.0% and 10.2% respectively compared with Topformerf. The overall performance of the proposed model in liver tumor segmentation is better than that of other networks, the segmentation of tumor is more accurate, and the proposed model has important reference value for diagnosis of liver cancer.

[0115] In summary, the cavity convolution used in the present application can expand the receptive field of the network, so as to obtain more rich local and global context information and improve the segmentation accuracy of the network. Further, the present application fuses multi-scale information extraction features by designing two multi-branch modules of SPCB and SPPB, fuses multiple feature maps to learn multi-scale features, enhances the perception of different scales and different semantic information of the convolutional neural network, and introduces the attention mechanism to improve the feature learning ability of the network for edges and complete tumors.

[0116] The above disclosed is only a preferred embodiment of the present application, of course, cannot be limited by this to limit the scope of the present application, the person skilled in the art can understand that the implementation of all or part of the above processes, and according to the equivalent changes of the claims of the present application, still belongs to the scope covered by the present application.

Claims

1. A deep learning-based automatic liver tumor CT image segmentation method, characterized in that, The method comprises the following steps: extracting a CT image to be segmented; extracting multi-scale features of the CT image to be segmented by using a multi-scale feature recognition network; the multi-scale feature recognition network comprises two or more convolution modules, and adjacent two convolution modules are connected through a pooling module; each convolution module comprises three convolution layers, a first adder and a normalization layer; the dilation rate of any convolution layer is greater than or equal to 1; the input end of any convolution layer is connected with a pooling module or an import layer; the three convolution layers are in parallel, and the three output ends of the three convolution layers are connected to the first adder; the first adder is connected with the normalization layer through an attention mechanism channel; the normalization layer is connected with another pooling module; each pooling module comprises a standard convolution layer, a pooling layer and a subtractor, the input ends of the standard convolution layer and the pooling layer are connected with the same convolution module, the output ends of the standard convolution layer and the pooling layer are connected with the subtractor, and the subtractor is connected with another convolution module; The input channel number and the output channel number of the pooling module satisfy the following rules: Wherein, C out represents the output channel number of the pooling module, C in represents the input channel number of the pooling module, and ΔC represents the compensation channel number of the pooling module. identifying a lesion area in the CT image to be segmented by using an attention mechanism network based on the multi-scale features; segmenting a liver tumor image in the CT image to be segmented according to the identification result.

2. The deep learning-based automatic segmentation method of the liver tumor CT image according to claim 1, wherein the CT image to be segmented is a CT image containing only a liver region.

3. The deep learning-based automatic segmentation method of the liver tumor CT image according to claim 2, wherein the process of extracting the CT image to be segmented comprises the following steps: obtaining an original CT data set; adjusting the window width and window level of any CT image in the original CT data set, and marking a liver label in the CT image; according to the marking result, removing the CT images without the liver label in the original CT data set to obtain a target CT data set; pre-segmenting any CT image in the target CT data set to obtain a CT image data set to be segmented.

4. The deep learning-based automatic segmentation method of the liver tumor CT image according to claim 3, wherein the attention mechanism network comprises two or more up-sampling modules, adjacent two up-sampling modules are connected through a residual module, and the number of the up-sampling modules is the same as that of the residual modules.

5. The deep learning-based automatic segmentation method of the liver tumor CT image according to claim 4, wherein the residual module comprises a first convolution layer, a second convolution layer, a third convolution layer, a CA attention mechanism layer and a second adder; the input ends of the first convolution layer and the third convolution layer are respectively connected with an up-sampling module; the second convolution layer is connected in series with the first convolution layer, and the CA attention mechanism layer is connected in series with the third convolution layer; the input ends of the third convolution layer and the CA attention mechanism layer are both connected with the second adder; the second adder is connected with another up-sampling module or an output network. ​ ​ ​ ​

Images