Enhanced CT image anatomical atlas segmentation method and system based on deep learning
Through the neural network-based liver lesion segmentation model and combined with the Couinaud liver segment division method, the problem of inaccurate anatomical structure map segmentation in large areas in CT images was solved, which improved the segmentation effect and accuracy, and reduced the burden on doctors to label.
Patent Information
- Application Number
- CN202410988799.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-19
Smart Images

Figure CN118982664A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image segmentation, and specifically relates to a method and system for enhanced CT image anatomical atlas segmentation based on deep learning. Background Art
[0002] In the current medical field, the acquisition of anatomical structure atlases still mainly relies on manual outlining by doctors. Although this method is accurate, it is undoubtedly a time-consuming and laborious task, especially in clinical environments that require large-scale applications, its limitations become more and more obvious. In addition, since there may be certain differences in the subjective judgment of each doctor, this also leads to inconsistencies and deviations in the results of the atlas outlining. Therefore, exploring a more efficient and accurate method to generate anatomical structure atlases has become an urgent need.
[0003] In recent years, the application of deep learning in the field of image processing has made significant progress, providing new possibilities for the automatic prediction and generation of anatomical structure maps. By training a large amount of image data, the deep learning model can learn the characteristics of the anatomical structure, and then automatically identify and outline the key parts in the image. This method not only greatly improves the efficiency of obtaining the atlas, but also helps to reduce the interference of human factors and improve the accuracy of the results. However, when there are large areas of lesions in the image, the prediction effect of existing deep learning methods may be limited to a certain extent. The lesion area may cause distortion or loss of image features, making it difficult for the model to accurately identify and outline the anatomical structure. Summary of the invention
[0004] The present invention provides a method and system for enhanced CT image anatomical atlas segmentation based on deep learning to solve the problem of inaccurate anatomical structure atlas segmentation when there are large-area lesions in the image.
[0005] In a first aspect, the present invention provides a method for segmenting an anatomical atlas of enhanced CT images based on deep learning, the method comprising the following steps:
[0006] Constructing a liver lesion segmentation model based on a neural network, wherein the liver lesion segmentation model includes a region of interest clipping module, a target positioning module, a precise segmentation network module, and a liver atlas prediction network module;
[0007] Acquire the CT image to be segmented and multiple abdominal enhanced CT images;
[0008] Based on the Couinaud liver segmentation method, multi-category liver anatomical masks are annotated for all the abdominal enhanced CT images, and the region of interest cropping module is used to divide all the abdominal enhanced CT images into training set images and test set images;
[0009] Preprocessing all the training set images and the test set images;
[0010] Combining the preprocessed training set images and the test set images, respectively completing the model training of the target positioning module, the precise segmentation network module, and the liver atlas prediction network module;
[0011] Preprocessing the CT image to be segmented into three-dimensional image data;
[0012] Inputting the three-dimensional image data into the liver lesion segmentation model, and generating a positioning binary mask through the target positioning module;
[0013] Based on the positioning binary mask, a cropping process is performed through the region of interest cropping module to obtain cropped three-dimensional data;
[0014] Performing prediction processing on the cropped three-dimensional data through the precise segmentation network module to obtain a segmentation binary mask;
[0015] The segmentation binary mask is processed by using the liver atlas prediction network module to obtain a liver anatomical atlas prediction mask, and the anatomical atlas segmentation of the CT image to be segmented is completed according to the liver anatomical atlas prediction mask.
[0016] Optionally, the step of labeling multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method, and dividing all the abdominal enhanced CT images into training set images and test set images using the region of interest cropping module comprises the following steps:
[0017] Annotating multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method;
[0018] Cropping the abdominal enhanced CT image into a cropped CT image according to a preset first cropping number and using the region of interest cropping module;
[0019] According to a preset second cropping quantity and using the region of interest cropping module, the multi-category liver anatomical mask is cropped into a cropped multi-category mask;
[0020] Binarizing the remaining liver anatomical mask to obtain a binary mask;
[0021] According to a preset third cropping quantity and using the region of interest cropping module, the binary mask is cropped into a cropped binary mask;
[0022] Combining the remaining abdominal enhanced CT images and the binary mask to form a first data set, and dividing the first data set into a first training set image and a first test set image according to a preset division ratio;
[0023] Combining the cropped CT image and the cropped binary mask to form a second data set, and dividing the second data set into a second training set image and a second test set image according to the division ratio;
[0024] The cropped binary mask and the cropped multi-category mask are combined to form a third data set, and the third data set is divided into a third training set image and a third test set image according to the division ratio.
[0025] Optionally, the combining of the preprocessed training set images and the test set images to respectively complete the model training of the target positioning module, the precise segmentation network module and the liver atlas prediction network module comprises the following steps:
[0026] Training the target positioning module by combining the preprocessed first training set images and the first test set images, and calculating the first training computing power required for the training process;
[0027] Training the precise segmentation network module in combination with the preprocessed second training set images and the second test set images, and calculating the second training computing power required for the training process;
[0028] Training the liver atlas prediction network module in combination with the preprocessed third training set images and the third test set images, and calculating the third training computing power required for the training process;
[0029] If the sum of the computing power of the first training computing power, the second training computing power and the third training computing power is less than a preset computing power consumption threshold, the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module are completed in parallel.
[0030] Optionally, the preprocessing of the CT image to be segmented into three-dimensional image data comprises the following steps:
[0031] Redirecting the CT image to be segmented so that the image coordinate axis of the CT image to be segmented is aligned to a preset standard direction;
[0032] Resampling the CT image to be segmented to adjust the voxel size of the CT image to be segmented;
[0033] The image grayscale value of the CT image to be segmented is normalized to obtain pre-processed image three-dimensional data.
[0034] Optionally, the step of performing cropping processing based on the positioning binary mask and using the region of interest cropping module to obtain cropped three-dimensional data comprises the following steps:
[0035] Extracting non-zero valued coordinate indices from the positioning binary mask and determining initial coordinates;
[0036] Adjusting the initial coordinates based on a preset offset to obtain a clipping target coordinate;
[0037] A cropping bounding box is generated according to the cropping target coordinates, and the target area included in the cropping bounding box is cropped into cropped three-dimensional data.
[0038] In a second aspect, the present invention further provides an enhanced CT image anatomical atlas segmentation system based on deep learning, the system comprising:
[0039] A model building subsystem, used to build a liver lesion segmentation model based on a neural network, wherein the liver lesion segmentation model includes a region of interest clipping module, a target positioning module, a precise segmentation network module, and a liver atlas prediction network module;
[0040] An image acquisition subsystem, used for acquiring CT images to be segmented and multiple abdominal enhanced CT images;
[0041] A data set construction subsystem, for annotating multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method, and for dividing all the abdominal enhanced CT images into training set images and test set images using the region of interest cropping module;
[0042] A first preprocessing subsystem, used for preprocessing all the training set images and the test set images;
[0043] A model training subsystem, used to complete the model training of the target positioning module, the precise segmentation network module and the liver atlas prediction network module respectively by combining the preprocessed training set images and the test set images;
[0044] A second preprocessing subsystem, used for preprocessing the CT image to be segmented into three-dimensional image data;
[0045] A target positioning subsystem, used for inputting the three-dimensional image data into the liver lesion segmentation model, and generating a positioning binary mask through the target positioning module;
[0046] A clipping subsystem, configured to perform clipping processing based on the positioning binary mask and through the region of interest clipping module to obtain clipped three-dimensional data;
[0047] A segmentation prediction subsystem, used for performing prediction processing on the cropped three-dimensional data through the precise segmentation network module to obtain a segmentation binary mask;
[0048] The image segmentation subsystem is used to process the segmentation binary mask using the liver atlas prediction network module to obtain a liver anatomical atlas prediction mask, and complete the anatomical atlas segmentation of the CT image to be segmented according to the liver anatomical atlas prediction mask.
[0049] Optionally, the data set construction subsystem includes:
[0050] A labeling unit, used for labeling multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method;
[0051] A first cropping unit, configured to crop the abdominal enhanced CT image into a cropped CT image according to a preset first cropping quantity and using the region of interest cropping module;
[0052] A second cropping unit, configured to crop the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping quantity and using the region of interest cropping module;
[0053] A binarization unit, configured to crop the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping quantity and using the region of interest cropping module;
[0054] A third cropping unit, configured to crop the binary mask into a cropped binary mask according to a preset third cropping quantity and using the region of interest cropping module;
[0055] A first data set construction unit, used to combine the remaining abdominal enhanced CT images and the binary mask to form a first data set, and divide the first data set into a first training set image and a first test set image according to a preset division ratio;
[0056] A second data set construction unit, configured to combine the cropped CT image and the cropped binary mask to form a second data set, and divide the second data set into a second training set image and a second test set image according to the division ratio;
[0057] The third data set construction unit is used to combine the cropped binary mask and the cropped multi-category mask to form a third data set, and divide the third data set into a third training set image and a third test set image according to the division ratio.
[0058] Optionally, the model training subsystem includes:
[0059] A first module training unit, used to train the target positioning module in combination with the preprocessed first training set images and the first test set images, and to count the first training computing power required for the training process;
[0060] A second module training unit, used to train the precise segmentation network module in combination with the preprocessed second training set images and the second test set images, and to count the second training computing power required for the training process;
[0061] A third module training unit, used to train the liver atlas prediction network module in combination with the preprocessed third training set images and the third test set images, and to count the third training computing power required for the training process;
[0062] A parallel training unit is used to control the first module training unit, the second module training unit and the third module training unit to complete the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module in parallel when the sum of the computing power of the first training computing power, the second training computing power and the third training computing power is less than a preset computing power consumption threshold.
[0063] Optionally, the second preprocessing subsystem includes:
[0064] a redirection unit, configured to perform a redirection operation on the CT image to be segmented so that the image coordinate axis of the CT image to be segmented is aligned to a preset standard direction;
[0065] A resampling unit, used for performing a resampling process on the CT image to be segmented so as to adjust the voxel size of the CT image to be segmented;
[0066] The normalization unit is used to normalize the image grayscale value of the CT image to be segmented to obtain pre-processed three-dimensional image data.
[0067] Optionally, the cutting subsystem includes:
[0068] A coordinate confirmation unit, used to extract non-zero value coordinate indexes from the positioning binary mask and determine initial coordinates;
[0069] A coordinate adjustment unit, used to adjust the initial coordinates based on a preset offset to obtain a clipping target coordinate;
[0070] The area clipping unit is used to generate a clipping boundary box according to the clipping target coordinates, and clip the target area included in the clipping boundary box into clipping three-dimensional data.
[0071] The beneficial effects of the present invention are:
[0072] The deep learning-based enhanced CT image anatomical atlas segmentation method adopted in the present invention can effectively alleviate the problem that the traditional image segmentation method has poor effect in the presence of large-area lesion tissue. And by establishing a liver lesion segmentation model, the demand for large-scale accurately labeled samples is greatly reduced, and the workload of atlas annotation for clinicians is reduced. At the same time, in the case of lesion tissue occupation, the patient's anatomical structure information is accurately obtained to meet the clinician's demand for accurate judgment of the anatomical condition of the patient's lesion site. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1 This is a flow chart of an enhanced CT image anatomical atlas segmentation method based on deep learning in one of the embodiments of the present application. DETAILED DESCRIPTION
[0074] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0075] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0076] Figure 1 FIG. 1 is a flow chart of an enhanced CT image anatomical atlas segmentation method based on deep learning in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 1 As shown, the method for segmenting an enhanced CT image anatomical atlas based on deep learning disclosed in the present invention specifically includes the following steps:
[0077] S101. Construct a liver lesion segmentation model based on neural network.
[0078] Among them, the liver lesion segmentation model includes a region of interest cropping module, a target positioning module, a precise segmentation network module and a liver atlas prediction network module. Region of interest cropping module: The purpose of this module is to cut the input abdominal enhanced CT image into regions of interest, namely the liver region. This module can be implemented through threshold segmentation, edge detection or machine learning-based methods. A common method is to use threshold segmentation. By setting a threshold, the pixels in the image that are greater than the threshold are set as the foreground, and the pixels that are less than the threshold are set as the background, thereby obtaining the region of interest.
[0079] Target localization module: The purpose of this module is to locate the position of liver lesions in the region of interest. This can be achieved using deep learning-based target detection algorithms, such as Faster R-CNN, YOLO, etc. These algorithms can learn the characteristics of liver lesions by training a neural network, and then perform target detection in the region of interest to obtain the location information of the lesions.
[0080] Precise segmentation network module: The purpose of this module is to accurately segment the liver lesions in the region of interest. This can be achieved using deep learning-based image segmentation algorithms, such as U-Net, SegNet, etc. These algorithms can learn the shape and texture features of liver lesions by training a neural network, and then perform pixel-level segmentation in the region of interest to obtain the segmentation results of the lesions.
[0081] Liver atlas prediction network module: The purpose of this module is to predict the anatomical atlas of the liver, that is, the distribution of different tissue types in the liver. This can be achieved using deep learning-based image segmentation algorithms such as U-Net and SegNet. These algorithms can learn the characteristics of different liver tissue types by training a neural network, and then perform pixel-level classification in the region of interest to obtain the prediction results of the anatomical atlas.
[0082] S102. Obtain a CT image to be segmented and multiple abdominal enhanced CT images.
[0083] A data set containing CT images to be segmented and multiple abdominal enhanced CT images is obtained through a database of a hospital or research institution or other means. These images should contain samples of liver lesions and samples of normal liver for training and testing.
[0084] S103. Based on the Couinaud liver segmentation method, multi-category liver anatomical masks are annotated for all abdominal enhanced CT images, and the region of interest cropping module is used to divide all abdominal enhanced CT images into training set images and test set images.
[0085] Among them, according to the Couinaud liver segmentation method, the liver anatomy of the abdominal enhanced CT image is annotated. The Couinaud liver segmentation method divides the liver into 8 different liver segments, each of which corresponds to different functions and blood supplies. The annotated liver anatomy mask should contain the liver segment information corresponding to each pixel. Then, the region of interest cropping module is used to crop the region of interest from all abdominal enhanced CT images. Specifically, a threshold can be set as needed, and the pixels in the image that are greater than the threshold are set as the foreground, and the pixels that are less than the threshold are set as the background, so as to obtain the region of interest. The cropped image is then divided into training set images and test set images according to a certain ratio.
[0086] S104. Preprocess all training set images and test set images.
[0087] Among them, all training set images and test set images are preprocessed to improve the training and testing effects of the model. The preprocessing steps may include grayscale, normalization, denoising, etc. of the image. Common image processing algorithms such as histogram equalization and Gaussian filtering can be used to achieve this.
[0088] S105. Combine the preprocessed training set images and test set images to complete the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module respectively.
[0089] Among them, the preprocessed training set images and test set images are used to train the models of the target positioning module, the precise segmentation network module, and the liver atlas prediction network module respectively. Common deep learning frameworks such as TensorFlow and PyTorch can be used to implement this. During the training process, the cross entropy loss function can be used to measure the difference between the predicted results and the true labels, and the gradient descent algorithm can be used to optimize the model parameters. Data enhancement techniques such as random rotation, translation, and scaling can also be used to expand the training data set and increase the generalization ability of the model.
[0090] S106. Preprocess the CT image to be segmented into three-dimensional image data.
[0091] The CT image to be segmented is preprocessed and converted into three-dimensional image data. The CT image can be cut into multiple two-dimensional images, and then these two-dimensional images are sequentially combined into a three-dimensional data. Other methods such as voxel interpolation can also be used to achieve this.
[0092] S107. Input the three-dimensional image data into the liver lesion segmentation model, and generate a positioning binary mask through the target positioning module.
[0093] Among them, in this embodiment, the target positioning module of the present invention adopts a lightweight 3DUNet network to achieve fast and accurate liver organ positioning and serve subsequent processing. The network adopts a three-layer 3DUNet infrastructure, including a first encoder and a first decoder part, as follows:
[0094] In this network structure, all 3D convolution kernels have the same size of (3, 3, 3), stride (1, 1, 1) and padding (1, 1, 1), and the only difference is the number of input channels and output channels. The settings for 3D max pooling are all pooling kernel size of 2, stride parameter of 2, padding parameter of 0, and expansion parameter of 1.
[0095] Implement the following steps in the first encoder part:
[0096] Step a: The four-dimensional data (channel dimension, depth, height, width) of input (1, 128, 512, 512) is sequentially subjected to the first three-dimensional convolution (input channel number 1, output channel number 2), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (input channel number 2, output channel number 2), three-dimensional batch normalization, and outputs a feature map of size (2, 128, 512, 512);
[0097] Step b: After three-dimensional maximum pooling processing, output a feature map of size (2, 64, 256, 256);
[0098] Step c: After the first three-dimensional convolution (the number of input channels is 2, the number of output channels is 4), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 4, the number of output channels is 4), three-dimensional batch normalization, the output feature map of size (4, 64, 256, 256) is output;
[0099] Step d: After three-dimensional maximum pooling processing, output feature maps of size (4, 32, 128, 128);
[0100] Step e: After the first three-dimensional convolution (the number of input channels is 4, the number of output channels is 8), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 8, the number of output channels is 8), three-dimensional batch normalization, the output is a feature map of size (8, 32, 128, 128);
[0101] Step f: After three-dimensional maximum pooling processing, output a feature map of size (8, 16, 64, 64);
[0102] Step g: After the first three-dimensional convolution (the number of input channels is 8 and the number of output channels is 16), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (16, 16, 64, 64) is output;
[0103] In the first decoder part the following steps are implemented:
[0104] Step a: Input feature map of size (16, 16, 64, 64), and sequentially pass through 3D transposed convolution (input channel number is 16, output channel number is 16), 3D batch normalization, and LeakyReLU activation function to obtain feature map of size (16, 32, 128, 128), and concatenate it with the feature map output in step e of the encoder along the channel dimension to obtain feature map of size (16+8, 32, 128, 128). Then pass through the first 3D convolution (input channel number is 24, output channel number is 8), 3D batch normalization, LeakyReLU activation function, the second 3D convolution (input channel number is 8, output channel number is 8), 3D batch normalization, and output feature map of size (8, 32, 128, 128);
[0105] Step b: After three-dimensional transposed convolution (the number of input channels is 8, the number of output channels is 8), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (8, 64, 256, 256) is obtained, and it is spliced with the feature map output by step c in the encoder along the channel dimension to obtain a feature map of (8+4, 64, 256, 256). Then after the first three-dimensional convolution (the number of input channels is 12, the number of output channels is 4), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 4, the number of output channels is 4), three-dimensional batch normalization, the output is a feature map of size (4, 64, 256, 256);
[0106] Step c: After three-dimensional transposed convolution (the number of input channels is 4, the number of output channels is 4), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (2, 128, 512, 512) is obtained, and it is spliced with the feature map output in step a of the first encoder along the channel dimension to obtain a feature map of (4+2, 128, 512, 512). After the first three-dimensional convolution (the number of input channels is 6, the number of output channels is 2), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 4, the number of output channels is 4), three-dimensional batch normalization, the output is a feature map of size (2, 128, 512, 512);
[0107] Step d: After three-dimensional convolution (the number of input channels is 2, the number of output channels is 2), three-dimensional batch normalization, and LeakyReLU activation function, the predicted probability map of size (2, 128, 512, 512) is output.
[0108] Step e: Use the Softmax activation function to process the predicted probability map to obtain the predicted location binary mask.
[0109] S108. Perform cropping processing based on the positioning binary mask and through the region of interest cropping module to obtain cropped three-dimensional data.
[0110] Among them, based on the generated positioning binary mask, the image three-dimensional data is cropped using the region of interest cropping module to obtain cropped three-dimensional data. The cropped data only contains the region of interest, that is, the region where the liver lesion is located.
[0111] S109. Predictively process the cropped three-dimensional data through the precise segmentation network module to obtain a segmentation binary mask.
[0112] Among them, after being processed by the region of interest cropping module, the image size obtained is (64, 256, 256), which is 8 times smaller than the original (128, 512, 512) size data, and irrelevant information is deleted, reducing the amount of calculation. In this embodiment, the precise segmentation network module adopts a 5-layer 3DUNet infrastructure, including a second encoder and a second decoder part, as follows:
[0113] In this network structure, all 3D convolutions have a uniform kernel size of (3, 3, 3), stride (1, 1, 1) and padding (1, 1, 1), with the only difference being the number of input channels and the number of output channels. The settings for 3D max pooling are all pooling kernel size of 2, stride parameter of 2, padding parameter of 0, and expansion parameter of 1.
[0114] Implement the following steps in the second encoder part:
[0115] Step a: The four-dimensional data (channel dimension, depth, height, width) of input (1, 64, 256, 256) is sequentially subjected to the first three-dimensional convolution (input channel number 1, output channel number 4), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (input channel number 4, output channel number 4), three-dimensional batch normalization, and outputs a feature map of size (4, 64, 256, 256);
[0116] Step b: After three-dimensional maximum pooling processing, output feature maps of size (4, 32, 128, 128);
[0117] Step c: After the first three-dimensional convolution (the number of input channels is 4 and the number of output channels is 8), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 8 and the number of output channels is 8), three-dimensional batch normalization, the feature map of size (8, 32, 128, 128) is output;
[0118] Step d: After three-dimensional maximum pooling processing, output feature maps of size (8, 16, 64, 64);
[0119] After the first 3D convolution (the number of input channels is 8 and the number of output channels is 16), 3D batch normalization, LeakyReLU activation function, the second 3D convolution (the number of input channels is 16 and the number of output channels is 16), 3D batch normalization, the output feature map is (16, 16, 64, 64).
[0120] Step e: After three-dimensional maximum pooling processing, output feature maps of size (16, 8, 32, 32);
[0121] Step f: After the first three-dimensional convolution (the number of input channels is 16 and the number of output channels is 32), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 32 and the number of output channels is 32), three-dimensional batch normalization, the feature map of size (32, 8, 32, 32) is output;
[0122] Step g: After three-dimensional maximum pooling processing, output a feature map of size (32, 4, 16, 16);
[0123] After the first three-dimensional convolution (the number of input channels is 32, the number of output channels is 64), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 64, the number of output channels is 64), three-dimensional batch normalization, the output feature map of size (64, 4, 16, 16);
[0124] Step h: After three-dimensional maximum pooling processing, output a feature map of size (64, 2, 8, 8);
[0125] Step i: After the first three-dimensional convolution (the number of input channels is 64 and the number of output channels is 128), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (128, 2, 8, 8) is output;
[0126] In the second decoder part the following steps are implemented:
[0127] Step a: Input feature map of size (128, 2, 8, 8), and sequentially pass through three-dimensional transposed convolution (the number of input channels is 128, and the number of output channels is 128), three-dimensional batch normalization, and LeakyReLU activation function to obtain feature map of size (128, 4, 16, 16), and concatenate it with the feature map output in step i of the second encoder along the channel dimension to obtain feature map of size (128+64, 4, 16, 16). Then pass through the first three-dimensional convolution (the number of input channels is 128+64, and the number of output channels is 64), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 64, and the number of output channels is 64), and three-dimensional batch normalization, and output feature map of size (64, 4, 16, 16);
[0128] Step b: After three-dimensional transposed convolution (the number of input channels is 64, the number of output channels is 64), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (64, 8, 32, 32) is obtained, and it is spliced with the feature map output by step g in the second encoder along the channel dimension to obtain a feature map of (64+32, 8, 32, 32). After the first three-dimensional convolution (the number of input channels is 64+32, the number of output channels is 32), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 32, the number of output channels is 32), three-dimensional batch normalization, the output feature map of size (32, 8, 32, 32);
[0129] Step c: After three-dimensional transposed convolution (the number of input channels is 32, the number of output channels is 32), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (32, 16, 64, 64) is obtained, and it is spliced with the feature map output in step e of the second encoder along the channel dimension to obtain a feature map of (32+16, 16, 64, 64). After the first three-dimensional convolution (the number of input channels is 32+16, the number of output channels is 16), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 16, the number of output channels is 16), three-dimensional batch normalization, the output feature map of size (16, 16, 64, 64);
[0130] Step d: After three-dimensional transposed convolution (the number of input channels is 16, the number of output channels is 16), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (16, 32, 128, 128) is obtained, and it is spliced with the feature map output by step c in the second encoder along the channel dimension to obtain a feature map of (16+8, 32, 128, 128). After the first three-dimensional convolution (the number of input channels is 16+8, the number of output channels is 8), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 8, the number of output channels is 8), three-dimensional batch normalization, the output is a feature map of size (8, 32, 128, 128);
[0131] Step e: After three-dimensional transposed convolution (the number of input channels is 8, the number of output channels is 8), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (8, 64, 256, 256) is obtained, and it is spliced with the feature map output by step a in the second encoder along the channel dimension to obtain a feature map of (8+4, 64, 256, 256). Then after the first three-dimensional convolution (the number of input channels is 8+4, the number of output channels is 4), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 4, the number of output channels is 4), three-dimensional batch normalization, the output is a feature map of size (4, 64, 256, 256);
[0132] Step f: After three-dimensional convolution (the number of input channels is 4 and the number of output channels is 2), three-dimensional batch normalization, and LeakyReLU activation function, the predicted probability map of size (2, 64, 256, 256) is output.
[0133] Step g: Use the Softmax activation function to process the predicted probability map to obtain the predicted segmentation binary mask.
[0134] S110. Process the segmentation binary mask using the liver atlas prediction network module to obtain a liver anatomical atlas prediction mask, and complete the anatomical atlas segmentation of the CT image to be segmented based on the liver anatomical atlas prediction mask.
[0135] Among them, in this embodiment, the liver atlas prediction network module continues to use the 5-layer 3DUNet infrastructure, including the third encoder and the third decoder part, as follows:
[0136] In this network structure, all 3D convolutions have a uniform kernel size of (3, 3, 3), stride (1, 1, 1) and padding (1, 1, 1), with the only difference being the number of input channels and the number of output channels. The settings for 3D max pooling are all pooling kernel size of 2, stride parameter of 2, padding parameter of 0, and expansion parameter of 1.
[0137] Implement the following steps in the third encoder part:
[0138] Step a: The four-dimensional data (channel dimension, depth, height, width) of input (1, 64, 256, 256) is sequentially subjected to the first three-dimensional convolution (input channel number 1, output channel number 4), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (input channel number 4, output channel number 4), three-dimensional batch normalization, and outputs a feature map of size (4, 64, 256, 256);
[0139] Step b: After three-dimensional maximum pooling processing, output feature maps of size (4, 32, 128, 128);
[0140] Step c: After the first three-dimensional convolution (the number of input channels is 4 and the number of output channels is 8), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 8 and the number of output channels is 8), three-dimensional batch normalization, the feature map of size (8, 32, 128, 128) is output;
[0141] Step d: After three-dimensional maximum pooling processing, output feature maps of size (8, 16, 64, 64);
[0142] Step e: After the first three-dimensional convolution (the number of input channels is 8 and the number of output channels is 16), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 16 and the number of output channels is 16), three-dimensional batch normalization, the feature map of size (16, 16, 64, 64) is output;
[0143] Step f: After three-dimensional maximum pooling processing, output a feature map of size (16, 8, 32, 32);
[0144] Step g: After the first three-dimensional convolution (the number of input channels is 16 and the number of output channels is 32), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 32 and the number of output channels is 32), three-dimensional batch normalization, the feature map of size (32, 8, 32, 32) is output;
[0145] Step h: After three-dimensional maximum pooling processing, output a feature map of size (32, 4, 16, 16);
[0146] Step i: After the first three-dimensional convolution (the number of input channels is 32 and the number of output channels is 64), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 64 and the number of output channels is 64), three-dimensional batch normalization, the feature map of size (64, 4, 16, 16) is output;
[0147] Step j: After three-dimensional maximum pooling processing, output a feature map of size (64, 2, 8, 8);
[0148] Step k: After the first three-dimensional convolution (the number of input channels is 64 and the number of output channels is 128), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (128, 2, 8, 8) is output;
[0149] In the third decoder part the following steps are implemented:
[0150] Step a: Input a feature map of size (128, 2, 8, 8), and sequentially pass through three-dimensional transposed convolution (the number of input channels is 128, and the number of output channels is 128), three-dimensional batch normalization, and LeakyReLU activation function to obtain a feature map of size (128, 4, 16, 16), which is then concatenated with the feature map output in step i of the third encoder along the channel dimension to obtain a feature map of size (128+64, 4, 16, 16). Then pass through the first three-dimensional convolution (the number of input channels is 128+64, and the number of output channels is 64), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 64, and the number of output channels is 64), and three-dimensional batch normalization to output a feature map of size (64, 4, 16, 16);
[0151] Step b: After three-dimensional transposed convolution (the number of input channels is 64, the number of output channels is 64), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (64, 8, 32, 32) is obtained, and it is spliced with the feature map output in step g of the third encoder along the channel dimension to obtain a feature map of (64+32, 8, 32, 32). After the first three-dimensional convolution (the number of input channels is 64+32, the number of output channels is 32), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 32, the number of output channels is 32), three-dimensional batch normalization, the output feature map of size (32, 8, 32, 32);
[0152] Step c: After three-dimensional transposed convolution (the number of input channels is 32, the number of output channels is 32), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (32, 16, 64, 64) is obtained, and it is spliced with the feature map output in step e of the third encoder along the channel dimension to obtain a feature map of (32+16, 16, 64, 64). Then after the first three-dimensional convolution (the number of input channels is 32+16, the number of output channels is 16), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 16, the number of output channels is 16), three-dimensional batch normalization, the output feature map of size (16, 16, 64, 64);
[0153] Step d: After three-dimensional transposed convolution (the number of input channels is 16, the number of output channels is 16), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (16, 32, 128, 128) is obtained, and it is spliced with the feature map output in step c of the third encoder along the channel dimension to obtain a feature map of (16+8, 32, 128, 128). Then after the first three-dimensional convolution (the number of input channels is 16+8, the number of output channels is 8), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 8, the number of output channels is 8), three-dimensional batch normalization, the output feature map of size (8, 32, 128, 128);
[0154] Step e: After three-dimensional transposed convolution (the number of input channels is 8, the number of output channels is 8), three-dimensional batch normalization, and LeakyReLU activation function, a feature map of size (8, 64, 256, 256) is obtained, and it is spliced with the feature map output in step a of the third encoder along the channel dimension to obtain a feature map of (8+4, 64, 256, 256). Then after the first three-dimensional convolution (the number of input channels is 8+4, the number of output channels is 4), three-dimensional batch normalization, LeakyReLU activation function, the second three-dimensional convolution (the number of input channels is 4, the number of output channels is 4), three-dimensional batch normalization, the output feature map of size (4, 64, 256, 256);
[0155] Step f: After three-dimensional convolution (the number of input channels is 4 and the number of output channels is 2), three-dimensional batch normalization, and LeakyReLU activation function, the predicted probability map of size (2, 64, 256, 256) is output.
[0156] Step g: Use the Softmax activation function to process the predicted probability map to obtain the predicted multi-category mask.
[0157] In one implementation manner, step S103 specifically includes the following steps:
[0158] Based on the Couinaud liver segmentation method, multi-category liver anatomical masks are annotated for all abdominal enhanced CT images;
[0159] The abdominal enhanced CT image is cropped into a cropped CT image according to a preset first cropping quantity and using a region of interest cropping module;
[0160] Cropping the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping number and using a region of interest cropping module;
[0161] The remaining liver anatomical mask is binarized to obtain a binary mask;
[0162] According to a preset third cropping quantity and using a region of interest cropping module, the binary mask is cropped into a cropped binary mask;
[0163] Combining the remaining abdominal enhanced CT images and the binary mask to form a first data set, and dividing the first data set into a first training set image and a first test set image according to a preset division ratio;
[0164] Combining the cropped CT image and the cropped binary mask to form a second data set, and dividing the second data set into a second training set image and a second test set image according to a division ratio;
[0165] The cropped binary mask and the cropped multi-category mask are combined to form a third data set, and the third data set is divided into a third training set image and a third test set image according to the division ratio.
[0166] In this embodiment, for a given abdominal enhanced CT image and source label (multi-category liver anatomical mask), corresponding processing is performed to obtain three sets of data sets, named as the first data set, the second data set and the third data set. The training set and the test set in the above three data sets are divided in an 8:2 ratio, and the division results are consistent, that is, the samples corresponding to the training set and the test set of each data set are consistent.
[0167] The following three transformations are performed on the abdominal enhanced CT images and source labels:
[0168] 1. Using the abdominal enhanced CT image and source label, the cropped image and cropped multi-category mask are obtained through the region of interest cropping module;
[0169] 2. Binarize the source label. The specific operation is as follows: obtain the index of the non-zero area in the source label, assign the corresponding index position to 1.0, and obtain the binary mask;
[0170] 3. The binary labels are processed by the region of interest cropping module to obtain the cropped binary mask.
[0171] Then the above data are matched according to Table 1 to form three data sets.
[0172] Table 1 Comparison table of images and labels in the dataset
[0173]
[0174]
[0175] In one implementation manner, step S105 specifically includes the following steps:
[0176] The target positioning module is trained by combining the preprocessed first training set images and the first test set images, and the first training computing power required for the training process is counted;
[0177] The precise segmentation network module is trained by combining the preprocessed second training set images and the second test set images, and the second training computing power required for the training process is counted;
[0178] Training the liver atlas prediction network module by combining the preprocessed third training set images and the third test set images, and counting the third training computing power required for the training process;
[0179] If the sum of the computing power of the first training computing power, the second training computing power and the third training computing power is less than the preset computing power consumption threshold, the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module is completed in parallel.
[0180] In this embodiment, the three deep learning modules correspond to three purposes, namely, target positioning, high-precision segmentation, and liver anatomical atlas prediction. On the premise of obtaining a data set, the training set construction process can be used to obtain the training sets required by the three models at the same time. The three models are independent of each other, so if the computing power supports it, the training tasks can be submitted in parallel and wait for the training of all three tasks to be completed.
[0181] In one implementation manner, step S106 specifically includes the following steps:
[0182] Reorienting the CT image to be segmented so that the image coordinate axis of the CT image to be segmented is aligned to a preset standard direction;
[0183] Resampling the CT image to be segmented to adjust the voxel size of the CT image to be segmented;
[0184] The image grayscale value of the CT image to be segmented is normalized to obtain preprocessed image three-dimensional data.
[0185] In this embodiment, the CT image to be segmented may be acquired on different devices or scanners, so there may be inconsistencies in its image coordinate axes. In order to ensure the accuracy and stability of the model, it is necessary to redirect the image coordinate axes of the CT image to be segmented so that they are consistent with a preset standard direction. The redirection operation can be achieved by rotating, flipping or rearranging the coordinate axes of the image. The specific operation method can be determined according to specific needs and characteristics of the data set. For example, the coordinate axes of the image can be rotated to be consistent with the standard direction through a rotation operation, or the direction of the image can be adjusted by rearranging the coordinate axes.
[0186] The voxel size of the CT image to be segmented may be inconsistent with the input requirements of the model. In order to ensure the consistency of the model input, the image needs to be resampled and the voxel size needs to be adjusted. The resampling operation can be achieved through interpolation methods. Common interpolation methods include nearest neighbor interpolation, linear interpolation, cubic spline interpolation, etc. The selection of a suitable interpolation method depends on the specific requirements and characteristics of the data set. After resampling, the voxel size of the CT image to be segmented will be consistent with the input requirements of the model. The image grayscale value range of the CT image to be segmented may be inconsistent. In order to ensure the input consistency and stability of the model, the grayscale value of the image needs to be normalized. The normalization operation can map the grayscale value range of the image to a predetermined range, such as [0,1] or [-1,1], through linear transformation. The specific normalization method can be determined according to the specific requirements and characteristics of the data set. After normalization, the image grayscale values of the CT image to be segmented will be in the same range, which is convenient for model training and reasoning.
[0187] In one implementation manner, step S108 specifically includes the following steps:
[0188] Extract non-zero value coordinate indices from the localization binary mask and determine the initial coordinates;
[0189] Adjust the initial coordinates based on the preset offset to obtain the clipping target coordinates;
[0190] A cropping bounding box is generated according to the cropping target coordinates, and the target area contained in the cropping bounding box is cropped as cropping 3D data.
[0191] In this embodiment, the purpose of the region of interest cropping module is to crop the image area corresponding to the target organ three-dimensional bounding box generated by the previous positioning network to form an image block containing the organ. The input of this module is the original CT image and the corresponding bounding box coordinates, and the output is the cropped target organ image block. Its algorithm flow includes:
[0192] Step 1: Determine the width, height and depth coordinates (x_min, x_max, y_min, y_max, z_min, z_max) according to the binary segmentation mask. In the calculation process of the actual bounding box coordinates, first extract the non-zero value coordinate index of the target area from the binary mask. These non-zero value coordinate indexes represent the position of the target in the image or volume. Then, by analyzing the values of the non-zero value coordinate index on the x, y, and z axes, the minimum and maximum values of the target area on each axis can be obtained, namely x_min, x_max, y_min, y_max, z_min, z_max. x_min represents the leftmost position of the target on the x-axis, while x_max represents the rightmost position of the target on the x-axis; y_min represents the top position of the target on the y-axis, while y_max represents the bottom position of the target on the y-axis; z_min represents the closest position of the target on the z-axis, while z_max represents the farthest position of the target on the z-axis. Through these calculations, the bounding box information of the target in three-dimensional space can be determined.
[0193] Step 2: Based on the calculated coordinates, the original image is cropped by setting offsets to extract the image block containing the liver organ.
[0194] The specific method of cutting is as follows:
[0195] Determine the area to be cropped based on the x_min, x_max, y_min, y_max, z_min, and z_max values of the target bounding box. Adjust x_min, x_max, y_min, y_max, z_min, and z_max accordingly based on the given offsets parameter. For example, you can increase the value of offsets to expand the cropping range of the target bounding box and get a larger cropping area. Extract the adjusted three-dimensional area from the original matrix, that is, the area within the range of (x_min-offsets, x_max+offsets), (y_min-offsets, y_max+offsets), and (z_min-offsets, z_max+offsets). The extracted three-dimensional area is the result of cropping the target bounding box.
[0196] Step 3: Output the image block containing the target organ and pass it to subsequent use. The design consideration of this module is that the cropping range should fully include the target organ while avoiding too much background area.
[0197] The present invention also discloses an enhanced CT image anatomical atlas segmentation system based on deep learning, and the system specifically comprises:
[0198] A model building subsystem is used to build a liver lesion segmentation model based on a neural network. The liver lesion segmentation model includes a region of interest clipping module, a target positioning module, a precise segmentation network module, and a liver atlas prediction network module;
[0199] An image acquisition subsystem, used for acquiring CT images to be segmented and multiple abdominal enhanced CT images;
[0200] The dataset construction subsystem is used to annotate multi-category liver anatomical masks for all abdominal enhanced CT images based on the Couinaud liver segmentation method, and to divide all abdominal enhanced CT images into training set images and test set images using the region of interest cropping module;
[0201] A first preprocessing subsystem, used for preprocessing all training set images and test set images;
[0202] The model training subsystem is used to complete the model training of the target positioning module, the precise segmentation network module and the liver atlas prediction network module respectively by combining the preprocessed training set images and the test set images;
[0203] The second preprocessing subsystem is used to preprocess the CT image to be segmented into three-dimensional image data;
[0204] The target localization subsystem is used to input the three-dimensional image data into the liver lesion segmentation model and generate a localization binary mask through the target localization module;
[0205] A clipping subsystem, used for clipping based on the positioning binary mask and through the region of interest clipping module to obtain clipped three-dimensional data;
[0206] The segmentation prediction subsystem is used to predict the cropped 3D data through the precise segmentation network module to obtain the segmentation binary mask;
[0207] The image segmentation subsystem is used to process the segmentation binary mask using the liver atlas prediction network module to obtain the liver anatomical atlas prediction mask, and complete the anatomical atlas segmentation of the CT image to be segmented according to the liver anatomical atlas prediction mask.
[0208] In one embodiment, the data set construction subsystem includes:
[0209] An annotation unit, used to annotate multi-category liver anatomical masks for all abdominal enhanced CT images based on the Couinaud liver segmentation method;
[0210] A first cropping unit, used for cropping the abdominal enhanced CT image into a cropped CT image according to a preset first cropping quantity and using a region of interest cropping module;
[0211] A second cropping unit, used for cropping the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping quantity and using a region of interest cropping module;
[0212] A binarization unit, used for cropping the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping quantity and using a region of interest cropping module;
[0213] A third cropping unit, configured to crop the binary mask into a cropped binary mask according to a preset third cropping quantity and using the region of interest cropping module;
[0214] A first data set construction unit, used to combine the remaining abdominal enhanced CT images and the binary mask to form a first data set, and divide the first data set into a first training set image and a first test set image according to a preset division ratio;
[0215] A second data set construction unit is used to combine the cropped CT image and the cropped binary mask to form a second data set, and divide the second data set into a second training set image and a second test set image according to a division ratio;
[0216] The third data set construction unit is used to combine the cropped binary mask and the cropped multi-category mask to form a third data set, and divide the third data set into third training set images and third test set images according to a division ratio.
[0217] In one embodiment, the model training subsystem includes:
[0218] A first module training unit, used to train the target positioning module by combining the preprocessed first training set images and the first test set images, and to count the first training computing power required for the training process;
[0219] A second module training unit is used to train the precise segmentation network module by combining the preprocessed second training set images and the second test set images, and to count the second training computing power required for the training process;
[0220] A third module training unit, used to train the liver atlas prediction network module in combination with the preprocessed third training set images and the third test set images, and to count the third training computing power required for the training process;
[0221] The parallel training unit is used to control the first module training unit, the second module training unit and the third module training unit to complete the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module in parallel when the sum of the computing power of the first training computing power, the second training computing power and the third training computing power is less than a preset computing power consumption threshold.
[0222] In one embodiment, the second pre-processing subsystem includes:
[0223] A redirection unit, used for performing a redirection operation on the CT image to be segmented so that the image coordinate axis of the CT image to be segmented is aligned to a preset standard direction;
[0224] A resampling unit, used for resampling the CT image to be segmented to adjust the voxel size of the CT image to be segmented;
[0225] The normalization unit is used to normalize the image grayscale value of the CT image to be segmented to obtain pre-processed image three-dimensional data.
[0226] In one embodiment, the cutting subsystem includes:
[0227] A coordinate confirmation unit, used to extract non-zero value coordinate indexes from the positioning binary mask and determine the initial coordinates;
[0228] A coordinate adjustment unit, used to adjust the initial coordinates based on a preset offset to obtain a clipping target coordinate;
[0229] The region cropping unit is used to generate a cropping bounding box according to the cropping target coordinates, and to crop the target region contained in the cropping bounding box into cropping three-dimensional data.
[0230] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as above, which are not provided in detail for the sake of simplicity.
[0231] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.
Claims
1. A method for segmenting enhanced CT image anatomical atlas based on deep learning, characterized in that: The steps include: Constructing a liver lesion segmentation model based on a neural network, wherein the liver lesion segmentation model includes a region of interest clipping module, a target positioning module, a precise segmentation network module, and a liver atlas prediction network module; Acquire the CT image to be segmented and multiple abdominal enhanced CT images; Based on the Couinaud liver segmentation method, multi-category liver anatomical masks are annotated for all the abdominal enhanced CT images, and the region of interest cropping module is used to divide all the abdominal enhanced CT images into training set images and test set images; Preprocessing all the training set images and the test set images; Combining the preprocessed training set images and the test set images, respectively completing the model training of the target positioning module, the precise segmentation network module, and the liver atlas prediction network module; Preprocessing the CT image to be segmented into three-dimensional image data; Inputting the three-dimensional image data into the liver lesion segmentation model, and generating a positioning binary mask through the target positioning module; Based on the positioning binary mask, a cropping process is performed through the region of interest cropping module to obtain cropped three-dimensional data; Performing prediction processing on the cropped three-dimensional data through the precise segmentation network module to obtain a segmentation binary mask; The segmentation binary mask is processed by using the liver atlas prediction network module to obtain a liver anatomical atlas prediction mask, and the anatomical atlas segmentation of the CT image to be segmented is completed according to the liver anatomical atlas prediction mask.
2. The method for segmenting enhanced CT image anatomical atlas based on deep learning according to claim 1, characterized in that: The method of labeling multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method, and dividing all the abdominal enhanced CT images into training set images and test set images using the region of interest cropping module comprises the following steps: Annotating multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method; Cropping the abdominal enhanced CT image into a cropped CT image according to a preset first cropping number and using the region of interest cropping module; According to a preset second cropping quantity and using the region of interest cropping module, the multi-category liver anatomical mask is cropped into a cropped multi-category mask; Binarizing the remaining liver anatomical mask to obtain a binary mask; According to a preset third cropping quantity and using the region of interest cropping module, the binary mask is cropped into a cropped binary mask; Combining the remaining abdominal enhanced CT images and the binary mask to form a first data set, and dividing the first data set into a first training set image and a first test set image according to a preset division ratio; Combining the cropped CT image and the cropped binary mask to form a second data set, and dividing the second data set into a second training set image and a second test set image according to the division ratio; The cropped binary mask and the cropped multi-category mask are combined to form a third data set, and the third data set is divided into a third training set image and a third test set image according to the division ratio.
3. The method for segmenting enhanced CT image anatomical atlas based on deep learning according to claim 2, characterized in that: The step of combining the preprocessed training set images and the test set images to respectively complete the model training of the target positioning module, the precise segmentation network module, and the liver atlas prediction network module comprises the following steps: Training the target positioning module by combining the preprocessed first training set images and the first test set images, and calculating the first training computing power required for the training process; Training the precise segmentation network module in combination with the preprocessed second training set images and the second test set images, and calculating the second training computing power required for the training process; Training the liver atlas prediction network module in combination with the preprocessed third training set images and the third test set images, and counting the third training computing power required for the training process; If the sum of the computing power of the first training computing power, the second training computing power and the third training computing power is less than a preset computing power consumption threshold, the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module are completed in parallel.
4. The method for segmenting enhanced CT image anatomical atlas based on deep learning according to claim 1, characterized in that: The preprocessing of the CT image to be segmented into three-dimensional image data comprises the following steps: Redirecting the CT image to be segmented so that the image coordinate axis of the CT image to be segmented is aligned to a preset standard direction; Resampling the CT image to be segmented to adjust the voxel size of the CT image to be segmented; The image grayscale value of the CT image to be segmented is normalized to obtain pre-processed image three-dimensional data.
5. The method for segmenting enhanced CT image anatomical atlas based on deep learning according to claim 1, characterized in that: The method of performing cropping processing based on the positioning binary mask and using the region of interest cropping module to obtain cropped three-dimensional data includes the following steps: Extracting non-zero valued coordinate indices from the positioning binary mask and determining initial coordinates; Adjusting the initial coordinates based on a preset offset to obtain a clipping target coordinate; A cropping bounding box is generated according to the cropping target coordinates, and the target area included in the cropping bounding box is cropped into cropped three-dimensional data.
6. An enhanced CT image anatomical atlas segmentation system based on deep learning, characterized in that: The system comprises: A model building subsystem, used to build a liver lesion segmentation model based on a neural network, wherein the liver lesion segmentation model includes a region of interest clipping module, a target positioning module, a precise segmentation network module, and a liver atlas prediction network module; An image acquisition subsystem, used for acquiring CT images to be segmented and multiple abdominal enhanced CT images; A data set construction subsystem, for annotating multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method, and for dividing all the abdominal enhanced CT images into training set images and test set images using the region of interest cropping module; A first preprocessing subsystem, used for preprocessing all the training set images and the test set images; A model training subsystem, used to complete the model training of the target positioning module, the precise segmentation network module and the liver atlas prediction network module respectively by combining the preprocessed training set images and the test set images; A second preprocessing subsystem, used for preprocessing the CT image to be segmented into three-dimensional image data; A target positioning subsystem, used for inputting the three-dimensional image data into the liver lesion segmentation model, and generating a positioning binary mask through the target positioning module; A clipping subsystem, configured to perform clipping processing based on the positioning binary mask and through the region of interest clipping module to obtain clipped three-dimensional data; A segmentation prediction subsystem, used for performing prediction processing on the cropped three-dimensional data through the precise segmentation network module to obtain a segmentation binary mask; The image segmentation subsystem is used to process the segmentation binary mask using the liver atlas prediction network module to obtain a liver anatomical atlas prediction mask, and complete the anatomical atlas segmentation of the CT image to be segmented according to the liver anatomical atlas prediction mask.
7. The deep learning-based enhanced CT image anatomical atlas segmentation system according to claim 6, characterized in that: The data set construction subsystem includes: A labeling unit, used for labeling multi-category liver anatomical masks for all the abdominal enhanced CT images based on the Couinaud liver segmentation method; A first cropping unit, configured to crop the abdominal enhanced CT image into a cropped CT image according to a preset first cropping quantity and using the region of interest cropping module; A second cropping unit, configured to crop the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping quantity and using the region of interest cropping module; A binarization unit, configured to crop the multi-category liver anatomical mask into a cropped multi-category mask according to a preset second cropping quantity and using the region of interest cropping module; A third cropping unit, configured to crop the binary mask into a cropped binary mask according to a preset third cropping quantity and using the region of interest cropping module; A first data set construction unit, configured to combine the remaining abdominal enhanced CT images and the binary mask to form a first data set, and divide the first data set into a first training set image and a first test set image according to a preset division ratio; A second data set construction unit, configured to combine the cropped CT image and the cropped binary mask to form a second data set, and divide the second data set into a second training set image and a second test set image according to the division ratio; The third data set construction unit is used to combine the cropped binary mask and the cropped multi-category mask to form a third data set, and divide the third data set into a third training set image and a third test set image according to the division ratio.
8. The deep learning-based enhanced CT image anatomical atlas segmentation system according to claim 7, characterized in that: The model training subsystem includes: A first module training unit, used to train the target positioning module in combination with the preprocessed first training set images and the first test set images, and to count the first training computing power required for the training process; A second module training unit, used to train the precise segmentation network module in combination with the preprocessed second training set images and the second test set images, and to count the second training computing power required for the training process; A third module training unit, used to train the liver atlas prediction network module in combination with the preprocessed third training set images and the third test set images, and to count the third training computing power required for the training process; A parallel training unit is used to control the first module training unit, the second module training unit and the third module training unit to complete the model training of the target positioning module, the precise segmentation network module and the liver map prediction network module in parallel when the sum of the computing power of the first training computing power, the second training computing power and the third training computing power is less than a preset computing power consumption threshold.
9. The deep learning-based enhanced CT image anatomical atlas segmentation system according to claim 6, characterized in that: The second pre-processing subsystem comprises: a redirection unit, configured to perform a redirection operation on the CT image to be segmented so that the image coordinate axis of the CT image to be segmented is aligned to a preset standard direction; A resampling unit, used for performing a resampling process on the CT image to be segmented so as to adjust the voxel size of the CT image to be segmented; The normalization unit is used to normalize the image grayscale value of the CT image to be segmented to obtain pre-processed three-dimensional image data.
10. The deep learning-based enhanced CT image anatomical atlas segmentation system according to claim 6, characterized in that: The cutting subsystem comprises: A coordinate confirmation unit, used to extract non-zero value coordinate indexes from the positioning binary mask and determine initial coordinates; A coordinate adjustment unit, used to adjust the initial coordinates based on a preset offset to obtain a clipping target coordinate; The area clipping unit is used to generate a clipping boundary box according to the clipping target coordinates, and clip the target area included in the clipping boundary box into clipping three-dimensional data.
Citation Information
Cited By
Neural CT map segmentation method based on high-precision registration
CN119649040A
