A method and device for segmenting liver tumors

Abstract

The present invention discloses a method and device for liver tumor segmentation, which adopts a staged processing strategy, combines a large tumor segmentation model, a small tumor detection model, a suppression model and a small tumor segmentation network, thereby improving the tumor detection rate, and also uses a suppression network to reduce the false positive rate, further improve the segmentation accuracy, and achieve effective identification and accurate segmentation of different types of tumors. The method steps mainly include reasoning with a large tumor segmentation model; reasoning with a small tumor detection model; removing redundant detection frames; suppressing the model; and segmenting small tumors and merging the results.

Description

Technical Field

[0001] The present invention relates to medical image processing technology, and in particular to the technical field of liver tumor segmentation. Background Art

[0002] In medical image analysis, accurate segmentation of liver tumors is crucial for early diagnosis, treatment planning, and efficacy evaluation. In recent years, with the development of computer vision technology and deep learning algorithms, image segmentation methods based on artificial intelligence have gradually become a research hotspot. However, existing liver tumor segmentation methods still face many challenges:

[0003] Tumor size varies greatly: The size of liver tumors can range from small lesions of a few millimeters to large masses of more than 20 centimeters. A single segmentation network is difficult to take into account tumors of different sizes, and is particularly prone to missing small tumors.

[0004] False positive problem: Although some existing segmentation models can detect larger tumors well, they are often accompanied by a high false positive rate when detecting small tumors, that is, non-tumor areas are incorrectly marked as tumors, which causes trouble for clinical decision-making. Summary of the invention

[0005] The present invention aims to overcome the deficiencies in the prior art and provide a method and device for liver tumor segmentation. The method adopts a staged processing strategy, combines a large tumor segmentation model, a small tumor detection model, an inhibition model and a small tumor segmentation network, thereby improving the detection rate of tumors. The inhibition network is also used to reduce the false positive rate, further improve the segmentation accuracy, and achieve effective identification and precise segmentation of different types of tumors.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for segmenting a liver tumor according to the present invention comprises the following steps:

[0008] After the abdominal CT images are preprocessed, they are input into a mature large tumor segmentation model for inference to obtain the liver mask and liver tumor mask. Connected domain analysis is performed to take the liver tumor mask with a diameter greater than 15 mm as the first liver tumor mask, and the liver tumor mask with a diameter less than or equal to 15 mm as the second liver tumor mask.

[0009] After the abdominal CT image is preprocessed, it is input into the small tumor detection model for inference to obtain a preliminary predicted bbox and a confidence C, where the diameter of the predicted bbox is less than or equal to 30 mm;

[0010] Delete the redundant predicted bboxes in the preliminary predicted bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox;

[0011] Segment a second bbox of the second liver tumor mask, mix the first bbox and the second bbox into a mixed bbox, input the trained suppression model into the mixed bbox, obtain the probability P of being a positive sample, multiply the probability P by the confidence of the corresponding bbox to obtain the final probability P1 of the mixed bbox being a positive sample, and obtain a positive sample whose final probability P1 is greater than or equal to the probability threshold; the confidence value of the second bbox is 1, and the confidence value of the first bbox is confidence C;

[0012] The first bbox belonging to the positive sample is input into the trained small tumor segmentation network, and the inference result is analyzed for connected domains. The connected domain whose center is closest to the predicted bbox center is taken as the small tumor mask;

[0013] The first liver tumor mask, the second liver tumor mask belonging to the positive sample, and the small tumor mask are restored to the corresponding positions of the original image to complete the liver tumor segmentation.

[0014] Preferably, the small tumor detection model adopts a RetinaNet detection network, and the training method of the small tumor detection model includes:

[0015] In the training set samples, based on the pre-labeled liver tumor GT mask, the circumscribed cube of the tumor with a diameter less than or equal to 30 mm was extracted as the training GT Bbox through connected domain analysis;

[0016] The abdominal CT image is interpolated to 0.75 in the x, y, and z directions. After normalization, a 64*64*64 data block is cut out with the center of the gt bbox as the center and input into the small tumor detection model for training.

[0017] Preferably, the step of inputting the abdominal CT image into the small tumor detection model for reasoning after preprocessing further includes: after the abdominal CT image is resampled and normalized, it is cut into blocks with a sliding window size of 128*128*128 and input into the small tumor detection model for reasoning.

[0018] Preferably, the step of deleting redundant predicted bboxes in the preliminary predicted bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox further includes:

[0019] Delete the bbox whose center coordinates are outside the liver mask segmented by the large tumor segmentation model;

[0020] Delete the bbox whose center coordinates are within the range of the first liver tumor mask and the second liver tumor mask.

[0021] As a preferred method, the probability threshold is determined by:

[0022] Input the data of the validation set into the suppression model and obtain the probability P2 of each validation set sample target after inference, and sort the inferred targets from large to small according to P2;

[0023] The threshold is set starting from the maximum value of P2 and gradually decreasing. As the threshold decreases, the detection rate and false positive rate will increase; until the threshold with an average of two false positives per sample image is used as the final probability threshold.

[0024] Preferably, the large tumor segmentation model adopts a 3D UNet segmentation network based on the nnUNet framework, and the abdominal CT image is input into the network after resampling and normalization, and the liver mask and liver tumor mask are output.

[0025] Preferably, the training method of the inhibition model comprises:

[0026] For the training set samples, the second bbox of the second liver tumor mask is segmented by the large tumor segmentation model, and the first bbox detected by the small tumor detection model is mixed with the first bbox and the second bbox into a mixed bbox.

[0027] Calculate the IoU value of the mixed bbox and the gt bbox pre-labeled in the training set. The mixed bbox with IoU>=0.1 is used as a positive sample, and the bbox with IoU=0 is used as a negative sample.

[0028] After resampling and normalization, the abdominal CT images of the training set are cut out into blocks of size 48*48*48 with the center of the mixed bbox as the center and input into the network for training.

[0029] Preferably, the small tumor segmentation network uses a simple 3D Unet for segmentation, and the small tumor segmentation network training method includes:

[0030] The abdominal CT images of the training set were resampled and normalized.

[0031] Taking the center of the first bbox after inference and deletion of redundancy of the small tumor detection model as the center, a 48*48*48 block is cut out and sent to the network for training.

[0032] The present invention also provides a device for liver tumor segmentation, the device comprising:

[0033] The large tumor segmentation module is used to input the abdominal CT image into a mature large tumor segmentation model for inference after preprocessing, and obtain the liver mask and liver tumor mask; perform connected domain analysis, and use the liver tumor mask with a diameter greater than 15 mm as the first liver tumor mask, and the liver tumor mask with a diameter less than or equal to 15 mm as the second liver tumor mask;

[0034] A small tumor detection module is used to pre-process the abdominal CT image and input it into the small tumor detection model for inference to obtain a preliminary predicted bbox and a confidence C, where the diameter of the predicted bbox is less than or equal to 30 mm;

[0035] The redundant detection box removal module is used to delete the redundant prediction bboxes in the preliminary prediction bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox;

[0036] A suppression module is used to segment a second bbox of a second liver tumor mask, mix the first bbox and the second bbox into a mixed bbox, input the mixed bbox into a trained suppression model, obtain a probability P of a positive sample, multiply the probability P by the confidence of the corresponding bbox to obtain a final probability P1 that the mixed bbox is a positive sample, and obtain a positive sample whose final probability P1 is greater than or equal to a probability threshold; the confidence value of the second bbox is 1, and the confidence value of the first bbox is confidence C;

[0037] The small tumor segmentation module is used to input the trained small tumor segmentation network according to the first bbox belonging to the positive sample, perform connected domain analysis on the inference result, and take the connected domain whose center is closest to the predicted bbox center as the small tumor mask;

[0038] The result merging module is used to restore the first liver tumor mask, the second liver tumor mask belonging to the positive sample, and the small tumor mask to the corresponding positions of the original image to complete the liver tumor segmentation.

[0039] This solution uses a segmentation network to focus on the segmentation of large tumors; for small tumors, the detection network is first used to determine the location of the tumor, then the image near the small tumor is cut out, and the small tumor is segmented through the segmentation network; finally, the segmentation results of the large tumor are combined with the segmentation results of the small tumor, thereby improving the detection rate of the tumor. This solution also uses a suppression network to reduce the false positive rate and further improve the segmentation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The figure is a flow chart of a method for liver tumor segmentation according to the present invention.

[0041] Figure 2FIG. 4 is another flow chart of a method for segmenting a liver tumor according to the present invention.

[0042] Figure 3 The figure is a principle block diagram of a device for segmenting liver tumors according to the present invention. DETAILED DESCRIPTION

[0043] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0044] like Figure 1 , Figure 2 As shown, a method for segmenting a liver tumor according to the present invention comprises the following steps:

[0045] Step S1, after the abdominal CT image is preprocessed, a mature large tumor segmentation model is input for inference to obtain a liver mask and a liver tumor mask; a connected domain analysis is performed, and the liver tumor mask with a diameter greater than 15 mm is used as the first liver tumor mask, and the liver tumor mask with a diameter less than or equal to 15 mm is used as the second liver tumor mask.

[0046] The large tumor segmentation model can adopt a mature model in the prior art. Because such models in the prior art do not have directional optimization for small-sized tumors, they are prone to miss small-sized tumors. The large tumor segmentation model described in this embodiment adopts a 3D UNet segmentation network based on the nnUNet framework. The abdominal CT image is resampled and normalized and then input into the network to output a liver mask and a liver tumor mask.

[0047] Step S2: After the abdominal CT image is preprocessed, it is input into the small tumor detection model for reasoning to obtain a preliminary predicted bbox and a confidence C, where the diameter of the predicted bbox is less than or equal to 30 mm.

[0048] The specific steps of abdominal CT image preprocessing are: after resampling and normalization, the abdominal CT image is cut into blocks with a sliding window size of 128*128*128 and input into the small tumor detection model for inference.

[0049] The small tumor detection model adopts the RetinaNet detection network, and the training method of the small tumor detection model includes:

[0050] Step 201, in the training set samples, based on the pre-labeled liver tumor GT mask, through connected domain analysis, the circumscribed cube of the tumor with a diameter less than or equal to 30 mm is extracted as the training GT BBox;

[0051] Step 202, the abdominal CT image is interpolated to 0.75 in the three directions of x, y, and z, and after normalization, a data block of 64*64*64 is cut out with the center of the gt bbox as the center, and input into the small tumor detection model for training.

[0052] The small tumor detection model in step 2 is trained to identify and locate liver tumors with a diameter of less than or equal to 30 mm, ensuring that the model focuses on learning the characteristics of small tumors to make up for the problem of missed detection of small tumors that may result from using the segmentation network alone. The confidence C of each bbox is used to evaluate the reliability of the detection results.

[0053] Image preprocessing: The image is interpolated to 0.75 in the x, y, and z directions and normalized to ensure the consistency and quality of the input. During inference, the image is resampled and normalized, and then cut into 128*128*128 sliding windows and input into the network to ensure that the entire image is covered and local features are captured.

[0054] Step S3, delete the redundant prediction bboxes in the preliminary prediction bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox.

[0055] Specifically, the step S3 further includes:

[0056] Step 301, deleting the bbox whose center coordinates are outside the liver mask range segmented by the large tumor segmentation model;

[0057] Step 302: Delete the bbox whose center coordinates are within the range of the first liver tumor mask and the second liver tumor mask.

[0058] The step 3 of removing redundant detection frames is mainly used to reduce false positives, eliminate detection frames that are outside the liver range or overlap with large tumor masks, avoid false positives caused by background noise and repeated detection; improve accuracy, ensure that the retained detection frames more accurately correspond to the actual small tumors, and reduce interference in subsequent processing; optimize resources: reduce the amount of data that needs further analysis, improve processing efficiency, and reduce computing costs. Focus on key areas: let the model focus on those areas that are most likely to be tumors, rather than being distracted by a large number of false positive detections.

[0059] Step S4, segmenting the second bbox of the second liver tumor mask, mixing the first bbox and the second bbox into a mixed bbox, inputting the trained suppression model into the mixed bbox, obtaining the probability P of a positive sample, multiplying the probability P by the confidence of the corresponding bbox to obtain the final probability P1 that the mixed bbox is a positive sample, and obtaining a positive sample whose final probability P1 is greater than or equal to the probability threshold; the confidence value of the second bbox is 1, and the confidence value of the first bbox is confidence C.

[0060] The suppression model uses a res2net network to perform binary classification of characters, and its training method includes the following steps:

[0061] Step S401, for the training set samples, a second bbox of the second liver tumor mask is segmented by the large tumor segmentation model, and the first bbox detected by the small tumor detection model is mixed with the first bbox and the second bbox into a mixed bbox.

[0062] Step S402, calculate the IoU value between the mixed bbox and the gt bbox pre-labeled in the training set, and the mixed bbox with IoU>=0.1 is used as a positive sample, and the bbox with IoU=0 is used as a negative sample.

[0063] IoU (Intersection over Union) is an indicator to measure the degree of overlap between two bounding boxes. It is usually used to evaluate the match between the predicted box and the true label box in object detection tasks.

[0064] Step S403, after the abdominal CT images of the training set are resampled and normalized, a block of 48*48*48 size is cut out with the center of the mixed bbox as the center, and input into the network for training.

[0065] Method for determining the probability threshold:

[0066] Input the data of the validation set into the suppression model and obtain the probability P2 of each validation set sample target after inference, and sort the inferred targets from large to small according to P2;

[0067] The threshold is set starting from the maximum value of P2 and gradually decreasing. As the threshold decreases, the detection rate and false positive rate will increase; until the threshold with an average of two false positives per sample image is used as the final probability threshold.

[0068] Clinically, two false positives in each CT image are acceptable, so the threshold with an average of two false positives in each image is taken as the final threshold.

[0069] The main function of step 4 is to reduce false positives (FP) in the segmentation and detection process, that is, those areas that are mistakenly identified as tumors but are not actually tumors. By reducing false positives to improve the performance of the model, the suppression model can significantly improve the overall accuracy and reliability of liver tumor segmentation.

[0070] Step S5, the first bbox belonging to the positive sample is input into the trained small tumor segmentation network, and the inference result is subjected to connected domain analysis, and the connected domain whose center is closest to the predicted bbox center is taken as the small tumor mask.

[0071] Specifically, the small tumor segmentation network uses a simple 3D Unet for segmentation, and the small tumor segmentation network training method includes:

[0072] The abdominal CT images of the training set were resampled and normalized.

[0073] Taking the center of the first bbox after inference and deletion of redundancy of the small tumor detection model as the center, a 48*48*48 block is cut out and sent to the network for training.

[0074] Step S6, restoring the first liver tumor mask, the second liver tumor mask belonging to the positive sample, and the small tumor mask to corresponding positions of the original image to complete the liver tumor segmentation.

[0075] like Figure 3 As shown, the present invention also provides a device for liver tumor segmentation, the device comprising:

[0076] The large tumor segmentation module is used to input the abdominal CT image into a mature large tumor segmentation model for inference after preprocessing, and obtain the liver mask and liver tumor mask; perform connected domain analysis, and use the liver tumor mask with a diameter greater than 15 mm as the first liver tumor mask, and the liver tumor mask with a diameter less than or equal to 15 mm as the second liver tumor mask;

[0077] A small tumor detection module is used to pre-process the abdominal CT image and input it into the small tumor detection model for inference to obtain a preliminary predicted bbox and a confidence C, where the diameter of the predicted bbox is less than or equal to 30 mm;

[0078] The redundant detection box removal module is used to delete the redundant prediction bboxes in the preliminary prediction bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox;

[0079] A suppression module is used to segment a second bbox of a second liver tumor mask, mix the first bbox and the second bbox into a mixed bbox, input the mixed bbox into a trained suppression model, obtain a probability P of a positive sample, multiply the probability P by the confidence of the corresponding bbox to obtain a final probability P1 that the mixed bbox is a positive sample, and obtain a positive sample whose final probability P1 is greater than or equal to a probability threshold; the confidence value of the second bbox is 1, and the confidence value of the first bbox is confidence C;

[0080] The small tumor segmentation module is used to input the trained small tumor segmentation network according to the first bbox belonging to the positive sample, perform connected domain analysis on the inference result, and take the connected domain whose center is closest to the predicted bbox center as the small tumor mask;

[0081] The result merging module is used to restore the first liver tumor mask, the second liver tumor mask belonging to the positive sample, and the small tumor mask to the corresponding positions of the original image to complete the liver tumor segmentation.

[0082] This solution uses a segmentation network to focus on the segmentation of large tumors; for small tumors, the detection network is first used to determine the location of the tumor, then the image near the small tumor is cut out, and the small tumor is segmented through the segmentation network; finally, the segmentation results of the large tumor are combined with the segmentation results of the small tumor, thereby improving the detection rate of the tumor. This solution also uses a suppression network to reduce the false positive rate and further improve the segmentation accuracy.

Claims

1. A method for segmenting a liver tumor, characterized in that: The method comprises the following steps: After the abdominal CT images are preprocessed, they are input into a mature large tumor segmentation model for inference to obtain the liver mask and liver tumor mask. Connected domain analysis is performed to take the liver tumor mask with a diameter greater than 15 mm as the first liver tumor mask, and the liver tumor mask with a diameter less than or equal to 15 mm as the second liver tumor mask. After the abdominal CT image is preprocessed, it is input into the small tumor detection model for inference to obtain a preliminary predicted bbox and a confidence C, where the diameter of the predicted bbox is less than or equal to 30 mm; Delete the redundant predicted bboxes in the preliminary predicted bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox; Segment a second bbox of the second liver tumor mask, mix the first bbox and the second bbox into a mixed bbox, input the trained suppression model into the mixed bbox, obtain the probability P of being a positive sample, multiply the probability P by the confidence of the corresponding bbox to obtain the final probability P1 of the mixed bbox being a positive sample, and obtain a positive sample whose final probability P1 is greater than or equal to the probability threshold; the confidence value of the second bbox is 1, and the confidence value of the first bbox is confidence C; The first bbox belonging to the positive sample is input into the trained small tumor segmentation network, and the inference result is analyzed for connected domains. The connected domain whose center is closest to the predicted bbox center is taken as the small tumor mask; The first liver tumor mask, the second liver tumor mask belonging to the positive sample, and the small tumor mask are restored to the corresponding positions of the original image to complete the liver tumor segmentation.

2. A method for segmenting a liver tumor according to claim 1, characterized in that: The small tumor detection model adopts the RetinaNet detection network, and the training method of the small tumor detection model includes: In the training set samples, based on the pre-labeled liver tumor GT mask, the circumscribed cube of the tumor with a diameter less than or equal to 30 mm was extracted as the training GT Bbox through connected domain analysis; The abdominal CT image is interpolated to 0.75 in the x, y, and z directions. After normalization, a 64*64*64 data block is cut out with the center of the gt bbox as the center and input into the small tumor detection model for training.

3. A method for segmenting a liver tumor according to claim 2, characterized in that: The step of inputting the abdominal CT image into the small tumor detection model for reasoning after the abdominal CT image is preprocessed further includes: after the abdominal CT image is resampled and normalized, it is cut into blocks with a sliding window size of 128*128*128, and input into the small tumor detection model for reasoning.

4. The method for segmenting a liver tumor according to claim 1, wherein: The step of deleting redundant prediction bboxes in the preliminary prediction bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox further includes: Delete the bbox whose center coordinates are outside the liver mask segmented by the large tumor segmentation model; Delete the bbox whose center coordinates are within the range of the first liver tumor mask and the second liver tumor mask.

5. The method for segmenting a liver tumor according to claim 1, characterized in that: Method for determining the probability threshold: Input the data of the validation set into the suppression model and obtain the probability P2 of each validation set sample target after inference, and sort the inferred targets from large to small according to P2; The threshold is set starting from the maximum value of P2 and gradually decreasing. As the threshold decreases, the detection rate and false positive rate will increase; until the threshold with an average of two false positives per sample image is used as the final probability threshold.

6. The method for segmenting a liver tumor according to claim 1, characterized in that: The large tumor segmentation model adopts a 3D UNet segmentation network based on the nnUNet framework. The abdominal CT image is resampled and normalized and then input into the network to output a liver mask and a liver tumor mask.

7. The method for segmenting a liver tumor according to claim 1, characterized in that: The training method of the inhibition model includes: For the training set samples, the second bbox of the second liver tumor mask is segmented by the large tumor segmentation model, and the first bbox detected by the small tumor detection model is mixed with the first bbox and the second bbox into a mixed bbox. Calculate the IoU value of the mixed bbox and the gt bbox pre-labeled in the training set. The mixed bbox with IoU>=0.1 is used as a positive sample, and the bbox with IoU=0 is used as a negative sample. After resampling and normalization, the abdominal CT images of the training set are cut out into blocks of size 48*48*48 with the center of the mixed bbox as the center and input into the network for training.

8. The method for segmenting a liver tumor according to claim 1, characterized in that: The small tumor segmentation network uses a simple 3D Unet for segmentation, and the small tumor segmentation network training method includes: The abdominal CT images of the training set were resampled and normalized. Taking the center of the first bbox after inference and deletion of redundancy of the small tumor detection model as the center, a 48*48*48 block is cut out and sent to the network for training.

9. A device for segmenting liver tumors, characterized in that: The device comprises: The large tumor segmentation module is used to input the abdominal CT image into a mature large tumor segmentation model for inference after preprocessing, and obtain the liver mask and liver tumor mask; perform connected domain analysis, and use the liver tumor mask with a diameter greater than 15 mm as the first liver tumor mask, and the liver tumor mask with a diameter less than or equal to 15 mm as the second liver tumor mask; A small tumor detection module is used to pre-process the abdominal CT image and input it into the small tumor detection model for inference to obtain a preliminary predicted bbox and a confidence C, where the diameter of the predicted bbox is less than or equal to 30 mm; The redundant detection box removal module is used to delete the redundant prediction bboxes in the preliminary prediction bbox that are not within the liver range or overlap with the tumor mask segmented by the large tumor segmentation model to obtain the first bbox; A suppression module is used to segment a second bbox of a second liver tumor mask, mix the first bbox and the second bbox into a mixed bbox, input the mixed bbox into a trained suppression model, obtain a probability P of a positive sample, multiply the probability P by the confidence of the corresponding bbox to obtain a final probability P1 that the mixed bbox is a positive sample, and obtain a positive sample whose final probability P1 is greater than or equal to a probability threshold; the confidence value of the second bbox is 1, and the confidence value of the first bbox is confidence C; The small tumor segmentation module is used to input the trained small tumor segmentation network according to the first bbox belonging to the positive sample, perform connected domain analysis on the inference result, and take the connected domain whose center is closest to the predicted bbox center as the small tumor mask; The result merging module is used to restore the first liver tumor mask, the second liver tumor mask belonging to the positive sample, and the small tumor mask to the corresponding positions of the original image to complete the liver tumor segmentation.

Images