An intelligent delineation system and method for clinical target volume and normal organs in cervical cancer

Abstract

This invention relates to an intelligent delineation system and method for cervical cancer clinical target areas and normal organs. The system includes a training unit and a prediction unit. The training unit includes an input module, an image preprocessing module, a 3D-Unet deep learning model construction module, a training network, and an output module. The prediction unit includes an input terminal, a confidence map overlay output module, and an output terminal. The input terminal is used to input new identical multimodal images into three different trained 3D-Unet deep learning models to obtain confidence maps for each of the three different trained 3D-Unet deep learning models. The confidence map overlay output module is used to overlay the maximum values ​​of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ segmentation results, and output the prediction results through the output terminal.

Description

Technical Field

[0001] This invention relates to the field of medical assistance technology, specifically to an intelligent delineation system and method for cervical cancer clinical target areas and normal organs based on artificial intelligence (deep learning). Background Technology

[0002] Radiation therapy (RT) is a non-surgical option for many cancers. Similarly, RT is an effective way to improve survival rates for patients with cervical cancer, especially those with locally advanced cervical cancer and those whose physical condition is unsuitable for surgery. The preferred method of RT for locally advanced cervical cancer is intensity-modulated radiotherapy (IMRT). To achieve optimal treatment results, the radiation dose needs to be increased for the target area, while radiation damage to surrounding normal tissues and organs needs to be reduced. Therefore, the key to successfully performing IMRT is accurately mapping the clinical target volume (CTV) and organs at risk (OARs). Currently, manual segmentation of the CTV by physicians remains the standard, but this is a time-consuming and laborious task, still requiring at least 30 minutes even for experienced physicians. Even with guidelines, different physicians have different habits, and even the same physician may have different segmentation results at different times; there are also reports of observer-to-observer variability in the literature. It is important to note that most CTVs do not have well-defined boundaries (unlike OARs, which mostly have well-defined boundaries), and their outlines include not only the apparent lesion volume but also regional lymph nodes and other suspected pathways of tumor spread. CTV varies considerably depending on individual differences, lesion location, and cancer stage. Furthermore, even patients at the same stage may exhibit varying degrees of tumor invasion and lymph node involvement. All of these factors contribute to different segmentation results.

[0003] Traditional automated segmentation methods, such as supervised and unsupervised machine learning, based on Atlas models and statistical models, can achieve good segmentation results, but the results still require time-consuming manual editing by physicians. Unfortunately, both methods have a limitation: they cannot handle the significant differences between different images and different patients. While large datasets can address this issue, medical databases, though large, are diverse (i.e., different types, different equipment, significant differences in data quality, and significant individual patient differences), and many other factors are unmanageable. Therefore, the field requires using a limited number of available samples to achieve the desired results. All of these factors have led to the development of deep learning (DL) networks.

[0004] Compared to manual segmentation, automated segmentation has shown great potential since its inception, such as reducing physician workload, decreasing patient wait times, and improving cancer treatment. During IMRT for cervical cancer, dramatic anatomical changes also necessitate advanced adaptive radiotherapy (ART) strategies. Simultaneously, in low- and middle-income regions and areas with limited medical resources, adhering to guidelines for radiotherapy is challenging. In such contexts, automated segmentation can enhance healthcare both locally and globally.

[0005] Although deep learning networks have existed since the 1940s, it wasn't until 2006 that deep learning became a branch of machine learning and was hailed as one of the top ten technological breakthroughs since 2013. Initially, image segmentation in deep learning was accomplished using convolutional neural networks (CNNs). CNNs typically consist of convolutional layers, pooling layers, and fully connected layers. Their complex structure resulted in a large enough sample size and a significant amount of time and computational power required to train the model. Furthermore, due to the fixed number of nodes, CNNs were limited in their ability to handle image sizes. This problem was subsequently solved by the advent of fully convolutional networks (FCNs), which use convolutional layers instead of the fully connected layers of CNNs, allowing FCN models to handle any image size. Moreover, FCNs improved the segmentation efficiency of CNNs due to their skip connections. Unfortunately, the model also suffered from excessive multiplier upsampling, leading to insufficient integration of contextual information and a decrease in segmentation accuracy. The most popular FCN architecture for medical image segmentation is U-net, which uses an equal number of upsampling and downsampling convolutional layers. The upsampling layers can accept features extracted from the corresponding downsampling layers, and because each corresponding layer has a skip connection, segmentation accuracy is improved. U-net supports end-to-end training, requiring no large number of training samples or pre-training. With the development of CNNs, FCNs, and U-net, the accuracy of medical image segmentation has greatly improved, indicating that this field has entered the fourth generation of segmentation algorithms.

[0006] Existing technologies are generally based on 2D-Unet, residual basis neural networks and their improvements, rarely taking into account the three-dimensional distribution of organ segmentation, which leads to the disruption of the continuity between layers and the low Dyson correlation coefficient. 3D-Unet can perfectly solve this problem. Moreover, in order to reduce the occupation of dynamic memory, this application converts the three-dimensional image into a 64*64*64 patch input network to maximize the drawing effect. Summary of the Invention

[0007] The present invention aims to provide an intelligent delineation system and method for cervical cancer clinical target areas and normal organs based on artificial intelligence (deep learning). The technical problems to be solved include at least how to improve accuracy and how to reduce dynamic memory usage and improve efficiency.

[0008] To achieve the above objectives, this invention provides an intelligent delineation system for cervical cancer clinical target areas and normal organs based on artificial intelligence (deep learning), comprising a training unit and a prediction unit. The training unit includes an input module, an image preprocessing module, a 3D-Unet deep learning model construction module, a training network, and an output module. The input module is used to input multimodal images. The image preprocessing module preprocesses the multimodal images input by the input module and inputs the preprocessed images into the 3D-Unet deep learning model construction module. The 3D-Unet deep learning model construction module constructs three different 3D-Unet deep learning models based on three different loss functions. The training network performs multiple cycles on the three different 3D-Unet deep learning models based on the images preprocessed by the image preprocessing module. The training process continues until the offset between the cervical cancer CTV and normal organ results output by the output module and the known true cervical cancer CTV and normal organ results of the multimodal images input by the input module meets a predetermined threshold range, resulting in three different trained 3D-Unet deep learning models. The prediction unit includes an input terminal, a confidence map overlay output module, and an output terminal. The input terminal is used to input new identical multimodal images into the three different trained 3D-Unet deep learning models respectively, obtaining confidence maps for each of the three different trained 3D-Unet deep learning models. The confidence map overlay output module is used to overlay the maximum values ​​of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ results, and output the prediction results through the output terminal.

[0009] Preferably, the preprocessing refers to converting the 3D image into a 64*64*64 patch.

[0010] Preferably, the multimodal images include CT, MR, or PET images.

[0011] Preferably, the three different loss functions include the cross-entropy loss function, the generalized Dessian similarity loss function, and the Tversky loss function.

[0012] Preferably, the cross-entropy loss function is used to calculate the cross-entropy loss between the network prediction and the target value for single-label and multi-label classification tasks. The formula for calculating the cross-entropy loss function is as follows:

[0013]

[0014] Where N is the number of observations, K is the number of classes; T ni This is the result of segmentation of real CTV and normal organs, Y niIt predicts the results of CTV and normal organ segmentation;

[0015] Preferably, the formula for calculating the generalized Dessian similarity loss function is as follows:

[0016]

[0017] Where K is the number of categories, M is the segmentation result Y along the predicted CTV and normal organs. km The number of elements in the first two dimensions, and Wk, are weighting factors specific to each category, controlling the contribution of each category to the result; T km This is the result of segmenting real CTV and normal organs;

[0018] The generalized Dessian similarity loss is based on Sorenson-Dessian similarity and is used to measure the overlap between two segmented images.

[0019] Preferably, the Tversky loss function is calculated using the following formula:

[0020]

[0021] Where 'c' corresponds to class, This corresponds to something that is not in class C;

[0022] Y cm It predicts CTV and normal organ segmentation results, T cm This is the result of segmenting real CTV and normal organs;

[0023] M is the segmentation result along the predicted CTV and normal organ. cm The number of elements in the first two dimensions;

[0024] α is a weighting factor that controls the contribution of false positive loss to each class;

[0025] β is a weighting factor that controls the contribution of false negatives to the loss for each class;

[0026] The Tversky loss function is based on the Tversky exponent and is used to measure the overlap between two segmented images.

[0027] This invention also provides a method for automatically delineating the clinical target area of ​​cervical cancer based on deep learning, comprising the following steps:

[0028] S1. Input multimodal images such as CT, MR, and PET;

[0029] S2. Preprocess the input multimodal image;

[0030] S3. Construct three different 3D-Unet deep learning models based on three different loss functions;

[0031] S4. Train the three different 3D-Unet deep learning models for multiple cycles based on the preprocessed image until the offset between the output cervical cancer CTV and normal organ segmentation results and the known true cervical cancer CTV and normal organ segmentation results of the input multimodal image meets the predetermined threshold range, thus obtaining three different trained 3D-Unet deep learning models.

[0032] S5. Input new identical multimodal images such as CT, MR, and PET into three different trained 3D-Unet deep learning models respectively to obtain the confidence maps of the three different trained 3D-Unet deep learning models.

[0033] S6. Superimpose the maximum values ​​of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ segmentation results, and output the prediction results through the output terminal.

[0034] Beneficial effects

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] 1. The intelligent delineation system and method for cervical cancer clinical target area and normal organ segmentation based on deep learning described in this invention is based on the application of 3D-Unet for multimodal images. Compared with the previous two-dimensional segmentation network for single-modal images, it improves the correlation between layers and increases the accuracy. Moreover, the patch-based design reduces dynamic memory usage and improves efficiency.

[0037] 2. This invention combines multiple loss functions to obtain multiple training models, and finally integrates the segmentation results based on the confidence map, which improves accuracy compared with the design of a single loss function. Attached Figure Description

[0038] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the specific embodiments of this application to explain the technical solutions of the present invention and do not constitute a limitation on the technical solutions of the present invention.

[0039] Figure 1 This is a schematic diagram illustrating the principle of the intelligent delineation system for cervical cancer clinical target areas based on deep learning, as described in this invention.

[0040] Figure 2 This is a schematic diagram of the process of obtaining the final segmentation result image. Detailed Implementation

[0041] The invention is described in more detail below to aid in understanding it.

[0042] like Figure 1 As shown, the intelligent delineation system for cervical cancer clinical target area and normal organ segmentation based on deep learning described in this invention includes a training unit and a prediction unit. The training unit includes an input module, an image preprocessing module, a 3D-Unet deep learning model construction module, a training network, and an output module. The input module is used to input multimodal images such as CT, MR, and PET. The image preprocessing module preprocesses the multimodal images input by the input module and inputs the preprocessed images into the 3D-Unet deep learning model construction module. The 3D-Unet deep learning model construction module constructs three different 3D-Unet deep learning models based on three different loss functions. The training network trains the three different 3D-Unet deep learning models for multiple cycles based on the images preprocessed by the image preprocessing module. Three different trained 3D-Unet deep learning models are obtained when the offset between the cervical cancer CTV result output by the output module and the known true cervical cancer CTV and normal organ segmentation results of the multimodal images input by the input module meets a predetermined threshold range. The prediction unit includes an input end, a confidence map overlay output module, and an output end. The input end is used to input new identical multimodal images such as CT, MR, and PET into the three different trained 3D-Unet deep learning models respectively, and obtain the confidence maps of the three different trained 3D-Unet deep learning models. The confidence map overlay output module is used to overlay the maximum points of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ segmentation results, and output the prediction results through the output end.

[0043] Preferably, the preprocessing refers to converting the 3D image into 64*64*64 image patches.

[0044] The criteria for determining the optimal threshold are derived by combining comprehensive indicators such as segmentation accuracy, false positive rate, and false negative rate. For example, the confidence plot values ​​are distributed between 0 and 1. By calculating the relevant indicators at five thresholds: 0.9, 0.8, 0.7, 0.6, and 0.5, confidence levels below 0.5 are considered unreliable and not selected. Then, curves are plotted to find the optimal threshold. This optimal threshold achieves the best control, maximizing segmentation accuracy and minimizing false positives and false negatives. The prediction results obtained with this optimal threshold have better overall performance than those obtained with non-optimal thresholds. The relationship between the segmentation result and the optimal threshold is as follows: Figure 2As shown. A patch can be simply understood as an image block. When the resolution of the image to be processed is too large and resources are limited (such as video memory, computing power, etc.), the image can be divided into small blocks, and these small image blocks are called patches.

[0045] To improve accuracy, this invention employs three loss functions, including:

[0046] Loss Function 1: Cross-entropy loss function, used to calculate the cross-entropy loss between the network prediction and the target value for single-label and multi-label classification tasks. The formula for calculating the cross-entropy loss function is:

[0047]

[0048] Where N is the number of observations, K is the number of classes; T ni This is the result of segmentation of real CTV and normal organs, Y ni It predicts the CTV and normal organ segmentation results.

[0049] Loss Function 2: Generalized Dessian Similarity Loss, based on Sorenson-Dessian similarity, is used to measure the overlap between two segmented images. The formula for calculating the Generalized Dessian Similarity Loss function is as follows:

[0050]

[0051] Where K is the number of categories, M is the segmentation result along the predicted Y. km The number of elements in the first two dimensions, and W k These are category-specific weights that control the contribution of each category to the results. These weights help offset the influence of larger regions on the Dessian similarity coefficient. T km It is the result of segmentation of real CTV and normal organs.

[0052] Loss Function 3: Tversky Loss Function (Tversky Loss Function) is based on the Tversky exponent and is used to measure the overlap between two segmented images. The formula for calculating the Tversky loss function is:

[0053]

[0054] Where 'c' corresponds to class, This corresponds to something that is not in class C;

[0055] Y cm It predicts CTV and normal organ segmentation results, T cm This is the result of segmenting real CTV and normal organs;

[0056] M is the segmentation result along the predicted CTV and normal organ. cm The number of elements in the first two dimensions;

[0057] α is a weighting factor that controls the contribution of false positive loss to each class;

[0058] β is a weighting factor that controls the contribution of false negatives to the loss for each class.

[0059] This invention also provides a method for automatically delineating the clinical target area of ​​cervical cancer and normal organs based on deep learning, comprising the following steps:

[0060] S1. Input multimodal images such as CT, MR, and PET;

[0061] S2. Preprocess the input multimodal image;

[0062] S3. Construct three different 3D-Unet deep learning models based on three different loss functions;

[0063] S4. Train the three different 3D-Unet deep learning models for multiple cycles based on the preprocessed image until the offset between the output cervical cancer CTV and normal organ segmentation results and the known true cervical cancer CTV and normal organ segmentation results of the input multimodal image meets the predetermined threshold range, thus obtaining three different trained 3D-Unet deep learning models.

[0064] S5. Input new identical multimodal images such as CT, MR, and PET into three different trained 3D-Unet deep learning models respectively to obtain the confidence maps of the three different trained 3D-Unet deep learning models.

[0065] S6. Superimpose the maximum values ​​of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ segmentation results, and output the prediction results through the output terminal.

[0066] The key points and advantages of this invention include:

[0067] 1. The application of 3D-Unet based on multimodal images improves the correlation between layers and increases accuracy compared to the previous single-modal image 2D segmentation network. Moreover, the patch-based design reduces dynamic memory usage and improves efficiency.

[0068] 2. By combining multiple loss functions to obtain multiple training models, and finally integrating the segmentation results based on the confidence map, the accuracy is improved compared with the design of a single loss function.

[0069] The key technical points of this invention include at least:

[0070] 1. Design of 3D-Unet based on multimodal images and multiple loss functions.

[0071] 2. The design combines multiple loss functions to obtain multiple training models, and finally integrates the segmentation results based on the confidence map threshold.

[0072] The preferred embodiments of the present invention have been described above, but are not intended to limit the invention. Those skilled in the art can make modifications and variations to the embodiments disclosed herein without departing from the scope and spirit of the invention.

Claims

1. An intelligent contouring system for clinical target volume and normal organs in cervical cancer, characterized in that, The intelligent delineation system for cervical cancer clinical target areas and normal organs includes a training unit and a prediction unit. The training unit includes an input module, an image preprocessing module, a 3D-Unet deep learning model construction module, a training network, and an output module. The input module is used to input multimodal images. The image preprocessing module preprocesses the multimodal images input by the input module and inputs the preprocessed images into the 3D-Unet deep learning model construction module. The 3D-Unet deep learning model construction module constructs three different 3D-Unet deep learning models based on three different loss functions. The training network trains the three different 3D-Unet deep learning models for multiple cycles based on the images preprocessed by the image preprocessing module until the output module outputs the results. When the offset between the cervical cancer CTV and normal organ segmentation results and the known true cervical cancer CTV and normal organ segmentation results of the multimodal image input by the input module meets a predetermined threshold range, three different trained 3D-Unet deep learning models are obtained. The prediction unit includes an input end, a confidence map overlay output module, and an output end. The input end is used to input new identical multimodal images into the three different trained 3D-Unet deep learning models respectively, and obtain the confidence maps of the three different trained 3D-Unet deep learning models. The confidence map overlay output module is used to overlay the maximum points of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ segmentation results, and output the prediction results through the output end. The three different loss functions mentioned include the cross-entropy loss function, the generalized Dessian similarity loss function, and the Tversky loss function; The cross-entropy loss function is used to calculate the cross-entropy loss between the network prediction and the target value for single-label and multi-label classification tasks; The formula for calculating the cross-entropy loss function is as follows: ； where N is the observation, K is the number of classes; T ni is the true CTV and normal organ segmentation results, Y ni is the predicted CTV and normal organ segmentation results; The formula for calculating the generalized Dessian similarity loss function is as follows: ； where K is the number of categories, M is the number of elements along the first two dimensions of the predicted CTV and normal organ outcomes Y km and Wkis a weight factor specific to each category, controlling the contribution of each category to the outcome; T km are true CTV and normal organ segmentation results; The generalized Dessian similarity loss, based on Sorenson-Dessian similarity, is used to measure the overlap between two segmented images; The formula for calculating the Tversky loss function is as follows: ； Where 'c' corresponds to class, c corresponds to not being in class c; Y cm is the predicted CTV and normal organ segmentation result, T cm is the true CTV and normal organ segmentation result; M is the number of elements along the first two dimensions of the predicted CTV and normal organ outcomes Y cm Y α is a weighting factor that controls the contribution of false positive loss to each class; β is a weighting factor that controls the contribution of false negatives to the loss for each class; The Tversky loss function is based on the Tversky exponent and is used to measure the overlap between two segmented images.

2. The intelligent delineation system for cervical cancer clinical target areas and normal organs according to claim 1, characterized in that, The preprocessing mentioned refers to converting the 3D image into... Image blocks.

3. The intelligent delineation system for cervical cancer clinical target areas and normal organs according to claim 1, characterized in that, The multimodal images mentioned include CT, MR, or PET images.

4. An automatic delineation method for an intelligent delineation system for cervical cancer clinical target areas and normal organs according to any one of claims 1 to 3, characterized in that, Includes the following steps: S1. Input multimodal images such as CT, MR, and PET; S2. Preprocess the input multimodal image; S3. Construct three different 3D-Unet deep learning models based on three different loss functions; S4. Train the three different 3D-Unet deep learning models for multiple cycles based on the preprocessed image until the offset between the output cervical cancer CTV and normal organ segmentation results and the known true cervical cancer CTV and normal organ segmentation results of the input multimodal image meets the predetermined threshold range, thus obtaining three different trained 3D-Unet deep learning models. S5. Input new identical multimodal images such as CT, MR, and PET into three different trained 3D-Unet deep learning models respectively to obtain the confidence maps of the three different trained 3D-Unet deep learning models. S6. Superimpose the maximum values ​​of the confidence maps of the three different trained 3D-Unet deep learning models, take the optimal threshold, predict the cervical cancer CTV and normal organ segmentation results, and output the prediction results through the output terminal.

5. The automatic outlining method according to claim 4, characterized in that, The preprocessing mentioned refers to converting the 3D image into... Image blocks.

6. The automatic outlining method according to claim 4, characterized in that, The multimodal images mentioned include CT, MR, or PET images.

Images