Multi-modal, multi-resolution deep learning neural networks for segmentation, outcomes prediction and longitudinal response monitoring to immunotherapy and radiotherapy

Abstract

Systems and methods for multi-modal, multi-resolution deep learning neural networks for segmentation, outcomes prediction and longitudinal response monitoring to immunotherapy and radiotherapy are detailed herein. A structure-specific Generational Adversarial Network (SSGAN) is used to synthesize realistic and structure-preserving images not produced using state-of-the art GANs and simultaneously incorporate constraints to produce synthetic images. A deeply supervised, Multi-modality, Multi-Resolution Residual Networks (DeepMMRRN) for tumor and organs-at-risk (OAR) segmentation may be used for tumor and OAR segmentation. The DeepMMRRN may combine multiple modalities for tumor and OAR segmentation. Accurate segmentation is may be realized by maximizing network capacity by simultaneously using features at multiple scales and resolutions and feature selection through deep supervision. DeepMMRRN Radiomics may be used for predicting and longitudinal monitoring response to immunotherapy. Auto-segmentations may be combined with radiomics analysis for predicting response prior to treatment initiation. Quantification of entire tumor burden may be used for automatic response assessment.

Description

[0001] Cross References to Related Applications [0002] The application, filed on 30 July 2018 under the requirements of PCT Article 8, is entitled "Multimodal, multiresolutional deep learning neural networks for segmentation, outcome prediction and longitudinal response monitoring of immunotherapy and radiotherapy ( MULTI-MODAL, MULTI-RESOLUTION DEEP LEARNING NEURAL NETWORKS FOR SEGMENTATION, OUTCOMES PREDICTION AND LONGITUDINAL RESPONSE MONITORING TO IMMUNOTHERAPY ANDRADIOTHERAPY), which is hereby incorporated by reference in its entirety. technical field [0003] The present disclosure generally relates to systems and methods for image segmentation. Background technique [0004] For some image classification applications, such as those related to the military or medical fields, accurate and reliable segmentation of images can be very important and can often be performed by specially trained personnel. It is difficult to accurately and reliably segment various objects w...

AI Technical Summary

Problems solved by technology

It is difficult to accurately and reliably segment various objects within an image in an automated manner

Images