Systems and methods for rapid neural network-based image segmentation and radiopharmaceutical uptake determination

Abstract

To provide systems and methods for rapid neural network-based image segmentation and radiopharmaceutical uptake determination.SOLUTION: Presented herein are systems and methods that provide for automated analysis of three-dimensional (3D) medical images of a subject in order to automatically identify specific 3D volumes within the 3D images that correspond to specific organs and / or tissue. In certain embodiments, the accurate identification of one or more such volumes can be used to determine quantitative metrics that measure uptake of radiopharmaceuticals in particular organs and / or tissue regions. These uptake metrics can be used to assess disease state in a subject, determine a prognosis for a subject, and / or determine efficacy of a treatment modality.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 614,935, filed January 8, 2018, U.S. Patent Application No. 16 / 003,006, filed June 7, 2018, and U.S. Provisional Application No. 62 / 749,574, filed October 23, 2018, the contents of each of which are incorporated herein by reference in their entirety.

[0002] The present invention relates generally to methods, systems, and architectures for the automated analysis and / or representation of medical image data. More specifically, in certain embodiments, the present invention relates to the automated identification of one or more specific regions of interest (e.g., corresponding to particular organs or tissues) within an image of a subject and the determination of radiopharmaceutical uptake within such regions, e.g., for the identification and / or disease classification of diseases such as, for example, prostate cancer. [Background technology]

[0003] Targeted imaging involves the use of radiolabeled small molecules that bind to specific receptors, enzymes, and proteins in the body that are altered during the progression of disease. After being administered to a patient, these molecules circulate in the blood until they find their intended target. The bound radiopharmaceutical remains at the site of the disease, while the remainder of the active drug clears from the body. The radioactive portion of the molecule acts as a beacon to obtain images depicting the location and accumulation of disease using commonly available nuclear medicine cameras, known as single-photon emission computed tomography (SPECT) cameras or positron emission tomography (PET) cameras, found in most hospitals around the world. Physicians can then use this information to determine the presence and extent of disease in patients. Physicians can use this information to recommend a course of treatment for patients and to track disease progression.

[0004] There are various software-based analysis techniques available for the analysis and enhancement of PET and SPECT images that can be used by radiologists or physicians. There are also several radiopharmaceuticals available for imaging specific types of cancer. For example, the small molecule diagnostic 1404 targets the extracellular domain of prostate-specific membrane antigen (PSMA), a protein found on the surface of over 95% of prostate cancer cells and a validated target for detecting primary and metastatic prostate cancer. 1404 is labeled with technetium-99m, a gamma-radioactive isotope with spectral properties favorable for nuclear medicine imaging applications that is relatively low-cost, widely available, and easy to efficiently prepare.

[0005] Another example of a radiopharmaceutical is PyL™ ([ 18 F]DCFPyL), which is a fluorinated PSMA-targeted PET imaging agent for prostate cancer in clinical disease. Journal of Molecular Imaging A proof-of-concept study published in the April 2015 issue of Cancer Biology demonstrated that PET imaging with PyL™ showed high levels of PyL™ uptake at sites of presumed metastatic disease and primary tumors, suggesting potentially high sensitivity and specificity in detecting prostate cancer.

[0006] An oncologist may use images from a patient's targeted PET or SPECT study as input in assessing whether the patient has a particular disease, such as prostate cancer, what the apparent stage of the disease is, what the recommended course of treatment (if any) is, whether surgical intervention is indicated, and the likely prognosis. The oncologist may use a radiologist's report in this assessment. A radiologist's report is a technical evaluation of the PET or SPECT images prepared by a radiologist for the physician who requested the imaging study, including, for example, the type of study performed, the medical history, comparisons between images, the technique used to perform the study, the radiologist's observations and findings, and any overall impression and recommendations the radiologist may have based on the results of the imaging study. The radiologist's signed report is sent to the physician who ordered the study for medical review, followed by a discussion between the physician and the patient regarding the results and recommendations for treatment.

[0007] Thus, the process involves a radiologist performing an imaging exam on a patient, analyzing the resulting images, generating a radiologist report, forwarding the report to a requesting physician, the physician formulating an assessment and treatment recommendation, and the physician communicating the results, recommendations, and risks to the patient. The process may also include repeating the imaging exam for inconclusive results or ordering further testing based on initial results.

[0008] When imaging tests indicate that a patient has a particular disease or condition (e.g., cancer), doctors consider various treatment options, including surgery and the risks of doing nothing without surgery or adopting a watchful waiting or active surveillance approach.

[0009] There are limitations associated with this process from both the physician's and patient's perspectives. While radiologist reports are certainly helpful, physicians must ultimately rely on their own experience to formulate assessments and recommendations for their patients. Furthermore, patients must place a great deal of trust in their physicians. Although physicians may show patients PET / SPECT images and tell them the numerical risks or possible specific prognoses associated with various treatment options, patients will likely struggle to understand the meaning of this information. Furthermore, the patient's family will likely have questions, especially if the patient chooses not to undergo surgery despite a cancer diagnosis. Patients and / or their families may search online for additional information and misunderstand the risks of the diagnosed condition. This ordeal can become even more traumatic. Summary of the Invention [Problem to be solved by the invention]

[0010] Thus, there remains a need for systems and methods for improving the analysis of medical imaging tests and communicating their results, diagnoses, prognoses, treatment recommendations, and associated risks to patients. [Means for solving the problem]

[0011] The systems and methods presented herein provide for automated analysis of three-dimensional (3D) medical images of a subject to automatically identify specific 3D volumes within the 3D images that correspond to specific organs and / or tissues. In certain embodiments, accurate identification of one or more such volumes is used to automatically determine quantitative metrics representative of the uptake of a radiopharmaceutical in specific organs and / or tissue regions. These uptake metrics may be used to assess the subject's medical condition, determine the subject's prognosis, and / or determine the efficacy of a therapy.

[0012] The systems and methods described herein pave the way for streamlined medical image analysis workflows with improvements in accuracy, consistency, and reproducibility of results, particularly by identifying 3D volumes in medical images and automatically determining uptake metrics. For example, embodiments of the image analysis techniques described herein can be used to perform an initial, fully automated assessment of a patient's cancer status. A physician then reviews the fully automated analysis and chooses to (i) accept the automated assessment or (ii) semi-automatically adjust parameters under the guidance of the image analysis system. This approach reduces the burden on physicians and / or supporting technicians to prepare and review images for final diagnosis compared to conventional approaches that perform neither automatic 3D segmentation nor initial status assessment. In these conventional approaches, physicians and / or technicians must instead painstakingly scroll through 3D images slice by slice to manually identify two-dimensional regions of interest. In addition to being time-consuming, such conventional approaches are heavily based on the subjective judgment of the image reviewer (e.g., physician and / or technician) and are therefore prone to inter-viewer and / or intra-viewer variability. In contrast, results based on the image analysis workflows, systems and methods described herein are obtained in a fully or semi-automated manner that completely eliminates or dramatically reduces subjective variability.

[0013] For example, the systems and methods described herein may involve administering a radionuclide-labeled PSMA binding agent (e.g., 99m Tc-MIP-1404, for example [ 18 The present invention can be used for automated analysis of medical images to determine uptake metrics that provide quantitative measures of uptake of radiopharmaceuticals such as [F]DCFPyL. Such uptake metrics are relevant to assessing a patient's risk for prostate cancer and / or the severity / stage of prostate cancer within a subject. For example, it has been discovered that high sensitivity and specificity can be achieved for automated classification of clinically significant versus clinically insignificant prostate cancer.

[0014] The image analysis techniques described herein, in certain embodiments, utilize a combination of 3D anatomical and functional images obtained of a subject. Anatomical images, such as X-ray computed tomography (CT) images, provide detailed anatomical / structural information. Functional images convey information about the internal physiological activity of specific organs and / or tissues, such as metabolism, blood flow, local chemical composition, and / or absorption. Of particular relevance are nuclear medicine images, such as single-photon emission computed tomography (SPECT) images and / or positron emission tomography (PET) images, which are obtained by detecting radiation emitted from a subject and can be used to infer the spatial distribution of administered radiopharmaceuticals within the subject.

[0015] For example, SPECT imaging uses radiopharmaceuticals 99m Tc-MIP-1404( 99m In certain embodiments, the uptake of Tc-labeled 1404 in the prostate of a subject can be assessed. 99m To assess the uptake of Tc-MIP-1404, a CT image of the subject and a corresponding SPECT image are obtained so that anatomical / structural information in the CT image can be correlated with functional information in the corresponding SPECT image. Often, the CT image and the SPECT image are acquired using a single multimodal imaging system in two separate scans (e.g., a first scan for the CT image and a second scan for the SPECT image), with the subject in a substantially fixed position over the two scans. In this way, a mapping between voxels in the CT image and voxels in the SPECT image is established, and volumes within the CT image identified as corresponding to specific organs and / or tissue regions can be used to identify voxels within the SPECT image corresponding to the same specific organs and / or tissue regions.

[0016] Thus, the image analysis techniques described herein, in certain embodiments, utilize a convolutional neural network (CNN) to accurately identify a prostate volume within a CT image that corresponds to the subject's prostate gland. The identified prostate volume can be used to identify voxels in a SPECT image that also correspond to the subject's prostate gland. Thus, the presence of a contrast agent (e.g., a labeled PSMA binding agent, e.g., 99m Tc-MIP-1404 or [ 18 An uptake metric, providing a measure of uptake of [F]DCFPyL), can be calculated using the intensities of SPECT image voxels corresponding to the subject's prostate. The uptake metric can then be converted to identify whether the subject has prostate cancer, and / or to quantify the subject's risk of having prostate cancer, and / or to disease classification (e.g., as part of tracking the disease over time), which can be used by a physician, for example, to advise treatment options and / or to monitor the efficacy of a given therapy.

[0017] Various advances are described herein that improve the performance of automated measurement of uptake metrics in the prostate (or, more broadly, specific organs or tissue regions of interest) using this multi-image approach, i.e., analyzing 3D anatomical images in combination with 3D functional images. These advances include the automatic identification (e.g., using a first convolutional neural network (CNN)) of bounding boxes within the 3D anatomical images, for example, to identify the pelvic region in which the prostate is located.

[0018] For example, a set of 3D anatomical images with identified physiological functions (e.g., identified pelvic regions) is used to train the first CNN on points representing the boundaries of the pelvic region (e.g., vertices of a rectangular bounding box) so that the first CNN can be used to automatically identify the pelvic region in the 3D anatomical image of the subject. This results in an initial volumetric region of a more standard size of the 3D anatomical image, where, for example, the bounding box indicates the boundaries of the region of the image corresponding to the subject's prostate and / or bladder and / or rectum and / or gluteal muscles, which is then processed (e.g., by a second CNN) for detailed segmentation of the region of interest within the initial volumetric region. Contrast agent uptake metrics can then be determined from portions of the 3D functional image that map to one or more of the identified regions of the 3D anatomical image. Note that a "bounding box," as used herein, is not necessarily a rectangular parallelepiped and may have other shapes. In a specific embodiment, the bounding box is a rectangular parallelepiped.

[0019] The first CNN's determination of the bounding box may use as input a significantly lower resolution than that used by the second CNN for segmenting the prostate and / or other organs within the bounding box. For example, to find the bounding box, the first CNN may process a 3D anatomical image of the whole body having a first number of voxels (e.g., 81×68×96 voxels), and then the second CNN may process a higher resolution image having more voxels than the first number of voxels (e.g., 94×138×253 voxels) that corresponds to the bounding box region.

[0020] For example, a "bounding box" approach that identifies one or more portions of the 3D anatomical image that are relevant to the analysis at hand before applying a second CNN (for fine segmentation) improves computational efficiency by removing large portions of the initial 3D anatomical image before more computationally intensive subsequent processing. This approach is computationally more efficient than performing fine segmentation on the entire initial 3D anatomical image because, for example, identifying the pelvic region (e.g., the vertices of a rectangular bounding box) is easier than fine segmentation of the prostate, bladder, and / or other tissues of interest. Not only is this approach more computationally efficient, it also results in more accurate subsequent processing, such as the finer segmentation provided by the second CNN. This is because, for example, 3D anatomical images obtained using different machines at different medical institutions will vary in size (e.g., different sizes mean that there will be a different number of image voxels and / or different volumes of the patient's tissue represented in the image), and for automatic detailed segmentation of the prostate, training the second CNN using portions of 3D anatomical training images with more standardized image volume sizes that contain the organ and other tissue regions of interest will result in more robust and accurate segmentation.

[0021] Another advancement described herein that improves the performance of this multi-image approach (i.e., analyzing 3D anatomical images in combination with 3D functional images) for automated measurement of uptake metrics in the prostate (or, more broadly, specific organs or tissue regions of interest) involves accurately identifying one or more tissue regions in addition to the prostate and accounting for contrast agent uptake in those regions in determining (i) uptake metrics in the prostate and / or (ii) prostate cancer identification and / or disease classification. Certain contrast agents, including PSMA-binding agents, have high uptake in certain organs, which can affect the identification of diseased tissue (e.g., prostate cancer). For example, uptake of a radionuclide-labeled PSMA-binding agent by the bladder can result in scattering in the 3D functional images, reducing the accuracy of the contrast agent intensity measured in the prostate gland, which is located near the bladder. Training a second CNN for detailed segmentation of both the prostate and bladder of interest allows for automatic and accurate accounting of “bleed-through” or “crosstalk” effects and / or other effects resulting from contrast agent uptake by the bladder. Furthermore, by training a second CNN to identify reference regions within the 3D anatomical image, such as the gluteus muscle, it is possible to more accurately weight / normalize the contrast agent intensity measurements, improving the accuracy and diagnostic value of uptake measurements in the subject's prostate.

[0022] Thus, the systems and methods described herein utilize a unique combination of two CNN modules in certain embodiments, where a first CNN module identifies an initial volume of interest (VOI) within a CT image, and a second CNN module receives the VOI as input and identifies the prostate volume therein. As described herein, this approach allows the second CNN module to operate on a smaller input size (e.g., the VOI as opposed to the entire CT image). The computational resource (e.g., memory, e.g., processing time) savings from reducing the input size in this manner can be allocated to improving the accuracy of the second CNN module and / or used to improve the speed of the image processing approach.

[0023] In certain embodiments, the systems and methods described herein identify various other tissue volumes within a CT image along with the prostate. For example, in addition to the prostate, other tissue volumes corresponding to the subject's pelvic bone, bladder, rectum, and gluteal muscles may be identified. As described herein, the identification of such other tissue volumes may be used for various functions and provides advantages over other approaches, such as binary classification approaches, such as whether a voxel in a CT image is identified as corresponding to the prostate. In particular, the identification of other tissue volumes may, for example, (i) improve the accuracy with which a CNN module identifies the prostate volume within a CT image, (ii) provide identification of a reference region that can be used to calculate a normalization value for calculating an uptake metric, and (iii) allow the intensity of SPECT image voxels corresponding to the prostate to be corrected for crosstalk due to, for example, the accumulation of a radiopharmaceutical within the bladder.

[0024] The image analysis techniques described herein may be used in certain embodiments to analyze various anatomical and functional images, and are not limited to CT and SPECT images. For example, positron emission tomography (PET) is another functional imaging modality that provides information about the distribution of radiopharmaceuticals within a subject. PET images, like SPECT images, can be used in combination with CT images to determine uptake metrics for various organs and tissue regions of interest. The techniques described herein may also be applied to various organs and / or tissue regions of interest, such as bone, lymph nodes, liver, and lungs.

[0025] Thus, the systems and methods described herein provide for rapid and accurate identification of specific organs and tissue regions within a medical image, thereby providing accurate, automated determination of uptake metrics that provide quantitative measures of radiopharmaceutical uptake within various organs and tissue regions within a subject. The automatically determined uptake metrics provide a valuable tool for assessing a patient's internal disease risk, status, and progression, as well as treatment efficacy.

[0026] In one aspect, the present invention is directed to a method for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a prostate gland of a subject and to determine one or more uptake metrics indicative of an uptake of a radiopharmaceutical therein (i.e., in the prostate gland), the method comprising the steps of: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT) (e.g., whole-body CT image, e.g., regional CT image of the body), e.g., magnetic resonance imaging (MRI), e.g., 3D ultrasound), wherein at least a portion of a graphical representation of tissue (e.g., soft tissue and / or bone) within the subject included in the 3D anatomical image corresponds to a pelvic region of the subject; and (b) receiving, by the processor, a 3D functional image of the subject obtained using a functional imaging modality (e.g., single-photon emission computed tomography (SPECT), e.g., positron emission tomography (PET)), wherein each of a plurality of voxels included in the 3D functional image corresponds to a specific physical volume within the subject. (c) using a first module (e.g., a first machine learning module) to determine an initial volume of interest (VOI) within the 3D anatomical image (e.g., a parallelepiped, e.g., a rectangular parallelepiped), the initial VOI corresponding to tissue within the pelvic region of the subject and excluding tissue outside the pelvic region of the subject (e.g., the excluded voxels of the 3D anatomical image are greater than the included voxels, e.g., less than 25% of the included voxels of the 3D anatomical image, e.g., a majority of the voxels within the VOI represent a physical volume within the pelvic region of the subject); (d) using a second module (e.g., a second machine learning module) to identify a prostate volume within the initial VOI corresponding to the prostate of the subject; and (e) using a processor to determine an initial volume of interest (VOI) within the 3D anatomical image (e.g., a parallelepiped, e.g., a rectangular parallelepiped), the initial VOI corresponding to tissue within the pelvic region of the subject and excluding tissue outside the pelvic region of the subject (e.g., the excluded voxels of the 3D anatomical image are greater than the included voxels, e.g., less than 25% of the included voxels of the 3D anatomical image, e.g., a majority of the voxels within the VOI represent a physical volume within the pelvic region of the subject).and determining (e.g., determining and displaying) one or more uptake metrics using the 3D functional image and the prostate volume identified within the initial VOI of the 3D anatomical image (e.g., calculating the amount of radiopharmaceutical in the subject's prostate gland based on intensity values ​​of voxels of the 3D functional image corresponding to the prostate volume identified within the initial VOI of the 3D anatomical image, e.g., calculating the sum (e.g., weighted sum), average, and / or maximum of intensities of voxels of the 3D functional image representing the physical volume occupied by the subject's prostate gland) (e.g., the one or more uptake metrics include a tumor to background ratio (TBR) value, and / or the method includes determining (e.g., determining and displaying) a classification status of the prostate cancer as either (i) clinically significant or (ii) clinically insignificant based at least in part on the TBR value).

[0027] In one particular embodiment, a first module receives as input a 3D anatomical image and outputs a plurality of coordinate values ​​representing opposite corners of a rectangular volume within the 3D anatomical image (e.g., two sets of coordinate values ​​representing opposite corners of a rectangular volume).

[0028] In certain embodiments, step (c) includes using a first module to determine a 3D pelvic bone mask that identifies a volume of the 3D anatomical image that corresponds to the pelvic bones of the subject (e.g., one or more (all) of the sacrum, coccyx, left hip bone, and right hip bone).

[0029] In one particular embodiment, the first module is a convolutional neural network (CNN) module (e.g., a neural network module that utilizes one or more convolutional layers).

[0030] In certain embodiments, step (d) includes using a second module to identify one or more other tissue volumes within the 3D anatomical image, each of which corresponds to a specific tissue region within the subject, wherein the one or more other tissue volumes correspond to one or more specific tissue regions selected from the group consisting of the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hip bone, e.g., right hip bone), the subject's bladder, the subject's rectum, and the subject's gluteal muscles (e.g., left gluteal muscle, e.g., right gluteal muscle).

[0031] In certain embodiments, step (d) includes using a second module to classify each voxel within the initial VOI as corresponding to a particular tissue region from a set of (predetermined) distinct tissue regions within the subject {e.g., the set includes the prostate and, optionally, one or more other tissue regions (e.g., the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hipbone, e.g., right hipbone), the subject's bladder, the subject's rectum, and gluteal muscles (e.g., left gluteal muscle, e.g., right gluteal muscle)}. In certain embodiments, classifying each voxel within the initial VOI as corresponding to a particular tissue region from a set of (predetermined) distinct tissue regions within the subject {e.g., the set includes the prostate and, optionally, one or more other tissue regions (e.g., the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hipbone, e.g., right hipbone), the subject's bladder, the subject's rectum, and gluteal muscles (e.g., left gluteal muscle, e.g., right gluteal muscle)}. The step of determining includes a step of determining, by a second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, the set of likelihood values ​​including, for each of one or more tissue regions of the set of tissue regions, a corresponding likelihood value representing the likelihood (e.g., calculated by the second module) that the voxel represents a physical volume within the tissue region; and a step of classifying, for each of the plurality of voxels within the initial VOI, the voxel as corresponding to a particular tissue region based on the set of likelihood values ​​determined for the voxel.In a particular embodiment, the second module receives as input an initial VOI (e.g., the entire initial VOI) and generates, for each voxel within the initial VOI, (i) a value classifying the voxel (e.g., a value classifying the voxel as corresponding to a particular tissue region, e.g., a region selected from a predetermined set of different tissue regions (e.g., the subject's prostate), e.g., a pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hipbone, e.g., right hipbone), the subject's bladder, the subject's rectum, and the subject's gluteal muscles (e.g., left gluteal muscles, e.g., right gluteal muscles)); and (ii) a set of likelihood values ​​for the voxel (e.g., a value classifying the voxel as corresponding to a particular tissue region, e.g., a region selected from a predetermined set of different tissue regions (e.g., the subject's prostate), e.g., a pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hipbone, e.g., right hipbone), the subject's bladder, the subject's rectum, and the subject's gluteal muscles (e.g., left gluteal muscles, e.g., right gluteal muscles)). and (iii) a value identifying the voxel as not corresponding to any of the predetermined set of different tissue regions (e.g., a value identifying as unlikely to correspond or a value identifying the likelihood that the voxel does not correspond) (e.g., a value identifying the voxel as, for example, corresponding to a non-diagnostic background area or likely to correspond to a background area, or a value identifying the likelihood that the voxel corresponds to a background area) (e.g., such that the second module classifies and / or calculates likelihood values ​​for the entire VOI at once, as opposed to operating on each voxel one at a time). In certain embodiments, the (predetermined) set of different tissue regions includes one or more tissue regions selected from the group consisting of the subject's prostate gland, the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hip bone, e.g., right hip bone), the subject's bladder, the subject's rectum, and the subject's gluteal muscles (e.g., left gluteal muscle, e.g., right gluteal muscle).

[0032] In certain embodiments, step (d) includes using a second module to identify a set of one or more underlying tissue volumes including the identified prostate volume and one or more other tissue volumes, and the method further includes using one or more auxiliary modules (e.g., an auxiliary machine learning module) to identify, by the processor, one or more auxiliary tissue volumes within the 3D anatomical image, each of which corresponds to an underlying tissue volume identified by the second module {e.g., the same specific tissue region (e.g., the subject's prostate, the subject's pelvic bone (e.g., the sacrum, e.g., the coccyx, e.g., the left hipbone, e.g., the right hipbone), the subject's bladder, the subject's rectum, and the gluteal muscles (e.g., the left gluteal muscle, e.g., the right gluteal muscle)) is represented as this underlying tissue volume}, and merging, by the processor, each auxiliary tissue volume with the corresponding underlying tissue volume identified by the second module (e.g., augmenting the corresponding underlying tissue volume by incorporating a portion of the corresponding auxiliary tissue volume that was not included in the original underlying tissue volume).

[0033] In certain embodiments, the method includes identifying, by a processor (e.g., using a second module), a reference volume within the 3D anatomical image (e.g., within an initial VOI) that corresponds to a reference tissue region within the subject (e.g., a gluteus muscle), and in step (e) determining at least one of one or more uptake metrics using the 3D functional image and the reference volume identified within the 3D anatomical image (e.g., calculating a normalization value based on the intensity values ​​of voxels of the 3D functional image that correspond to the reference volume identified within the 3D anatomical image). In certain embodiments, at least one of the one or more uptake metrics determined using the 3D functional image and the reference volume includes a tumor-to-background ratio (TBR) value, and determining the TBR value includes: determining a target intensity value (e.g., the target intensity value is the maximum value of the intensities of the voxels in the 3D functional image corresponding to the prostate volume identified within the initial VOI in the 3D anatomical image) using intensity values ​​of one or more voxels in the 3D functional image corresponding to the reference volume identified within the 3D anatomical image; determining a background intensity value (e.g., the background intensity value is the average intensity of multiple (e.g., all) voxels in the 3D functional image corresponding to the identified reference volume); and determining the ratio of the target intensity value to the background intensity value as the TBR value. In certain embodiments, this method includes determining the prostate cancer status of the subject based on the TBR value compared to one or more thresholds (e.g., predetermined thresholds).In certain embodiments, the one or more thresholds are determined using multiple reference TBR values ​​(e.g., each reference TBR value determined from a corresponding set of reference images (e.g., a set of reference 3D anatomical images and reference 3D functional images, e.g., CT / SPECT images)), each reference TBR value being associated with (e.g., assigned by a physician) a particular classification of prostate cancer status (e.g., a Gleason grade determined for the same subject for which the reference TBR value was determined (e.g., based on histopathology from a radical prostatectomy)). In certain embodiments, the one or more thresholds are determined using a receiver operating characteristic (ROC) curve (e.g., using area under the curve (AUC) analysis to yield a particular sensitivity value and / or a particular specificity value). In certain embodiments, the one or more thresholds include multiple thresholds (e.g., each threshold correlated to an average Gleason score) for comparing the TBR value to the multiple thresholds to determine the prostate cancer status as a level on a non-binary scale (e.g., a scale with three or more levels (e.g., negative, probably negative, probably positive, positive)). In certain embodiments, the method includes determining the subject's prostate cancer status as (i) clinically significant if the TBR value exceeds a cutoff threshold, or (ii) clinically insignificant if the TBR value is below the cutoff threshold.

[0034] In a particular embodiment, this method includes a step of identifying, by a processor (e.g., using a second module), a bladder volume within the 3D anatomical image (e.g., within the initial VOI) corresponding to the target bladder; and in step (e), using the intensity of voxels of the 3D functional image corresponding to the identified bladder volume within the 3D anatomical image to correct for crosstalk from the bladder (e.g., radiopharmaceutical uptake in the bladder affects the intensity values ​​of voxels of the 3D functional image representing the physical volume within the prostate through scattering and / or partial volume effects) [e.g., by adjusting the intensity of voxels of the 3D functional image corresponding to the prostate volume based on their proximity to the identified bladder volume and / or bladder uptake (e.g., determined based on the intensity of voxels of the 3D functional image corresponding to the bladder volume), e.g., by establishing a model of radiation scattering from the bladder and using the intensity of voxels of the 3D functional image to adjust the intensity of voxels of the 3D functional image based on this model].In certain embodiments, correcting for crosstalk from the bladder includes determining one or more bladder intensity bleed functions that model the contribution of intensity from the radiopharmaceutical inside the bladder of the subject to the intensity of one or more voxels of the 3D functional image corresponding to one or more regions of the 3D anatomical image that are outside the identified bladder volume, wherein each of the one or more bladder intensity bleed functions (e.g., each of the one or more bladder intensity bleed functions fits a template function (e.g., an nth order polynomial) to the intensity along a particular direction of voxels of the 3D functional image that correspond to the identified bladder volume inside the 3D anatomical image). and modeling the intensity bleed along a particular direction] as a function of distance from the identified bladder volume; and for each of one or more voxels of the 3D functional image corresponding to the identified prostate volume within the 3D anatomical image, using one or more bladder intensity bleed functions to adjust the intensity of the voxel for bladder crosstalk [e.g., by evaluating the one or more bladder intensity bleed functions, determining a bladder intensity bleed value for the voxel, and subtracting the bladder intensity bleed value from the intensity of the voxel to obtain a corrected voxel intensity].

[0035] In one particular embodiment, the method includes the steps of: identifying, by a processor (e.g., using a second module), a bladder volume within the 3D anatomical image (e.g., within an initial VOI) that corresponds to the subject's bladder; determining, by the processor, an expanded bladder volume by applying a morphological expansion operation to the identified bladder volume; and, in step (e), using intensity values ​​of voxels of the 3D functional image, determining that one or more uptake metrics (i) correspond to the identified prostate volume within the VOI of the 3D anatomical image, but (ii) do not correspond to a region of the 3D anatomical image within the expanded bladder volume (thereby, for example, omitting from the calculation of the one or more uptake metrics voxels of the 3D functional image that correspond to locations in the 3D anatomical image within a predetermined distance from the identified bladder volume, and therefore are too close to the identified bladder volume).

[0036] In certain embodiments, the 3D functional image is a nuclear medicine image (e.g., a single photon emission computed tomography (SPECT) scan, e.g., a positron emission tomography (PET) scan) of the subject after administration of a radiopharmaceutical to the subject. In certain embodiments, the radiopharmaceutical is a PSMA binding agent (e.g., 99m Tc-MIP-1404, for example [ 18 In certain embodiments, the nuclear medicine image is a single photon emission computed tomography (SPECT) scan of the subject obtained after administering the radiopharmaceutical to the subject. In certain embodiments, the radiopharmaceutical 99m Contains Tc-MIP-1404.

[0037] The method, in certain embodiments, includes determining one or more diagnostic or prognostic values ​​for the subject (e.g., values ​​that provide a measure of the subject's condition, disease progression, life expectancy (e.g., overall survival), cure, etc. (e.g., Gleason score) based at least in part on the one or more uptake metrics. In certain embodiments, determining at least one of the one or more diagnostic or prognostic values ​​includes comparing the uptake metric to one or more thresholds. In certain embodiments, at least one of the one or more diagnostic or prognostic values ​​estimates the risk of clinically significant prostate cancer in the subject.

[0038] In certain embodiments, the method includes: (f) displaying, by the processor, an interactive graphical user interface (GUI) for presenting a visual display of the 3D anatomical image and / or the 3D functional image to a user; and (g) providing, by the processor, within the GUI, graphical renderings of the 3D anatomical image and / or the 3D functional image as selectable and superimposable layers such that either may be selected for display (e.g., by a corresponding user-selectable graphical control element (e.g., a toggle element)) and rendered separately, or both may be selected for display and rendered together by overlaying the 3D functional image on the 3D anatomical image. In certain embodiments, step (g) includes providing a graphical rendering of a selectable and superimposable segmentation layer including one or more identified specific tissue volumes within the 3D anatomical image, wherein, upon selection of the segmentation layer for display, graphics representing the one or more specific tissue volumes are superimposed (e.g., as outlines, e.g., as semi-transparent color-coded volumes) on the 3D anatomical image and / or the 3D functional image. In certain embodiments, the one or more specific tissue volumes include an identified prostate volume. In certain embodiments, the method includes, in step (g), rendering 2D cross-sectional views of the 3D anatomical and / or functional images within an interactive 2D viewer such that the position of the 2D cross-sectional views may be adjusted by a user. In certain embodiments, the method includes, in step (g), rendering an interactive (e.g., rotatable, e.g., sliceable) 3D view of the 3D anatomical and / or functional images.In certain embodiments, the method includes a step of displaying within the GUI a graphical element (e.g., a cross, target, colored marker, etc.) indicating a location corresponding to a voxel of the identified prostate volume (e.g., the location also corresponds to a voxel having the greatest intensity in the 3D functional image, e.g., the greatest corrected intensity compared to other voxels in the 3D functional image corresponding to the identified prostate volume), thereby facilitating user review and / or quality control of the method (e.g., a physician can verify that the identified location corresponds to the expected physical position of the subject's prostate). In certain embodiments, the method includes displaying within the GUI text and / or graphics representing the one or more intake metrics determined in step (e) (and, e.g., one or more prognostic values ​​determined therefrom, as appropriate) along with a quality control graphical widget (e.g., the quality control graphical widget includes a selectable graphical control element for receiving an input corresponding to the user's (i) approval of the one or more intake metrics automatically determined by the processor, or (ii) disapproval of the automatic determination of the one or more intake metrics) for guiding the user through a quality control and reporting workflow for reviewing and / or updating the one or more intake metrics.In one particular embodiment, the method includes receiving, by the quality control graphical widget, user input corresponding to approval of the automatic determination of one or more intake metrics (and, e.g., any prognostic value determined therefrom); and, in response to receiving the user input corresponding to approval of the automatic determination of the one or more intake metrics, generating, by the processor, a report for the subject including a representation of the one or more automatically determined intake metrics (e.g., the report including an identification of the subject (e.g., an anonymized patient ID number), a representation (e.g., text) of the one or more determined intake metrics (and, e.g., any prognostic value determined therefrom), and a representation (e.g., graphics and / or text) of the user's approval of the automatic determination of the one or more intake metrics (and, e.g., any prognostic value determined therefrom).

[0039] The method, in one particular embodiment, includes receiving, via a quality control graphical widget, a user input corresponding to a disapproval of the automated determination of one or more uptake metrics (and, e.g., any prognostic value determined therefrom); and, in response to receiving the user input corresponding to the disapproval of the automated determination of the one or more uptake metrics, displaying, by the processor, a voxel selection graphical element (e.g., a cursor, e.g., an adjustable cross mark) for user selection (e.g., directly or indirectly, e.g., by selecting voxels in the 3D anatomical image and subsequently determining corresponding voxels in the 3D functional image) of one or more voxels of the 3D functional image to be used to determine updated values ​​of the one or more uptake metrics; and displaying, via the voxel selection graphical element, a user selection of one or more voxels of the 3D functional image to be used to determine updated values ​​of the one or more uptake metrics (e.g., the user selection may include one or more corrected background intensity measurement locations and / or one or more corrected background intensity measurement locations). the processor updates the values ​​of the one or more intake metrics using the user-selected voxels; and the processor generates a report for the subject including a representation (e.g., text) of the one or more updated intake metrics (e.g., the report includes an identification of the subject (e.g., an anonymized patient ID number), a representation (e.g., text) of the one or more determined intake metrics (and, e.g., any prognostic value determined therefrom), and a user-approved representation (e.g., graphics and / or text) of the determination of the one or more intake metrics (and, e.g., any prognostic value determined therefrom) using the user-selected voxels (e.g., text indicating that the intake metrics were determined by semi-automated analysis involving manual selection of voxels from the user).

[0040] In certain embodiments, the method includes receiving, by a quality control graphical widget, user input corresponding to a rejection of the automated determination of one or more intake metrics (and, e.g., any prognostic value determined therefrom); receiving, by a quality control graphical widget, user input corresponding to a failure of the quality control (e.g., due to poor image quality); and generating, by a processor, a report for the subject that includes an identification of the failure of the quality control.

[0041] In certain embodiments, the voxels of the 3D functional image are related to the voxels of the 3D anatomical image by a known relationship (e.g., each voxel of the 3D functional image is related to one or more voxels of the 3D anatomical image, e.g., each of multiple sets of one or more voxels of the 3D functional image is related to one or more sets of voxels of the 3D anatomical image, e.g., coordinates associated with voxels of the 3D functional image are related to coordinates associated with voxels of the anatomical 3D image by a known functional relationship (e.g., by a known spatial relationship between a first imaging diagnostic modality and a second imaging diagnostic modality)).

[0042] In a particular embodiment, the first module is a first CNN (convolutional neural network) module, the second module is a second CNN module, and the first CNN module includes more convolution filters than the second CNN module (e.g., at least 1.5 times, e.g., at least 2 times, e.g., about 2 times, e.g., at least 3 times, e.g., about 3 times).

[0043] The method, in certain embodiments, includes performing steps (a) and (c) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each within one of the plurality of 3D anatomical images, wherein the variability in size of the initial VOIs is (e.g., substantially) less than the variability in size of the 3D anatomical images (e.g., variability in "size" means one or both of (i) the variability in one or more dimensions, measured, e.g., in mm, of the anatomical volume represented in the image, and (ii) the variability in the number of voxels in the image along each of the one or more dimensions of the image) [e.g., the variability in size of the 3D anatomical images (respective overall images from which each VOI is determined) along each of the one or more dimensions is at least 200 mm, and / or at least 300 mm, and / or at least 400 mm, and / or at least 500 mm, and / or on the order of 400 mm, and and / or on the order of 500 mm, and / or on the order of 1000 mm, and / or on the order of 1500 mm, e.g., the size variation along each of one or more dimensions of the 3D anatomical image (each overall image from which each VOI is determined) is at least 25 voxels, and / or at least 50 voxels, and / or at least 100 voxels, and / or at least 250 voxels, and / or at least 300 voxels, and / or on the order of 250 voxels, and / or on the order of 300 voxels, and / or on the order of 500 voxels, e.g., the size variation along each of one or more dimensions of the VOI is 200 mm or less (e.g., 100 mm or less, e.g., 50 mm or less), e.g., the size variation along each of one or more dimensions of the VOI is 250 voxels or less (e.g., 200 voxels or less, e.g., 150 voxels or less).

[0044] In a particular embodiment, the first module is a CNN module that receives as input a downsampled first resolution version of the 3D anatomical image (e.g., a low resolution version of the 3D anatomical image) and operates thereon, and the second module is a CNN module that receives as input a high resolution version of the 3D anatomical image cropped to the initial VOI and having a second resolution, the second resolution being higher (e.g., at least 2 times, e.g., at least 4 times, e.g., at least 8 times) than the first resolution. In a particular embodiment, the first module receives as input at least a portion of the 3D anatomical image, and the physical volume represented by the input to the first module is (i) at least 2 times (e.g., 4 times, e.g., 8 times) larger than the physical volume represented by the initial VOI and / or (ii) 2 times (e.g., 4 times, e.g., 8 times) larger than the physical volume represented along at least one dimension by the initial VOI.

[0045] In one particular embodiment, the first module is a CNN module trained to identify a graphical representation of a pelvic region within a 3D anatomical image, and the second module is a CNN module trained to identify a graphical representation of prostate tissue within a 3D anatomical image.

[0046] In another aspect, the present invention is directed to a method for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a target tissue region within the subject and to determine one or more uptake metrics indicative of the uptake of a radiopharmaceutical therein, the method comprising the steps of: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT), e.g., magnetic resonance imaging (MRI), e.g., ultrasound), wherein the 3D anatomical image includes a graphical representation of tissue (e.g., soft tissue and / or bone) within a particular anatomical region of the subject (e.g., a particular group of related tissues such as the pelvic region, thoracic region, head region, and / or neck region), the particular anatomical region including the target tissue region; and (b) receiving, by the processor, a 3D functional image of the subject obtained using a functional imaging modality (e.g., single photon emission computed tomography (SPECT), e.g., positron emission tomography (PET)), wherein the 3D functional image includes a plurality of 3D volumes within the 3D anatomical region (e.g., soft tissue and / or bone). (c) using a first module (e.g., a first machine learning module) to determine an initial volume of interest (VOI) within the 3D anatomical image (e.g., a rectangular prism), wherein the initial VOI (e.g., the 3D anatomical image has more excluded voxels than included voxels, e.g., less than 25% of the 3D anatomical image has included voxels, e.g., a majority of the voxels within the VOI represent a physical volume within the specific anatomical region) corresponds to a specific anatomical region (e.g., a group of related tissues, such as the pelvic region, thoracic region, head region, and / or neck region) that includes the target tissue region; and (d) using a second module (e.g., a second machine learning module) to determine an initial volume of interest (VOI) within the 3D anatomical image (e.g., a rectangular prism), wherein the initial VOI (e.g., the 3D anatomical image has more excluded voxels than included voxels, e.g., less than 25% of the 3D anatomical image has included voxels, e.g., a majority of the voxels within the VOI represent a physical volume within the specific anatomical region) corresponds to a specific anatomical region (e.g., a group of related tissues, such as the pelvic region, thoracic region, head region, and / or neck region) that includes the target tissue region.(e) using the processor to determine one or more uptake metrics using the 3D functional image and the identified target volume within the VOI of the 3D anatomical image (e.g., calculating the amount of radiopharmaceutical in the target tissue region of the subject based on intensity values ​​of voxels of the 3D functional image corresponding to the identified target volume within the VOI of the 3D anatomical image, e.g., calculating the sum (e.g., weighted sum), average, and / or maximum of the intensities of voxels of the 3D functional image representing the physical volume occupied by the target tissue region of the subject).

[0047] In certain embodiments, the method has one or more of the features articulated in paragraphs

[0027] to

[0045] .

[0048] In another aspect, the present invention is directed to a method for automatically analyzing 3D functional images (e.g., nuclear medicine images (e.g., SPECT images, e.g., PET images)) to correct prostate voxel intensities for crosstalk from bladder radiopharmaceutical uptake, the method comprising the steps of: (a) receiving, by a processor of a computing device, a 3D anatomical image of a subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT), e.g., magnetic resonance imaging (MRI), e.g., 3D ultrasound), wherein at least a portion of a graphical representation of tissue (e.g., soft tissue and / or bone) within the subject included in the 3D anatomical image corresponds to the subject's bladder and prostate; and (b) receiving, by the processor, a 3D functional image of the subject, wherein a plurality of voxels included in the 3D functional image each represent a particular physical volume within the subject and have intensity values ​​that represent detected radiation emitted from the particular physical volume, and wherein at least a portion of the plurality of voxels in the 3D functional image represent the bladder and prostate of the subject. (c) automatically identifying, by a processor, within the 3D anatomical image (i) a prostate volume corresponding to the prostate of the subject, and (ii) a bladder volume corresponding to the bladder of the subject; (d) automatically identifying, by a processor, within the 3D functional image (i) a plurality of prostate voxels corresponding to the identified prostate volume, and (ii) a plurality of bladder voxels corresponding to the identified bladder volume; and (e) automatically identifying, by the processor, one or more measured intensities of the prostate voxels (e.g., one or more cumulative measurements and / or peak measurements and / or average measurements and / or median measurements of intensities corresponding to each and / or cumulative of the identified bladder volume and / or multiple regions of the identified bladder volume) based on one or more measured intensities of the bladder voxels.(f) using the adjusted intensities of the prostate voxels to determine, by the processor, one or more uptake metrics representative of the uptake of the radiopharmaceutical within the prostate of the subject.

[0049] In certain embodiments, the method has one or more of the features articulated in paragraphs

[0027] to

[0045] .

[0050] In another aspect, the present invention is directed to a method for detecting prostate cancer status and / or quantifying prostate cancer risk in a subject based on automated analysis of a 3D functional image (e.g., a SPECT image) of a portion of the subject, the method comprising: (a) acquiring a 3D functional image (e.g., a SPECT image) after administering to the subject a radiopharmaceutical comprising a PSMA binding agent; (b) identifying, by a processor of a computing device, a 3D target volume within the 3D functional image corresponding to the subject's prostate; (c) determining, by the processor, a target to background ratio (TBR) value using intensities of voxels of the 3D target volume; and (d) causing, by the processor, a graphical rendering of text and / or graphics representing the determined TBR value for display within an interactive graphical user interface (GUI) (e.g., the method includes determining (e.g., determining and displaying) (i) a clinically significant classification status or (ii) a clinically insignificant classification status of prostate cancer based at least in part on the TBR value).

[0051] In certain embodiments, the method has one or more of the features articulated in paragraphs

[0027] to

[0045] .

[0052] In another aspect, the present invention is directed to a system for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a subject's prostate gland and to determine one or more uptake metrics indicative of uptake of a radiopharmaceutical therein (i.e., in the prostate gland), the system comprising a processor and memory (e.g., external to the processor or incorporated within the processor) having instructions stored thereon, the processor executing the instructions to perform the following steps: (a) receiving a 3D anatomical image of the subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT) (e.g., whole-body CT image, e.g., regional CT image of the body), e.g., magnetic resonance imaging (MRI), e.g., 3D ultrasound), wherein at least a portion of a graphical representation of the subject's internal tissue (e.g., soft tissue and / or bone) contained in the 3D anatomical image corresponds to the subject's pelvic region; and (b) receiving a 3D anatomical image of the subject obtained using a functional imaging modality (e.g., single photon emission computed tomography (SPECT), e.g., positron emission tomography (PET)). (c) receiving a 3D functional image of a subject, wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the subject and have intensity values ​​representative of detected radiation emitted from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within a pelvic region of the subject; and (d) using a first module (e.g., a first machine learning module) to generate an initial target volume (V) within the 3D anatomical image (e.g., a parallelepiped, e.g., a rectangular parallelepiped). (d) determining an initial VOI (VOI) corresponding to tissues within the pelvic region of the subject and excluding tissues outside the pelvic region of the subject (e.g., the excluded voxels from the VOI are greater than the included voxels, e.g., less than 25% of the included voxels of the 3D anatomical image, e.g., the majority of the voxels within the VOI represent a physical volume within the pelvic region of the subject); and (d) using a second module (e.g., a second machine learning module),(e) using the 3D functional image and the identified prostate volume within the initial VOI in the 3D anatomical image to determine (e.g., determine and display) one or more uptake metrics (e.g., calculating the amount of radiopharmaceutical in the subject's prostate based on intensity values ​​of voxels in the 3D functional image corresponding to the identified prostate volume within the initial VOI in the 3D anatomical image, e.g., calculating the sum (e.g., weighted sum) and / or mean and / or maximum of intensities of voxels in the 3D functional image representing the physical volume occupied by the subject's prostate) (e.g., the one or more uptake metrics include a tumor to background ratio (TBR) value, and the processor executes the instructions to determine (e.g., determine and display) a classification status of the prostate cancer as either (i) clinically significant or (ii) clinically insignificant based at least in part on the TBR value).

[0053] In certain embodiments, the system has one or more of the features articulated in paragraphs

[0027] to

[0045] .

[0054] In another aspect, the present invention is directed to a system for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a target tissue region within the subject, and to determine one or more uptake metrics indicative of the uptake of a radiopharmaceutical therein (i.e., in the target tissue region), the system comprising a processor and memory (e.g., external to the processor or incorporated within the processor) having instructions stored thereon, the processor executing the instructions to perform the following steps: (a) receiving a 3D anatomical image of the subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT), e.g., magnetic resonance imaging (MRI), e.g., ultrasound), the 3D anatomical image comprising a graphical representation of tissue (e.g., soft tissue and / or bone) within a particular anatomical region of the subject (e.g., a particular group of related tissues, such as the pelvic region, thoracic region, head region, and / or neck region), the particular anatomical region comprising the target tissue region; and (b) receiving a 3D anatomical image of the subject obtained using an anatomical imaging modality (e.g., single photon emission computed tomography (SPECT), e.g., positron (c) receiving a 3D functional image of a subject obtained using a PET (Potassium Emission Tomography) system, wherein each of a plurality of voxels included in the 3D functional image represents a particular physical volume within the subject and has intensity values ​​representative of radiation emitted and detected from the particular physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within a particular anatomical region of the subject; and (d) using a first module (e.g., a first machine learning module) to generate a 3D functional image of a subject (e.g., a rectangular prism). determining an internal initial volume of interest (VOI), where the initial VOI (e.g., the VOI has more excluded voxels than included voxels of the 3D anatomical image, e.g., less than 25% of the included voxels of the 3D anatomical image, e.g., a majority of the voxels within the VOI represent a physical volume within the particular anatomical region) corresponds to a particular anatomical region (e.g., a group of related tissues such as the pelvic region, thoracic region, head region, and / or neck region) that includes the target tissue region;(d) using a second module (e.g., a second machine learning module) to identify a target volume within the initial VOI corresponding to the target tissue region of the subject; and (e) using the 3D functional image and the identified target volume within the VOI of the 3D anatomical image to determine one or more uptake metrics (e.g., calculating the amount of radiopharmaceutical in the target tissue region of the subject based on intensity values ​​of voxels in the 3D functional image corresponding to the target volume identified within the VOI of the 3D anatomical image, e.g., calculating the sum (e.g., weighted sum), and / or average and / or maximum of intensities of voxels in the 3D functional image representing the physical volume occupied by the target tissue region of the subject) (e.g., the one or more uptake metrics include a tumor to background ratio (TBR) value, and the processor executes the instructions to determine (e.g., determine and display) a classification status of the cancer as either (i) clinically significant or (ii) clinically insignificant based at least in part on the TBR value).

[0055] In certain embodiments, the system has one or more of the features articulated in paragraphs

[0027] to

[0045] .

[0056] In another aspect, the present invention is directed to a system for detecting prostate cancer status and / or quantifying prostate cancer risk in a subject based on automated analysis of a 3D functional image (e.g., a SPECT image) of a portion of the subject, the system comprising a processor and memory (e.g., external to the processor or incorporated into the processor) having instructions stored thereon, the processor executing the instructions to (a) receive a 3D functional image (e.g., a SPECT image) of a portion (e.g., any or all) of the subject's prostate gland after administering to the subject a radiopharmaceutical comprising a PSMA binding agent; and (b) receive a 3D functional image (e.g., a SPECT image) of a portion (e.g., any or all) of the subject's prostate gland within the 3D functional image. (c) identifying a 3D target volume corresponding to the target to background ratio (TBR) value; (c) determining a target to background ratio (TBR) value using the intensities of voxels in the 3D target volume; and (d) providing a graphical rendering of text and / or graphics representing the determined TBR value for display within an interactive graphical user interface (GUI) (e.g., the processor executes these instructions to determine (e.g., determine and display) (i) a clinically significant classification status or (ii) a clinically insignificant classification status of the prostate cancer based at least in part on the TBR value).

[0057] In certain embodiments, the system has one or more of the features articulated in paragraphs

[0027] to

[0045] .

[0058] In certain embodiments, the present invention is directed to a computer-aided detection (CADe) device comprising any of the systems described herein, in which a processor, upon instruction, identifies a classification of a subject as either clinically significant or clinically insignificant prostate cancer.

[0059] In certain embodiments, the present invention is directed to a computer-aided diagnosis (CADx) device comprising any of the systems described herein, in which a processor, in accordance with instructions, identifies a classification of a subject as either clinically significant or clinically insignificant prostate cancer.

[0060] The present invention, in certain embodiments, provides a method for producing a PSMA-binding agent, comprising: (a) a radiolabeled PSMA-binding agent (e.g., 99m Tc-MIP-1404, for example [ 18

[0013] The present invention is directed to a combination product comprising (a) a radiolabeled PSMA-binding agent (e.g., a radiolabeled PSMA-binding agent, e ...

[0061] In another aspect, the present invention is directed to a method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a target tissue region (e.g., prostate, e.g., lung, e.g., one or more bones, e.g., lymph nodes, e.g., brain), the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of a subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT), e.g., magnetic resonance imaging (MRI), e.g., ultrasound), the 3D anatomical image including a graphical representation of tissue (e.g., soft tissue and / or bone) within the subject; and (b) determining, by the processor, an initial volume of interest (VOI) within the 3D anatomical image (e.g., rectangular prism) using a first module (e.g., a first machine learning module). (c) using a second module (e.g., a second machine learning module) to identify a target volume within the initial VOI that corresponds to the target tissue region of interest; and (d) storing and / or providing, by the processor, a 3D segmentation mask corresponding to the identified target volume for display and / or further processing.

[0062] In one particular embodiment, a first module receives as input a 3D anatomical image and outputs a plurality of coordinate values ​​representing opposite corners of a rectangular volume within the 3D anatomical image (e.g., two sets of coordinate values ​​representing opposite corners of a rectangular volume).

[0063] In one particular embodiment, the first module is a convolutional neural network (CNN) module (e.g., a neural network module that utilizes one or more convolutional layers).

[0064] In certain embodiments, step (c) includes using a second module to identify one or more other tissue volumes within the 3D anatomical image, each of which corresponds to a particular tissue region within the subject.

[0065] In a particular embodiment, step (c) includes using a second module to classify each voxel within the initial VOI as corresponding to a particular tissue region from a set of (predetermined) different tissue regions within the subject {e.g., the set includes the target tissue region and, optionally, one or more other tissue regions (e.g., the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hip bone, e.g., right hip bone), the subject's bladder, the subject's rectum, and gluteal muscles (e.g., left gluteal muscle, e.g., right gluteal muscle)}.

[0066] In certain embodiments, the step of classifying each voxel within the initial VOI includes the steps of: determining, by a second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, wherein the set of likelihood values ​​includes, for each of one or more tissue regions of the set of tissue regions, a corresponding likelihood value representing the likelihood (e.g., calculated by the second module) that the voxel represents a physical volume within the tissue region; and, for each of the plurality of voxels within the initial VOI, classifying the voxel as corresponding to a particular tissue region based on the set of likelihood values ​​determined for the voxel.

[0067] In a particular embodiment, the second module receives as input an initial VOI (e.g., the entire initial VOI) and generates, for each voxel within the initial VOI, (i) a value classifying the voxel (e.g., a value classifying the voxel as corresponding to a particular tissue region, such as a region selected from a predetermined set of different tissue regions, e.g., a predetermined set including a target tissue region, e.g., the subject's prostate, e.g., the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hipbone, e.g., right hipbone), the subject's bladder, the subject's rectum, or the subject's gluteal muscles (e.g., left gluteal muscles, e.g., right gluteal muscles)) and (ii) a set of likelihood values ​​for the voxel (e.g., a value classifying the voxel as corresponding to a particular tissue region, such as a region selected from a predetermined set of different tissue regions, e.g., a predetermined set including a target tissue region, e.g., the subject's prostate, e.g., the subject's pelvic bone (e.g., sacrum, e.g., coccyx, e.g., left hipbone, e.g., right hipbone), the subject's bladder, the subject's rectum, or the subject's gluteal muscles (e.g., left gluteal muscles, e.g., right gluteal muscles)). and (iii) a value identifying the voxel as not corresponding to any of the predetermined set of different tissue regions (e.g., a value identifying as unlikely to correspond or a value identifying the likelihood that the voxel does not correspond) (e.g., a value identifying the voxel as, for example, corresponding to a non-diagnostic background area or likely to correspond to a background area, or a value identifying the likelihood that the voxel corresponds to a background area) (e.g., such that the second module classifies and / or calculates likelihood values ​​for the entire VOI at once, as opposed to operating on each voxel one at a time).

[0068] In certain embodiments, step (c) includes using a second module to identify a set of one or more underlying tissue volumes including the identified target volume and one or more other tissue volumes, and the method further includes using one or more auxiliary modules (e.g., an auxiliary machine learning module) to identify, by the processor, one or more auxiliary tissue volumes within the 3D anatomical image, each of which corresponds to an underlying tissue volume identified by the second module {e.g., the same specific tissue region (e.g., the subject's prostate, e.g., the subject's pelvic bone (e.g., the sacrum, e.g., the coccyx, e.g., the left hipbone, e.g., the right hipbone), the subject's bladder, the subject's rectum, and the gluteal muscles (e.g., the left gluteal muscle, e.g., the right gluteal muscle)) is represented as this underlying tissue volume}, and merging, by the processor, each auxiliary tissue volume with the corresponding underlying tissue volume identified by the second module (e.g., augmenting the corresponding underlying tissue volume by incorporating a portion of the corresponding auxiliary tissue volume that was not included in the original underlying tissue volume).

[0069] In one particular embodiment, the method includes: (e) displaying, by the processor, an interactive graphical user interface (GUI) to present a visual display of the 3D anatomical image to a user; and (f) providing, by the processor, within the GUI a graphical rendering of the 3D anatomical image together with selectable and superimposable segmentation layers that include one or more identified specific tissue volumes within the 3D anatomical image, wherein when a segmentation layer is selected for display, graphics representing the one or more specific tissue volumes are superimposed on the 3D anatomical image (e.g., as outlines, e.g., as translucent color-coded volumes).

[0070] In certain embodiments, the one or more specific tissue volumes include an identified target volume.

[0071] In certain embodiments, the method includes, in step (f), rendering a 2D cross-sectional view of the 3D anatomical image within an interactive 2D viewer such that the position of the 2D cross-sectional view can be adjusted by a user.

[0072] In certain embodiments, the method includes, in step (f), rendering an interactive (eg, rotatable, eg, sliceable) 3D view of the 3D anatomical image.

[0073] In a particular embodiment, the first module is a first CNN (convolutional neural network) module, the second module is a second CNN module, and the first CNN module includes more convolution filters than the second CNN module (e.g., at least 1.5 times, e.g., at least 2 times, e.g., about 2 times, e.g., at least 3 times, e.g., about 3 times).

[0074] The method, in certain embodiments, includes performing steps (a) and (b) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs each within one of the plurality of 3D anatomical images, wherein the variability in size of the initial VOIs is (e.g., substantially) less than the variability in size of the 3D anatomical images (e.g., variability in "size" means one or both of (i) the variability in one or more dimensions, measured, e.g., in mm, of the anatomical volume represented in the image, and (ii) the variability in the number of voxels in the image along each of the one or more dimensions of the image) [e.g., the variability in size of the 3D anatomical images (respective overall images from which each VOI is determined) along each of the one or more dimensions is at least 200 mm, and / or at least 300 mm, and / or at least 400 mm, and / or at least 500 mm, and / or on the order of 400 mm, and and / or on the order of 500 mm, and / or on the order of 1000 mm, and / or on the order of 1500 mm, e.g., the size variation along each of one or more dimensions of the 3D anatomical image (each overall image from which each VOI is determined) is at least 25 voxels, and / or at least 50 voxels, and / or at least 100 voxels, and / or at least 250 voxels, and / or at least 300 voxels, and / or on the order of 250 voxels, and / or on the order of 300 voxels, and / or on the order of 500 voxels, e.g., the size variation along each of one or more dimensions of the VOI is 200 mm or less (e.g., 100 mm or less, e.g., 50 mm or less), e.g., the size variation along each of one or more dimensions of the VOI is 250 voxels or less (e.g., 200 voxels or less, e.g., 150 voxels or less).

[0075] In one particular embodiment, the first module is a CNN module that receives as input a downsampled first resolution version of the 3D anatomical image (e.g., a low resolution version of the 3D anatomical image) and operates on it, and the second module is a CNN module that receives as input a high resolution version of the 3D anatomical image cropped to the initial VOI and having a second resolution, the second resolution being higher (e.g., at least 2 times, e.g., at least 4 times, e.g., at least 8 times) than the first resolution.

[0076] In certain embodiments, the first module receives as input at least a portion of a 3D anatomical image, and the physical volume represented by the input to the first module is (i) at least two times (e.g., four times, e.g., eight times) larger than the physical volume represented by the initial VOI, and / or (ii) two times (e.g., four times, e.g., eight times) larger than the physical volume represented along at least one dimension by the initial VOI.

[0077] In one particular embodiment, the first module is a CNN module trained to identify a graphical representation of a particular anatomical region within a 3D anatomical image, and the second module is a CNN module trained to identify a graphical representation of a target tissue region within the 3D anatomical image.

[0078] In certain embodiments, the method further includes receiving, by a processor, a 3D functional image of the subject obtained using a functional imaging diagnostic technique (e.g., single photon emission computed tomography (SPECT), e.g., positron emission tomography (PET)), wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the subject and have intensity values ​​representing radiation emitted and detected from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within a specific anatomical region; and identifying, within the 3D functional image, a 3D volume corresponding to the identified target volume using a 3D segmentation mask (e.g., by mapping the 3D segmentation mask to the 3D functional image).

[0079] In certain embodiments, the voxels of the 3D functional image are related to the voxels of the 3D anatomical image by a known relationship (e.g., each voxel of the 3D functional image is related to one or more voxels of the 3D anatomical image, e.g., each of multiple sets of one or more voxels of the 3D functional image is related to one or more sets of voxels of the 3D anatomical image, e.g., coordinates associated with voxels of the 3D functional image are related to coordinates associated with voxels of the anatomical 3D image by a known functional relationship (e.g., by a known spatial relationship between a first imaging diagnostic modality and a second imaging diagnostic modality)).

[0080] In another aspect, the present invention is directed to a system for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a target tissue region (e.g., prostate, e.g., lung, e.g., one or more bones, e.g., lymph nodes, e.g., brain), the system comprising: a processor of a computing device; and a memory storing instructions, the processor executing the instructions to (a) receive a 3D anatomical image of a subject obtained using an anatomical imaging modality (e.g., X-ray computed tomography (CT), e.g., magnetic resonance imaging (MRI), e.g., ultrasound), the 3D anatomical image including a graphical representation of tissue (e.g., soft tissue and / or bone) within the subject; and (b) using a first module (e.g., a first machine learning module) to generate an initial 3D volume within the 3D anatomical image (e.g., a rectangular prism). (c) determining an initial volume of interest (VOI), where the initial VOI (e.g., the excluded voxels from this VOI of the 3D anatomical image are more numerous than the included voxels, e.g., less than 25% of the included voxels of the 3D anatomical image, e.g., a majority of the voxels within the VOI represent a physical volume within the particular anatomical region) corresponds to a particular anatomical region (e.g., a group of related tissues such as the pelvic region, thoracic region, head region, and / or neck region) that includes the target region; (d) storing and / or providing a 3D segmentation mask corresponding to the identified target volume for display and / or further processing.

[0081] In certain embodiments, the system has one or more of the features articulated in paragraphs

[0062] to

[0079] .

[0082] Features of embodiments described with respect to one aspect of the invention may be applied with respect to another aspect of the invention. The present invention provides, for example, the following items. (Item 1) 1. A method for automatically processing a 3D image to identify a 3D volume within said 3D image that corresponds to a subject's prostate gland and to determine one or more uptake metrics indicative of an uptake of a radiopharmaceutical therein, comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using an anatomical imaging modality, the 3D anatomical image including at least a portion of a graphical representation of an internal structure of the subject corresponding to a pelvic region of the subject; (b) receiving by the processor a 3D functional image of the subject obtained using a functional imaging diagnostic method, wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the subject and have intensity values ​​representative of detected radiation emitted from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within the pelvic region of the subject; (c) determining, by the processor, using a first module, an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to tissue within the pelvic region of the subject and excluding tissue outside the pelvic region of the subject; (d) identifying, by the processor, a prostate volume within the initial VOI that corresponds to the prostate of the subject using a second module; (e) determining, by the processor, the one or more uptake metrics using the 3D functional image and the prostate volume identified within the initial VOI in the 3D anatomical image. (Item 2) Item 10. The method of item 1, wherein the first module receives the 3D anatomical image as input and outputs a plurality of coordinate values ​​representing opposite corners of a rectangular volume within the 3D anatomical image. (Item 3) 3. The method of claim 1, wherein step (c) includes using the first module to determine a 3D pelvic bone mask that identifies a volume of the 3D anatomical image that corresponds to a pelvic bone of the subject. (Item 4) 10. The method of any one of the preceding items, wherein the first module is a convolutional neural network (CNN) module. (Item 5) Step (d) includes using the second module to identify one or more other tissue volumes within the 3D anatomical image, each other tissue volume corresponding to a particular tissue region within the subject, the one or more other tissue volumes comprising: a pelvic bone of said subject; the bladder of said subject; the rectum of said subject, and 10. The method of any one of the preceding items, wherein the method corresponds to one or more specific tissue regions selected from the group consisting of gluteal muscles of the subject. (Item 6) 10. The method of claim 1, wherein step (d) comprises using the second module to classify each voxel within the initial VOI as corresponding to a specific tissue region among a set of (predetermined) different tissue regions within the subject. (Item 7) classifying each voxel within the initial VOI, determining, by the second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, the set of likelihood values ​​including, for each of one or more tissue regions of the set of tissue regions, a corresponding likelihood value that represents a likelihood that the voxel represents a physical volume within the tissue region; and for each of the plurality of voxels within the initial VOI, classifying the voxel as corresponding to the particular tissue region based on the set of likelihood values ​​determined for the voxel. (Item 8) The second module receives the initial VOI as input and, for each voxel within the initial VOI, (i) a value classifying said voxel; (ii) a set of likelihood values ​​for said voxels; and (iii) a value that identifies the voxel as not corresponding to any of a predetermined set of different tissue regions. (Item 9) said (predetermined) set of different tissue regions: the prostate gland of the subject; a pelvic bone of the subject; and a bladder of the subject; and a rectum of the subject; and 9. The method of any one of items 6 to 8, comprising one or more tissue areas selected from the group consisting of: gluteal muscles of the subject. (Item 10) and step (d) includes identifying a set of one or more underlying tissue volumes using the second module, the one or more underlying tissue volumes including the identified prostate volume and the one or more other tissue volumes, the method comprising: identifying, by the processor, one or more auxiliary tissue volumes within the 3D anatomical image using one or more auxiliary modules, each auxiliary tissue volume corresponding to an underlying tissue volume identified by the second module; 10. The method of any one of items 5 to 9, further comprising: merging, by the processor, each auxiliary tissue volume with the corresponding underlying tissue volume identified by the second module. (Item 11) identifying, by the processor, a reference volume within the 3D anatomical image corresponding to a reference tissue region within the subject; and determining at least one of the one or more uptake metrics using the 3D functional image and the reference volume identified within the 3D anatomical image in step (e). at least one of the one or more uptake metrics determined using the 3D functional image and the reference volume comprises a tumor to background ratio (TBR) value, and determining the TBR value comprises: determining a target intensity value using intensity values ​​of one or more voxels of the 3D functional image that correspond to the prostate volume identified within the initial VOI of the 3D anatomical image; determining a background intensity value using intensity values ​​of one or more voxels of the 3D functional image that correspond to the reference volume identified within the 3D anatomical image; and determining a ratio of the target intensity value to the background intensity value as the TBR value. (Item 13) 13. The method of claim 12, comprising determining the prostate cancer status of the subject based on a comparison of the TBR value with one or more thresholds. (Item 14) Item 14. The method of item 13, wherein the one or more thresholds are determined using a plurality of reference TBR values. (Item 15) 15. The method of item 13 or 14, wherein the one or more thresholds are determined using a receiver operating characteristic (ROC) curve. (Item 16) 16. The method of any one of items 13 to 15, wherein the one or more thresholds comprise a plurality of thresholds, and the prostate cancer status is determined as a level on a non-binary scale by comparing the TBR value with the plurality of thresholds. (Item 17) 14. The method of claim 13, comprising the step of determining the prostate cancer status of the subject as (i) clinically significant if the TBR value exceeds a cutoff threshold, or (ii) clinically insignificant if the TBR value is less than the cutoff threshold. (Item 18) identifying, by the processor, a bladder volume within the 3D anatomical image that corresponds to the subject's bladder; and in step (e), correcting for crosstalk from the bladder using intensities of voxels of the 3D functional image corresponding to the identified bladder volume within the 3D anatomical image. (Item 19) correcting for crosstalk from the bladder, determining one or more bladder intensity bleed functions that model the contribution of intensity from a radiopharmaceutical within the bladder of the subject to the intensity of one or more voxels of the 3D functional image corresponding to one or more regions of the 3D anatomical image that are outside the identified bladder volume, wherein the one or more bladder intensity bleed functions model the contribution as a function of distance from the identified bladder volume; and for each of one or more voxels of the 3D functional image corresponding to the identified prostate volume within the 3D anatomical image, adjusting the intensity of the voxel for bladder crosstalk using the one or more bladder intensity bleed functions. (Item 20) identifying, by the processor, a bladder volume within the 3D anatomical image that corresponds to the subject's bladder; determining, by the processor, an expanded bladder volume by applying a morphological expansion operation to the identified bladder volume; The method of any one of the preceding items, comprising, in step (e), using intensity values ​​of voxels of the 3D functional image to determine that the one or more uptake metrics (i) correspond to the prostate volume identified within the VOI of the 3D anatomical image, but (ii) do not correspond to a region of the 3D anatomical image within the expanded bladder volume. (Item 21) 10. The method of any one of the preceding items, wherein the 3D functional image is a nuclear medicine image of the subject after administering the radiopharmaceutical to the subject. (Item 22) 22. The method of claim 21, wherein the radiopharmaceutical comprises a PSMA binding agent. (Item 23) 22. The method of claim 21, wherein the nuclear medicine image is a single photon emission computed tomography (SPECT) scan of the subject obtained after administering the radiopharmaceutical to the subject. (Item 24) The radiopharmaceutical 99m 22. The method of claim 21, comprising Tc-MIP-1404. (Item 25) 10. The method of any one of the preceding items, comprising determining one or more diagnostic or prognostic values ​​for the subject based at least in part on the one or more uptake metrics. (Item 26) 26. The method of claim 25, wherein determining at least one of the one or more diagnostic or prognostic values ​​comprises comparing the uptake metric to one or more thresholds. (Item 27) 27. The method of item 25 or 26, wherein at least one of the one or more diagnostic or prognostic values ​​estimates the subject's risk for clinically significant prostate cancer. (Item 28) (f) displaying, by the processor, an interactive graphical user interface (GUI) to present a visual representation of the 3D anatomical image and / or the 3D functional image to the user; (g) providing, by the processor, within the GUI a graphical rendering of the 3D anatomical image and / or the 3D functional image as selectable and superimposable layers such that either may be selected for display and rendered separately, or both may be selected for display and rendered together by overlaying the 3D functional image on the 3D anatomical image. (Item 29) 30. The method of claim 28, wherein step (g) includes providing a graphical rendering of a selectable, superimposable segmentation layer that includes one or more identified specific tissue volumes within the 3D anatomical image, and upon selection of the segmentation layer for display, graphics representing the one or more specific tissue volumes are superimposed on the 3D anatomical image and / or the 3D functional image. 30. The method of claim 29, wherein the one or more specific tissue volumes include the identified prostate volume. (Item 31) 31. The method of any one of items 28 to 30, comprising, in step (g), rendering the 2D cross-sectional views of the 3D anatomical image and / or the 3D functional image inside an interactive 2D viewer such that the position of the 2D cross-sectional views can be adjusted by the user. (Item 32) 32. The method of any one of items 28 to 31, comprising in step (g) rendering an interactive 3D view of the 3D anatomical image and / or the 3D functional image. (Item 33) 33. The method of any one of items 28 to 32, comprising a step of facilitating user review and / or quality control of the method by displaying within the GUI graphical elements indicating locations corresponding to voxels of the identified prostate volume. (Item 34) 34. The method of any one of items 28 to 33, comprising displaying within the GUI text and / or graphics representing the one or more intake metrics determined in step (e) along with a quality control graphical widget for guiding the user through a quality control and reporting workflow for reviewing and / or updating the one or more intake metrics. (Item 35) receiving a user input via the quality control graphical widget corresponding to approval of the automated determination of the one or more intake metrics; and generating, by the processor, a report for the subject including a representation of the one or more automatically determined intake metrics in response to receiving the user input corresponding to the approval of the automatic determination of the one or more intake metrics. (Item 36) receiving a user input via the quality control graphical widget corresponding to a disapproval of the automated determination of the one or more intake metrics; in response to receiving a user input corresponding to the disapproval of the automatic determination of the one or more intake metrics, displaying, by the processor, a voxel selection graphical element for user selection of one or more voxels of the 3D functional image for use in determining updated values ​​of the one or more intake metrics; receiving, via the voxel selection graphical element, the user selection of one or more voxels of the 3D functional image to use in determining updated values ​​of the one or more uptake metrics; updating, by the processor, values ​​of the one or more intake metrics using the user-selected voxels; and generating, by the processor, a report for the subject including a representation of the one or more updated intake metrics. (Item 37) receiving a user input via the quality control graphical widget corresponding to a disapproval of the automated determination of the one or more intake metrics; receiving user input via the quality control graphical widget corresponding to a quality control failure; and generating, by the processor, a report for the subject that includes an identification of the quality control failure. (Item 38) 10. The method of any one of the preceding items, wherein voxels of the 3D functional image are related to voxels of the 3D anatomical image by a known relationship. (Item 39) A method according to any one of the preceding items, comprising the step of performing steps (a) and (c) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each of which is within one of the plurality of 3D anatomical images, wherein the variability in size of the initial VOIs is less than the variability in size of the 3D anatomical images. (Item 40) 10. The method of claim 1, wherein the first module is a CNN module that receives as input a downsampled version of the 3D anatomical image having a first resolution and operates on the same, and the second module is a CNN module that receives as input a high-resolution version of the 3D anatomical image having a second resolution higher than the first resolution, cropped to the initial VOI, and operates on the same. (Item 41) 10. The method of claim 1, wherein the first module receives as input at least a portion of the 3D anatomical image, and wherein the physical volume represented by the input to the first module is (i) at least two times larger than the physical volume represented by the initial VOI, and / or (ii) two times larger along at least one dimension than the physical volume represented by the initial VOI. (Item 42) 10. The method of claim 1, wherein the first module is a CNN module trained to identify a graphical representation of a pelvic region within a 3D anatomical image, and the second module is a CNN module trained to identify a graphical representation of prostate tissue within a 3D anatomical image. (Item 43) 1. A method for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a target tissue region within a subject and to determine one or more uptake metrics indicative of uptake of a radiopharmaceutical therein, comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using anatomical imaging, the 3D anatomical image including a graphical representation of tissue within a particular anatomical region of the subject that includes the target tissue region; (b) receiving by the processor a 3D functional image of the object obtained using a functional imaging diagnostic method, wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the object and have intensity values ​​representative of detected radiation emitted from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within the specific anatomical region of the object; (c) determining, by the processor, using a first module, an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to the particular anatomical region that contains the target tissue region; (d) identifying, by the processor, a target volume within the initial VOI corresponding to the target tissue region of the subject using a second module; (e) determining, by the processor, the one or more uptake metrics using the 3D functional image and the target volume identified within the VOI in the 3D anatomical image. (Item 44) 1. A method for automatically analyzing 3D functional images to correct prostate voxel intensities for crosstalk from radiopharmaceutical uptake into the bladder, comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using an anatomical imaging modality, the 3D anatomical image including at least a portion of a graphical representation of tissues within the subject corresponding to the subject's bladder and prostate; (b) receiving, by the processor, the 3D functional image of the object, wherein a plurality of voxels included in the 3D functional image each represent a particular physical volume within the object and have intensity values ​​representative of detected radiation emitted from the particular physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent physical volumes within the bladder and / or prostate of the object; (c) automatically identifying, by the processor, within the 3D anatomical image: (i) a prostate volume corresponding to the subject's prostate gland, and (ii) a bladder volume corresponding to the subject's bladder; (d) automatically identifying, by the processor, within the 3D functional image: (i) a plurality of prostate voxels corresponding to the identified prostate volume, and (ii) a plurality of bladder voxels corresponding to the identified bladder volume; (e) adjusting, by the processor, the measured intensities of one or more of the prostate voxels based on the measured intensities of one or more of the bladder voxels; (f) determining, by the processor, one or more uptake metrics representative of uptake of a radiopharmaceutical within the prostate gland of the subject using the adjusted intensities of the prostate voxels. (Item 45) 1. A method of detecting prostate cancer status and / or quantifying prostate cancer risk in a subject based on automated analysis of 3D functional images of a portion of said subject, comprising: (a) acquiring a 3D functional image after administering to said subject a radiopharmaceutical comprising a PSMA binding agent; (b) identifying, by a processor of a computing device, a 3D target volume within the 3D functional image that corresponds to the subject's prostate; (c) determining, by the processor, a target to background ratio (TBR) value using the intensities of voxels of the 3D target volume; (d) causing, by the processor, a graphical rendering of text and / or graphics representing the determined TBR value for display within an interactive graphical user interface (GUI). (Item 46) 1. A system for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a subject's prostate gland and to determine one or more uptake metrics indicative of an uptake of a radiopharmaceutical therein, comprising: a processor; and a memory storing instructions, wherein the processor executes the instructions to: (a) receiving a 3D anatomical image of the subject obtained using an anatomical imaging modality, the 3D anatomical image including a graphical representation of an internal structure of the subject, at least a portion of which corresponds to a pelvic region of the subject; (b) receiving a 3D functional image of the subject obtained using a functional imaging diagnostic method, wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the subject and have intensity values ​​representative of detected radiation emitted from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within the pelvic region of the subject; (c) using a first module to determine an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to tissue within the pelvic region of the subject and excluding tissue outside the pelvic region of the subject; (d) using a second module to identify a prostate volume within the initial VOI that corresponds to the prostate of the subject; (e) determining the one or more uptake metrics using the 3D functional image and the prostate volume identified within the initial VOI in the 3D anatomical image. (Item 47) 1. A system for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a target tissue region within a subject and to determine one or more uptake metrics indicative of an uptake of a radiopharmaceutical therein, comprising: a processor; and a memory storing instructions, wherein the processor executes the instructions to: (a) receiving a 3D anatomical image of the subject obtained using an anatomical imaging modality, the 3D anatomical image including a graphical representation of tissue within a particular anatomical region of the subject that includes the target tissue region; (b) receiving a 3D functional image of the object obtained using a functional imaging diagnostic method, wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the object and have intensity values ​​representative of detected radiation emitted from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent physical volumes within the specific anatomical region; (c) using a first module to determine an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to the particular anatomical region that contains the target tissue region; (d) using a second module, identifying a target volume within the initial VOI that corresponds to the target tissue region of the subject; (e) determining the one or more uptake metrics using the 3D functional image and the target volume identified within the VOI in the 3D anatomical image. (Item 48) 1. A system for detecting prostate cancer status and / or quantifying prostate cancer risk in a subject based on automated analysis of 3D functional images of a portion of the subject, comprising: a processor; and a memory storing instructions, wherein the processor executes the instructions to: (a) receiving the 3D functional image of the portion of the subject after administering to the subject a radiopharmaceutical comprising a PSMA binding agent; (b) identifying a 3D target volume within the 3D functional image corresponding to the subject's prostate; (c) determining a target to background ratio (TBR) value using the intensities of voxels of the 3D target volume; (d) providing a graphical rendering of the determined TBR value for display within an interactive graphical user interface (GUI). (Item 49) 49. A computer-aided sensing (CADe) device comprising the system of any one of items 46 to 48. (Item 50) 49. A computer-aided diagnosis (CADx) device comprising the system according to any one of items 46 to 48. (Item 51) 50. The CADe device of item 49, wherein the processor, in accordance with the instructions, identifies a classification of the subject as either clinically significant or clinically insignificant prostate cancer. (Item 52) 51. The CADx device of item 50, wherein the processor, in accordance with the instructions, identifies a classification of the subject as either clinically significant or clinically insignificant prostate cancer. (Item 53) (a) a radiolabeled PSMA binding agent; (b) a computer-aided detection (CADe) device comprising the system according to any one of items 46 to 48. (Item 54) 54. The combination product of item 53, comprising a label that provides for the use of the radiolabeled PSMA-binding agent with a computer-assisted detection device. (Item 55) 1. A method for automatically processing a 3D image to identify a 3D volume within said 3D image that corresponds to a target tissue region, comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of a subject obtained using an anatomical imaging modality, the 3D anatomical image including a graphical representation of an internal structure of the subject; (b) determining, by the processor, using a first module, an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to a particular anatomical region that includes the target region; (c) identifying, by the processor, a target volume within the initial VOI corresponding to the target tissue region of the subject using a second module; (d) storing and / or providing, by the processor, a 3D segmentation mask corresponding to the identified target volume for display and / or further processing. (Item 56) Item 6. The method of item 5, wherein the first module receives the 3D anatomical image as input and outputs a plurality of coordinate values ​​representing opposite corners of a rectangular volume within the 3D anatomical image. (Item 57) 57. The method of claim 55 or 56, wherein the first module is a convolutional neural network (CNN) module. (Item 58) 59. The method of claim 55, wherein step (c) includes using the second module to identify one or more other tissue volumes within the 3D anatomical image, each of the other tissue volumes corresponding to a particular tissue region within the subject. 59. The method of any one of items 55 to 58, wherein step (c) comprises using the second module to classify each voxel within the initial VOI as corresponding to a specific tissue region among a set of (predetermined) different tissue regions within the subject. (Item 60) classifying each voxel within the initial VOI, determining, by the second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, the set of likelihood values ​​including, for each of one or more tissue regions of the set of tissue regions, a corresponding likelihood value that represents a likelihood that the voxel represents a physical volume within the tissue region; Item 59. The method of item 59, comprising: for each of the plurality of voxels within the initial VOI, classifying the voxel as corresponding to the particular tissue region based on a set of likelihood values ​​determined for the voxel. (Item 61) The second module receives the initial VOI as input and, for each voxel within the initial VOI, (i) a value classifying said voxel; (ii) a set of likelihood values ​​for said voxels; and (iii) a value that identifies the voxel as not corresponding to any of a predetermined set of different tissue regions. (Item 62) step (c) includes identifying a set of one or more underlying tissue volumes using the second module, the one or more underlying tissue volumes including the identified target volume and the one or more other tissue volumes; identifying, by the processor, one or more auxiliary tissue volumes within the 3D anatomical image using one or more auxiliary modules, each auxiliary tissue volume corresponding to an underlying tissue volume identified by the second module; 62. The method of any one of items 55 to 61, further comprising: merging, by the processor, each auxiliary tissue volume with the corresponding underlying tissue volume identified by the second module. (Item 63) (e) displaying, by the processor, an interactive graphical user interface (GUI) for presenting a visual representation of the 3D anatomical image to the user; (f) providing, by the processor, within the GUI a graphical rendering of the 3D anatomical image together with selectable and superimposable segmentation layers comprising one or more identified specific tissue volumes within the 3D anatomical image, wherein upon selection of the segmentation layer for display, graphics representing the one or more identified specific tissue volumes are superimposed on the 3D anatomical image. (Item 64) Item 64. The method of item 63, wherein the one or more specific tissue volumes include the identified target volume. (Item 65) Item 65. The method of item 63 or 64, comprising, in step (f), rendering the 2D cross-sectional view of the 3D anatomical image within an interactive 2D viewer such that the position of the 2D cross-sectional view can be adjusted by the user. (Item 66) 66. The method of any one of items 63 to 65, comprising in step (f) rendering an interactive 3D view of the 3D anatomical image. (Item 67) 67. A method according to any one of items 55 to 66, comprising performing steps (a) and (b) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each of which is within one of the plurality of 3D anatomical images, wherein the variability in size of the initial VOIs is less than the variability in size of the 3D anatomical images. (Item 68) 68. The method of any one of the preceding items 55 to 67, wherein the first module is a CNN module that receives as input a downsampled version of the 3D anatomical image having a first resolution and operates on the same, and the second module is a CNN module that receives as input a high-resolution version of the 3D anatomical image cropped to the initial VOI and having a second resolution higher than the first resolution and operates on the same. (Item 69) 69. The method of any one of items 55 to 68, wherein the first module receives as input at least a portion of the 3D anatomical image, and wherein the physical volume represented by the input to the first module is (i) at least two times larger than the physical volume represented by the initial VOI, and / or (ii) two times larger along at least one dimension than the physical volume represented by the initial VOI. (Item 70) 70. The method of any one of items 55 to 69, wherein the first module is a CNN module trained to identify a graphical representation of a specific anatomical region within a 3D anatomical image, and the second module is a CNN module trained to identify a graphical representation of the target tissue region within a 3D anatomical image. (Item 71) receiving, by the processor, a 3D functional image of the object obtained using a functional imaging diagnostic technique, wherein a plurality of voxels included in the 3D functional image each represent a specific physical volume within the object and have intensity values ​​representative of detected radiation emanating from the specific physical volume, and at least a portion of the plurality of voxels in the 3D functional image represent a physical volume within the specific anatomical region; 71. The method of any one of items 55 to 70, further comprising identifying within the 3D functional image a 3D volume that corresponds to the target volume identified using the 3D segmentation mask. (Item 72) 72. The method of claim 71, wherein the voxels of the 3D functional image are related to the voxels of the 3D anatomical image by a known relationship. (Item 73) 1. A system for automatically processing a 3D image to identify a 3D volume within said 3D image that corresponds to a target tissue region, comprising: a processor of a computing device; and a memory storing instructions, wherein the processor executes the instructions to: (a) receiving a 3D anatomical image of the object obtained using an anatomical imaging modality, the 3D anatomical image including a graphical representation of an internal structure of the object; (b) using a first module to determine an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to a particular anatomical region that includes the target region; (c) using a second module to identify a target volume within the initial VOI that corresponds to the target tissue region of the subject; (d) storing and / or providing a 3D segmentation mask corresponding to the identified target volume for display and / or further processing.

[0083] The patent or application file contains at least one color drawing. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0084] The foregoing and other objects, aspects, features, and advantages of the present disclosure will become more apparent and be better understood by referring to the following description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0085] [Figure 1] FIG. 1 is a block diagram illustrating a process for automatically identifying a 3D volume within a 3D image corresponding to a subject's prostate gland and determining an uptake metric representative of the uptake of a radiopharmaceutical therein, according to an exemplary embodiment.

[0086] [Figure 2A] FIG. 2A is an image showing a 3D view of a CT image including a graphical representation of soft tissue, according to an exemplary embodiment.

[0087] [Figure 2B] FIG. 2B is an image showing a 3D view of a CT image including a graphical representation of bones, according to an example embodiment.

[0088] [Figure 2C] FIG. 2C is an image showing a 3D view of a CT image including a SPECT image and graphics representing an identified tissue volume corresponding to the prostate organ inside the subject's pelvic bone superimposed on a graphical representation of the bone, according to an exemplary embodiment.

[0089] [Figure 2D]FIG. 2D is an image showing a 3D view of a CT image including a SPECT image and graphics representing an identified tissue volume corresponding to the prostate organ inside the subject's pelvic bone superimposed on a graphical representation of the bone, according to an exemplary embodiment.

[0090] [Figure 2E] FIG. 2E is an image showing a 3D view of a CT image with a SPECT image superimposed on a graphical representation of bone, according to an exemplary embodiment.

[0091] [Figure 3A] FIG. 3A is a set of images each showing a 2D cross-section of a 3D CT image overlaid with graphics representing an identified initial volume of interest (VOI) within the 3D CT image, according to an exemplary embodiment.

[0092] [Figure 3B] FIG. 3B is a set of images each showing a 2D cross-section of a 3D CT image overlaid with graphics representing an identified initial volume of interest (VOI) within the 3D CT image, according to an exemplary embodiment.

[0093] [Figure 4A] FIG. 4A is a set of images showing 2D cross-sections of a CT image of a subject and a cubic region identified within the CT image as an initial volume of interest, according to an example embodiment.

[0094] [Figure 4B] FIG. 4B is a set of images showing 2D cross-sections of a CT image of a subject along with identified tissue volumes corresponding to the subject's pelvic bone and prostate gland.

[0095] [Figure 5A] FIG. 5A is a set of images showing 2D cross-sections of a CT image of a subject and a cubic region identified within the CT image as an initial volume of interest, according to an example embodiment.

[0096] [Figure 5B] FIG. 5B is a set of images showing 2D cross-sections of a CT image of a subject along with identified tissue volumes corresponding to the subject's pelvic bone and prostate gland.

[0097] [Figure 6A] FIG. 6A is a set of images showing 2D cross-sections of a CT image of a subject and a cubic region identified within the CT image as an initial volume of interest, according to an example embodiment.

[0098] [Figure 6B-C] 6B and 6C are a set of images each showing a 2D cross-section of a 3D CT image overlaid with graphics representing an identified initial volume of interest (VOI) within the 3D CT image, according to an exemplary embodiment.

[0099] [Figure 7A] 7A-7E are block diagrams of a CNN module architecture (localization network) for identifying a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of a subject (the VOI is subsequently processed by a second CNN module for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region), according to an example embodiment. [Figure 7B] 7A-7E are block diagrams of a CNN module architecture (localization network) for identifying a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of a subject (the VOI is subsequently processed by a second CNN module for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region), according to an example embodiment. [Figure 7C]7A-7E are block diagrams of a CNN module architecture (localization network) for identifying a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of a subject (the VOI is subsequently processed by a second CNN module for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region), according to an example embodiment. [Figure 7D] 7A-7E are block diagrams of a CNN module architecture (localization network) for identifying a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of a subject (the VOI is subsequently processed by a second CNN module for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region), according to an example embodiment. [Figure 7E] 7A-7E are block diagrams of a CNN module architecture (localization network) for identifying a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of a subject (the VOI is subsequently processed by a second CNN module for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region), according to an example embodiment.

[0100] [Figure 7F] 7F-7J are block diagrams of a CNN module architecture (segmentation network) for processing previously identified VOIs for precise segmentation of the prostate and / or other tissues within the pelvic region, according to an exemplary embodiment. [Figure 7G] 7F-7J are block diagrams of a CNN module architecture (segmentation network) for processing previously identified VOIs for precise segmentation of the prostate and / or other tissues within the pelvic region, according to an exemplary embodiment. [Figure 7H]7F-7J are block diagrams of a CNN module architecture (segmentation network) for processing previously identified VOIs for precise segmentation of the prostate and / or other tissues within the pelvic region, according to an exemplary embodiment. [Figure 7I] 7F-7J are block diagrams of a CNN module architecture (segmentation network) for processing previously identified VOIs for precise segmentation of the prostate and / or other tissues within the pelvic region, according to an exemplary embodiment. [Figure 7J] 7F-7J are block diagrams of a CNN module architecture (segmentation network) for processing previously identified VOIs for precise segmentation of the prostate and / or other tissues within the pelvic region, according to an exemplary embodiment.

[0101] [Figure 8A] FIG. 8A is a set of images each showing a 2D cross-section of a 3D CT image of three different pelvic bones (left hip bone, right hip bone, and sacrum) with graphics representing corresponding identified tissue volumes overlaid, according to an exemplary embodiment.

[0102] [Figure 8B] FIG. 8B is an image showing a 3D view of a 3D CT image with graphics representing identified tissue volumes corresponding to three different pelvic bones (left hip bone, right hip bone, and sacrum) overlaid, according to an exemplary embodiment.

[0103] [Figure 9A] FIG. 9A is an image showing a 2D cross-section of a 3D CT image with graphics representing corresponding identified tissue volumes overlaid on the subject's pelvic bones (left and right hip bones) and prostate gland, according to an exemplary embodiment.

[0104] [Figure 9B]FIG. 9B is a set of images showing separate 2D cross-sections of a 3D CT image with graphics representing corresponding identified tissue volumes overlaid on the subject's pelvic bones, gluteal muscles, rectum, prostate, and bladder, according to an exemplary embodiment.

[0105] [Figure 10] FIG. 10 is an image showing a 2D cross section of a low quality 3D CT image with graphics representing corresponding identified tissue volumes overlaid on the pelvic bone and prostate of a subject, according to an exemplary embodiment.

[0106] [Figure 11] FIG. 11 is a schematic diagram illustrating an example architecture in which image segmentation for prostate volume identification as described herein is performed by a dedicated module, according to an example embodiment.

[0107] [Figure 12] FIG. 12 is a block flow diagram illustrating an example architecture of modules for performing CNN-based image segmentation, according to an example embodiment.

[0108] [Figure 13] FIG. 13 is a block flow diagram illustrating the structure of a CNN according to an exemplary embodiment.

[0109] [Figure 14] FIG. 14 is a block flow diagram illustrating the structure of a CNN that performs auxiliary prediction, according to an exemplary embodiment.

[0110] [Figure 15] FIG. 15 is a schematic diagram illustrating crosstalk between the bladder and prostate of a subject, in accordance with an exemplary embodiment.

[0111] [Figure 16A]FIG. 16A is a screenshot of a graphical user interface (GUI) for reviewing patient image data, showing a window for selecting data to analyze and / or review, according to an exemplary embodiment.

[0112] [Figure 16B] FIG. 16B is a screenshot of a graphical user interface (GUI) for reviewing patient image data, showing a window for selecting what data to analyze and / or review, along with graphical control elements for initiating processing and reviewing the patient image data, according to an exemplary embodiment.

[0113] [Figure 16C] FIG. 16C is a screenshot of a graphical user interface (GUI) for reviewing patient image data, showing a window for selecting what data to analyze and / or review, along with graphical control elements for initiating processing or reviewing the patient's image data, according to an exemplary embodiment.

[0114] [Figure 17A] FIG. 17A is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-section of the subject's CT image with overlaid graphics representing the subject's SPECT image and identified tissue volumes, according to an exemplary embodiment.

[0115] [Figure 17B] FIG. 17B is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-section of the subject's CT image with overlaid graphics representing the subject's SPECT image and identified tissue volumes, according to an exemplary embodiment.

[0116] [Figure 17C]FIG. 17C is a screenshot of a GUI for reviewing a patient's image data, showing a window containing a set of images, each showing a separate 2D cross-sectional view of the subject's CT image with overlaid graphics representing the subject's SPECT image and identified tissue volumes, according to an exemplary embodiment.

[0117] [Figure 17D] FIG. 17D is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a separate 2D cross-sectional view of the subject's CT image with overlaid graphics representing the subject's SPECT image and identified tissue volumes, according to an exemplary embodiment.

[0118] [Figure 17E] FIG. 17E is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a separate 2D cross-section of the subject's CT image with overlaid graphics representing the subject's SPECT image and identified tissue volumes, according to an exemplary embodiment.

[0119] [Figure 18A] FIG. 18A is a screenshot of a GUI for reviewing patient image data showing a window including graphical control elements for toggling the display of selectable layers, illustrating how a user can toggle the display of a SPECT image layer, in accordance with an exemplary embodiment.

[0120] [Figure 18B] FIG. 18B is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image with overlaid graphics representing identified tissue volumes, according to an exemplary embodiment.

[0121] [Figure 18C]FIG. 18C is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image with overlaid graphics representing identified tissue volumes, according to an exemplary embodiment.

[0122] [Figure 18D] FIG. 18D is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image with overlaid graphics representing identified tissue volumes, according to an exemplary embodiment.

[0123] [Figure 18E] FIG. 18E is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image with overlaid graphics representing identified tissue volumes, according to an exemplary embodiment.

[0124] [Figure 18F] FIG. 18F is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image with overlaid graphics representing identified tissue volumes, according to an exemplary embodiment.

[0125] [Figure 19A] FIG. 19A is a screenshot of a GUI for reviewing patient image data showing a window including graphical control elements for toggling the display of selectable layers, illustrating how a user can toggle the SPECT image layer on and the segmentation layer off, in accordance with an exemplary embodiment.

[0126] [Figure 19B]FIG. 19B is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-section of a subject's CT image overlaid with a SPECT image of the subject, in accordance with an exemplary embodiment.

[0127] [Figure 19C] FIG. 19C is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image overlaid with a SPECT image of the subject, in accordance with an exemplary embodiment.

[0128] [Figure 19D] FIG. 19D is a screenshot of a GUI for reviewing a patient's image data showing a window containing a set of images, each showing a different 2D cross-sectional view of a subject's CT image overlaid with a SPECT image of the subject, in accordance with an exemplary embodiment.

[0129] [Figure 20A] FIG. 20A is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of soft tissues overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0130] [Figure 20B] FIG. 20B is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of soft tissues overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0131] [Figure 20C] FIG. 20C is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of the subject's soft tissues overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0132] [Figure 21A] FIG. 21A is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0133] [Figure 21B] FIG. 21B is a screenshot of a GUI for reviewing a patient's image data showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0134] [Figure 22A] FIG. 22A is a screenshot of a GUI for reviewing a patient's image data showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0135] [Figure 22B] FIG. 22B is a screenshot of a GUI for reviewing a patient's image data showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0136] [Figure 22C] FIG. 22C is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of the subject's SPECT image and a CT image of the subject including a graphical representation of bones overlaid with graphics representing identified tissue volumes, in accordance with an exemplary embodiment.

[0137] [Figure 22D]FIG. 22D is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of the subject's SPECT image and the subject's CT image including a graphical representation of bones overlaid with graphics representing identified tissue volumes, in accordance with an exemplary embodiment.

[0138] [Figure 22E] FIG. 22E is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of the subject's SPECT image and a CT image of the subject including a graphical representation of bones overlaid with graphics representing identified tissue volumes, in accordance with an exemplary embodiment.

[0139] [Figure 22F] FIG. 22F is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of the subject's SPECT image and a CT image of the subject including a graphical representation of bones overlaid with graphics representing identified tissue volumes, in accordance with an exemplary embodiment.

[0140] [Figure 23A] FIG. 23A is a screenshot of a GUI for reviewing a patient's image data showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0141] [Figure 23B] FIG. 23B is a screenshot of a GUI for reviewing a patient's image data showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0142] [Figure 24]FIG. 24 is a screenshot of a GUI for reviewing patient image data showing a window containing a report generated for a subject, according to an exemplary embodiment.

[0143] [Figure 25] FIG. 25 is a block flow diagram illustrating a workflow for user interaction with a GUI in which the user reviews images, segmentation results, and intake metrics to generate a report, according to an exemplary embodiment.

[0144] [Figure 26A] FIG. 26A is a screenshot of a view of a GUI window that allows a user to upload an image, according to an exemplary embodiment.

[0145] [Figure 26B] FIG. 26B is a screenshot of a view of a GUI window showing a list of images uploaded by a user, according to an exemplary embodiment.

[0146] [Figure 27A-B] 27A and 27B are screenshots of a view of a GUI window showing a list of patients, according to an exemplary embodiment.

[0147] [Figure 27C] FIG. 27C is a screenshot of a view of a GUI window showing a selected patient and a menu giving the user options to review test data and generate reports, according to an exemplary embodiment.

[0148] [Figure 28A]FIG. 28A is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of a SPECT image of the subject and a CT image of the subject including a graphical representation of bones overlaid with graphics representing identified tissue volumes, in accordance with an exemplary embodiment.

[0149] [Figure 28B] FIG. 28B is a screenshot of a GUI for reviewing a patient's image data, showing a window containing an image showing a 3D view of a subject's CT image including a graphical representation of bones overlaid with a SPECT image of the subject, according to an exemplary embodiment.

[0150] [Figure 29A] FIG. 29A is a screenshot of a GUI for reviewing patient image data showing a window for reviewing determined uptake metrics, according to an exemplary embodiment.

[0151] [Figure 29B] FIG. 29B is a screenshot of a GUI for reviewing patient image data showing a quality control graphical widget, according to an exemplary embodiment.

[0152] [Figure 29C] FIG. 29C is a screenshot of a view of a quality control graphical widget that allows a user to sign a generated report, according to an exemplary embodiment.

[0153] [Figure 29D] FIG. 29D is a screenshot showing a generated report (eg, an automatically generated report) according to an exemplary embodiment.

[0154] [Figure 29E-H]Figure 29E is a screenshot of a view of a quality control graphical widget displayed in response to a user input of disapproval of the automatic determination of intake metrics, according to an exemplary embodiment. Figure 29F is a screenshot of a portion of a generated report showing a graphical indication of a quality control failure, according to an exemplary embodiment. Figure 29G is a screenshot of a quality control graphical widget that allows a user to manually update one or more values ​​used in determining intake metrics, according to an exemplary embodiment. Figure 29H is a screenshot of a quality control graphical widget that allows a user to manually update one or more values ​​used in determining intake metrics, according to an exemplary embodiment.

[0155] [Figure 30] FIG. 30 is a block diagram illustrating a microservices network architecture for performing image segmentation, identifying a prostate volume within an image, determining intake metrics, and providing results to a client, according to an example embodiment.

[0156] [Figure 31] FIG. 31 is a block flow diagram illustrating data flow between microservices of a cloud-based application for processing tests by performing image segmentation and determining intake metrics, according to an example embodiment.

[0157] [Figure 32] FIG. 32 is a block flow diagram illustrating communication between a cloud-based application and a client's microservices, according to an example embodiment.

[0158] [Figure 33]FIG. 33 is a block diagram of an example architecture for implementing a cloud-based platform including a cloud-based application for performing image segmentation and calculating intake metrics according to the systems and methods described herein, in accordance with an example embodiment.

[0159] [Figure 34] FIG. 34 is a block diagram illustrating a process for automatically identifying a 3D target volume within a 3D image and determining an uptake metric indicative of the uptake of a radiopharmaceutical therein, according to an exemplary embodiment.

[0160] [Figure 35] Figure 35A is a swarm plot of clinically insignificant and clinically significant images, and Figure 35B is a ROC curve determined based on varying the TBR threshold calculated by the systems and methods described herein.

[0161] [Figure 36] FIG. 36 is a block diagram of an exemplary cloud computing environment for use in certain embodiments.

[0162] [Figure 37] FIG. 37 is a block diagram of an example computing device and an example mobile computing device used in certain embodiments.

[0163] [Figure 38A] FIG. 38A is a screenshot of a GUI used in a conventional two-dimensional slice-based image analysis software package.

[0164] [Figure 38B] FIG. 38B is a screenshot of a portion of a GUI used in a conventional two-dimensional slice-based image analysis software package showing a CT image slice.

[0165] [Figure 38C] FIG. 38C is a screenshot of a portion of a GUI used in a conventional two-dimensional slice-based image analysis software package showing a CT image slice.

[0166] [Figure 39] FIG. 39 is a block flow diagram illustrating a workflow of AI-assisted image analysis, according to an exemplary embodiment.

[0167] [Figure 40A] FIG. 40A is a screenshot of a GUI window prompting completion of automatic analysis after uploading a SPECT / CT image, according to an exemplary embodiment.

[0168] [Figure 40B] FIG. 40B is a screenshot of a GUI window indicating potential errors in the automated analysis and the need for physician review after uploading a SPECT / CT image, according to an exemplary embodiment.

[0169] [Figure 40C] FIG. 40C is a screenshot of a GUI window indicating a failure to complete an automated analysis due to an incomplete data set, according to an exemplary embodiment.

[0170] [Figure 41] FIG. 41 is a screenshot of an image analysis GUI window showing a list of patients, according to an exemplary embodiment.

[0171] [Figure 42A] Figure 42A is a screenshot of an image analysis GUI for reviewing patient data showing a 2D slice view of a subject's CT image including SPECT image data of the subject's prostate and a graphical representation of bone and tissue overlaid with segmentation, in accordance with an exemplary embodiment.

[0172] [Figure 42B] FIG. 42B is a screenshot of a GUI for reviewing patient data showing a 3D view of a CT image including SPECT image data of a subject's prostate and a graphical representation of tissue and bone overlaid with segmentation, according to an exemplary embodiment.

[0173] [Figure 42C] FIG. 42C is a screenshot of a portion of a GUI for reviewing patient data showing a likelihood importance scale displayed within the GUI in one specific embodiment.

[0174] [Figure 42D] FIG. 42D is a screenshot of a GUI for reviewing patient data and allowing the viewer to update values ​​used in TBR calculations in a semi-automated, guided manner, according to an exemplary embodiment.

[0175] [Figure 42E] FIG. 42E is a screenshot of a GUI for reviewing patient data showing updated assessment results after viewer input to update values ​​used in TBR calculations, according to an exemplary embodiment.

[0176] [Figure 43A] FIG. 43A is a screenshot showing a generated report including fully automatically determined results, according to an exemplary embodiment.

[0177] [Figure 43B] FIG. 43B is a screenshot showing the generated report, including the semi-automatically determined results.

[0178] [Figure 44] FIG. 44 is a screenshot illustrating a report generated for an unassessable case, according to an exemplary embodiment.

[0179] [Figure 45A] FIG. 45A is a swarm plot of clinically insignificant and clinically significant images with three TBR thresholds overlaid.

[0180] [Figure 45B] FIG. 45B is a ROC curve determined based on varying the TBR threshold calculated by the systems and methods described herein, with three different TBR thresholds identified.

[0181] [Figure 45C] Figure 45C is a graph showing the sum of Gleason scores and calculated TBR values ​​for several patients.

[0182] [Figure 45D] FIG. 45D is a table showing the Gleason scores of cases classified according to their TBR values ​​compared to several predetermined thresholds.

[0183] [Figure 46] Figure 46 is a set of three histograms for three independent AI-assisted readers showing the time it took to evaluate patient data from a clinical trial.

[0184] [Figure 47] FIG. 47 is a set of graphs demonstrating the consistency and reproducibility of results obtained using one embodiment of the image analysis techniques described herein for analyzing clinical trial data.

[0185] [Figure 48A] FIG. 48A is a graph comparing the ROC curves of the results obtained by three independent AI-assisted readers and three independent manual readers.

[0186] [Figure 48B]FIG. 48B is a set of histograms showing AUC values ​​for three independent manual viewers and three independent AI-assisted viewers.

[0187] [Figure 49A-B] Figure 49A is a graph showing data comparing the results of a first manual reader with the results of an AI-assisted reader, and Figure 49B is a graph showing data comparing the results of a second manual reader with the results of an AI-assisted reader.

[0188] [Figure 49C] FIG. 49C is a graph showing data comparing the results of a third manual reader with the results of an AI-assisted reader. DETAILED DESCRIPTION OF THE INVENTION

[0189] The features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numbers generally indicate identical, functionally similar, and / or structurally similar elements.

[0190] The systems, architectures, devices, methods, and processes of the claimed inventions are intended to encompass variations and adaptations developed using information from the embodiments described herein. As contemplated by this description, adaptations and / or modifications of the systems, architectures, devices, methods, and processes described herein may be accomplished.

[0191] Throughout the description, when articles, devices, systems, and architectures are described as having, including, or comprising particular components, or when processes and methods are described as having, including, or comprising particular steps, it is contemplated that there are additionally articles, devices, systems, and architectures of the invention that consist of, or consist essentially of, the recited components, and processes and methods of the invention that consist of, or consist essentially of, the recited processing steps.

[0192] It should be understood that the order for performing steps or certain actions is immaterial so long as the invention remains operable. Moreover, two or more steps or actions may be conducted simultaneously.

[0193] The citation of any publication herein, for example in the Background section, is not an admission that the publication serves as prior art to any of the claims presented herein. The Background section is presented for clarity and is not intended as a description of prior art to any claim.

[0194] As indicated, documents are incorporated herein by reference. In the event of any discrepancy in the meaning of a particular term, the meaning given in the Definitions section above shall control.

[0195] Headings are provided for the convenience of the reader, but the presence and / or placement of headings is not intended to limit the scope of the subject matter described herein.

[0196] "Radionuclide," as used herein, refers to a moiety that contains a radioactive isotope of at least one element. Exemplary suitable radionuclides include, but are not limited to, those described herein. In some embodiments, the radionuclide is a radionuclide used in positron emission tomography (PET). In some embodiments, the radionuclide is a radionuclide used in single photon emission computed tomography (SPECT). In some embodiments, a non-limiting list of radionuclides is: 99m Tc, 111 In, 64 Cu, 67 Ga, 68 Ga, 186 Re, 188 Re, 153 Sm, 177 Lu, 67 Cu, 123 I, 124 I, 125 I, 126 I, 131 I, 11 C. 13 N, 15 O. 18 F, 153 Sm, 166 Ho, 177 Lu, 149 Pm, 90 Y, 213 Bi, 103 Pd, 109 Pd, 159 Gd, 140 La, 198 Au, 199 Au, 169 Yb, 175 Yb, 165 Dy, 166 Dy, 105 Rh, 111 Ag, 89 Zr, 225 Ac, 82 Rb, 75 Br, 76 Br, 77 Br, 80 Br, 80m Br, 82 Br, 83 Br, 211 At and 192Contains Ir.

[0197] The term "radiopharmaceutical," as used herein, refers to a compound containing a radionuclide. In certain embodiments, radiopharmaceuticals are used for diagnosis and / or treatment. In certain embodiments, radiopharmaceuticals include small molecules labeled with one or more radionuclides, antibodies labeled with one or more radionuclides, and antigen-binding portions of antibodies labeled with one or more radionuclides.

[0198] "3D" or "three-dimensional" with reference to an "image," as used herein, means conveying information relating to three dimensions. A 3D image may be rendered as a three-dimensional data set and / or may be displayed as a set of two-dimensional or three-dimensional representations.

[0199] An "image," e.g., a 3D image of an object, as used herein includes any visual representation, such as a photograph, a video frame, a stream of video, and any electronic, digital, or mathematical version of a photograph, video frame, or stream of video. Any apparatus described herein, in certain embodiments, includes a display for displaying an image or some other result generated by the processor. Any method described herein, in certain embodiments, includes displaying an image or some other result generated by the method.

[0200] "Subject," as used herein, means a human or other mammal (e.g., rodents (mice, rats, hamsters), pigs, cats, dogs, horses, primates, rabbits, etc.).

[0201] "Administering" an agent, as used herein, means introducing a substance (e.g., an imaging agent) into a subject. Generally, any route of administration may be utilized, including, for example, parenteral (e.g., intravenous), oral, topical, subcutaneous, peritoneal, intra-arterial, inhalation, vaginal, rectal, nasal, introduction into the cerebrospinal fluid, or infusion into a body compartment.

[0202] The terms "filter" and "filtering," as in "filtering function" or "filter," as used herein, refer to a function that operates on a local portion of an input array (e.g., a multidimensional array) of data (e.g., image data, e.g., values ​​computed by a layer of a CNN) to compute a response value for a given subpatch, referred to herein as a "subpatch." Generally, to compute multiple response values ​​for an array, a filter is applied in a sliding window manner across the array. In particular, for a given multidimensional array, a subpatch of the array may be a rectangular region of the array having a particular size (e.g., having the same number of dimensions as the array). For example, for a 6x3x3 array, a given 3x3x3 subpatch refers to a given 3x3x3 set of adjacent values ​​(e.g., neighborhood) of the array (each patch shifted one position along the first dimension), such that there are five distinct 3x3x3 subpatches in the 6x3x3 array.

[0203] For example, a filtering function can calculate a response value for a given subpatch of the array using the values ​​of the subpatch. The filtering function can be applied in a sliding window manner across the array to calculate a response value for each of multiple subpatches of the array. The calculated response values ​​can be stored in an output array such that a positional correspondence between the response values ​​and the subpatches of the input array is maintained.

[0204] For example, in a first step, the filter may start with a corner subpatch of the input array, calculate a first response value, and store the first response value in the corresponding corner of the output array. In a particular embodiment, then, in a second step, the filter is shifted one position along a particular dimension of the input array and calculates a second response value for the second subpatch. The second response value may be stored in a corresponding position in the output array, i.e., a position shifted one position along the same dimension of the output array. The steps of shifting the position of the subpatch, calculating response values, and storing the response values ​​in the corresponding position in the output array may be repeated for the entire input array along each dimension of the input array. In a particular embodiment (e.g., a strided filtering approach), the subpatch for which the filter calculates response values ​​is shifted more than one position at a time along a given dimension, and response values ​​are not calculated for all possible subpatches of the input array.

[0205] The term "convolutional neural network" (CNN), as used herein, refers to a type of artificial neural network in which at least one layer performs one or more filtering functions. The term "convolutional layer," as used herein, refers to a layer of a CNN that receives an input array as input and computes an output array, the values ​​of which are computed by applying one or more filters to the input array. In particular, a convolutional layer, in certain embodiments, receives an input array having n+1 dimensions as input and generates an output array that also has n+1 dimensions. The first n-dimensional input array and output array operated on by a filtering layer of a CNN are referred to herein as "spatial dimensions." The (n+1)th dimension of the input is referred to herein as the dimension of the "input channel." The size of the dimension of the input channel is referred to herein as the "number of input channels." The (n+1)th dimension of the output is referred to herein as the dimension of the "output channel." The size of the dimension of the input channel is referred to herein as the "number of output channels."

[0206] In certain embodiments, the convolutional layer computes response values by applying filters that operate on sub-patches, where the sub-patches are smaller than the input array along the spatial dimensions but span across the dimensions of all output channels. For example, an input array of size N×M×L×K0 has three spatial dimensions and K0 output channels. The filters of the convolutional layer can operate on sub-patches of size N f ×M f ×L f ×K0, where N f ≦N, M f ≦M and L f ≦L. In many cases, the filters of the convolutional layer operate on sub-patches of size N f <N, M f <M and / or L f <L. For example, in certain embodiments, N f <<N, M f <<M and / or L f <<L.

[0207] Thus, for each of the one or more filters applied by the convolutional layer, the response values computed by a given filter are stored in the corresponding output channel. Thus, a convolutional layer that receives an input array having n + 1 dimensions also computes an output array having n + 1 dimensions, where the (n + 1)-th dimension represents the output channels corresponding to the one or more filters applied by the convolutional layer. In this way, the output array computed by a given convolutional layer can be received as input by a subsequent convolutional layer.

[0208] The term "size," as used herein, in relation to a filter in a convolutional layer refers to the size along a spatial dimension of the subpatch on which the filter operates (e.g., the size of a subpatch along the output channel dimension is interpreted as a constant for the output channel). As used herein, the term "size," as used herein, in relation to a convolutional layer, such as "size of a convolutional layer," refers to the size of the filters in the convolutional layer (e.g., each filter having the same size in the convolutional layer). In certain embodiments, the filters in the convolutional layer have multiple variable parameters determined by a machine learning training process. In certain embodiments, the number of parameters for a given filter is equal to the number of values ​​in the subpatch on which the given filter operates. For example, a filter of size N operating on an input array with K output channels may be used. f ×M f ×L f The filter is N f ×M f ×L f ×K0. In one particular embodiment, the filter is implemented as an array, and the response value determined by the filter for a given subpatch is calculated as the dot product between the filter and the given subpatch.

[0209] The term "fully convolutional neural network" (FCNN), as used herein, refers to a CNN in which each layer of the CNN is a convolutional layer.

[0210] The term "volume," as used herein in reference to the input or output of a layer of a CNN, refers to the input array received by the CNN layer or the output array computed by the CNN layer.

[0211] The term "CNN module," as used herein, refers to a computer-implemented process that implements a particular CNN to determine one or more output values ​​for a given input, such as an image (e.g., a 2D image, e.g., a 3D image). For example, a CNN module receives as input a 3D image of a subject (e.g., a CT image, e.g., an MRI) and determines, for each voxel in the image, a value that represents the likelihood that the voxel is within a region of the 3D image that corresponds to a representation of a particular organ or tissue of the subject. A CNN module can be software and / or hardware. For example, a CNN module may be implemented entirely as software, or certain functions of the CNN module may be performed by specialized hardware (e.g., an application-specific integrated circuit (ASIC)).

[0212] The term "tissue," as used herein, refers to bone (bony tissue) as well as soft tissue.

[0213] The systems and methods described herein provide for automated analysis of medical images of a subject to automatically identify regions of interest corresponding to specific organs and / or tissues depicted in the images. In certain embodiments, a convolutional neural network (CNN) module is employed to accurately segment the images. In certain embodiments, the accurate automatic identification of specific organs and / or tissues in the subject's images allows quantitative metrics measuring the uptake of a radiopharmaceutical (e.g., a radiolabeled small molecule, e.g., a radiolabeled antibody, e.g., a radiolabeled antigen-binding portion of an antibody) in the specific organ to be determined.

[0214] In certain embodiments, the systems and methods described herein facilitate automated identification of regions of interest corresponding to specific organs or tissues where tumors and / or tumor metastases may reside. Radionuclide-labeled molecules that selectively bind to specific tumor cell surface proteins may be utilized in conjunction with the tumor imaging techniques described herein. For example, molecule 1404 specifically binds to prostate-specific membrane antigen (PSMA), which is overexpressed in many cancer cells. Molecule 1404 may be used in single-photon emission computed tomography (SPECT) imaging. 99m It may be labeled with a radionuclide such as Tc.

[0215] In certain embodiments, the uptake metrics of the radiopharmaceutical are relevant to assessing the disease state and prognosis of the subject. A. Prostate volume identification and uptake metric determination

[0216] The process 100 shown in FIG. 1 is for automatically processing 3D images to identify a 3D volume corresponding to the prostate gland of a subject and determine one or more uptake metrics representative of the uptake of a radiopharmaceutical therein.

[0217] See steps 102 and 104 of process 100 in FIG. 1. Various panels of FIG. 2 illustrate CT images (FIGS. 2A and 2B) showing bone and soft tissue, highlighting the pelvic bones (green 202 and blue 206 in FIGS. 2C and 2D) and the prostate (purple 204 in FIGS. 2C and 2D). The highlighted regions were automatically generated by the systems and methods described herein and identify the pelvic bones and prostate. FIG. 2E shows a SPECT image (light pink and purple regions 252, 254, and 256) superimposed on the CT image. As can be seen, the CT image provides detailed anatomical information, while the anatomical / structural information in the SPECT image is more limited. Therefore, it is possible to segment the CT image and use the mapping between the voxels of the CT image and the voxels of the SPECT image to identify specific voxels in the SPECT image that correspond to volumes of tissue of interest, such as the prostate, bladder, rectum, and gluteal muscles.

[0218] See steps 106 and 108 of process 100 in FIG. 1 . The approach for identifying a prostate volume within a CT image first uses a first machine learning module (e.g., a CNN module) to identify an initial volume of interest within the CT image. This generates standardized input for a second machine learning module (e.g., a second CNN module) that is responsible for identifying the prostate volume. The second CNN module may also identify other tissue volumes, such as the pelvic bones, gluteal muscles, rectum, and bladder. The first and second machine learning modules may be implemented in combination as a single module and / or a single software application. The first and second machine learning modules may also be implemented separately, for example, as separate software applications.

[0219] In certain embodiments, the architecture (e.g., layer types and organization) of the CNN implemented by the first machine learning module and the architecture of the CNN implemented by the second machine learning module are identical as described herein, but parameters such as the number of filters in various layers and input size may differ between the two CNNs. In addition, the two CNNs are trained to perform different functions. Specifically, the first CNN is trained to recognize graphical representations of a specific anatomical region, such as the pelvic region, within various 3D anatomical images. The second CNN is trained to recognize a specific target tissue, such as the prostate, within a more standardized input (initial target volume). Training the two CNNs differently in this manner generates different weights and biases for the filters that make up each CNN, enabling each CNN to perform a different function. i. Bounding box generation (localization machine)

[0220] Figures 3-5 present examples of the processing of various CT images, first showing the identification of the initial volume of interest (VOI) (Panel A), and second showing the segmentation of the prostate and pelvic bones (the prostate volume is represented in the center by the blue-green volume). As shown in these examples, the size of the initial CT image can depend substantially on the imaging system used, the imaging protocol followed by the radiologist, and other factors, but the initial VOI size is fairly standard.

[0221] For example, the size of the initial 3D anatomical image may range from 700 mm × 700 mm × 1870 mm (depth × width × height) (i.e., a full-body scan) or larger to 180 mm × 240 mm × 290 mm (including the pelvic bones) or smaller. In the examples described herein, the minimum size of the tested matrix of the initial 3D anatomical image is 256 × 256 × 76 pixels. The range of sizes of the bounding boxes (e.g., identified by the first CNN) for the examples described herein is approximately 180-220 mm × 240-320 mm × 290-380 mm. The range of bounding box matrix sizes for the examples described herein is 80-220 × 150-390 × 50-300 pixels.

[0222] In this example, the 3D anatomical images to train the first CNN to detect bounding boxes for the pelvic region had the following image dimensions: Training images (3D anatomical images): Number of lines: 247-512 (sp 1.37), size (mm): 340-700 Number of rows: 319~512 (sp 1.37), size (mm): 430~700 Number of slices: 274-624 (sp 3.0), size (mm): 820-1870

[0223] In this example, the 3D anatomical images for validating the first CNN for bounding box detection have the following image dimensions, resulting in a range of bounding box dimensions identified by the first CNN: Confirmation image (3D anatomical image): Number of lines: 256-512, Size (mm): 500-600 Number of rows: 256-512, Size (mm): 500-600 Number of slices: 76-427, Size (mm): 380-1070 Bounding box: Number of lines: 82-222, Size (mm): 180-220 Number of rows: 146-386, Size (mm): 240-320 Number of slices: 50-295, Size (mm): 290-380

[0224] Below are three example methods for automatically generating a bounding box (cuboid) for the pelvic region from an initial 3D anatomical image to be used in subsequent processing to identify the prostate in detail.

[0225] In the first approach, the first CNN takes a grayscale CT image (one input channel) as input and outputs the coordinates of opposite corners of a bounding box.

[0226] In the second approach, the grayscale CT image is processed by thresholding to generate a thresholded image with a rough identification of the pelvic region. In this second approach, the first CNN receives two input channels: the grayscale CT image and the thresholded image. The second CNN outputs the coordinates of opposite corners of the bounding box.

[0227] In the third approach, the first CNN is essentially a coarse-grained version of the second CNN; that is, the first CNN identifies the prostate, pelvic bone, and sacrum (and background). A bounding box is generated using the identified pelvic bone (e.g., by taking the smallest box that fits the pelvic bone, or perhaps adding some buffer distance). The difference is that the output of the first CNN is not simply the coordinates of the vertices of a cuboid. In this approach, a likelihood value is automatically determined for each voxel in the image indicating the likelihood of how the voxel will be classified, e.g., whether the voxel is prostate, left / right pelvic bone, sacrum, or background.

[0228] In one exemplary embodiment of this third approach, the localization network and the segmentation network are similar. The localization network segments the left pelvic bone, right pelvic bone, sacrum, and background (four classes in total) in a heavily downsampled image. A bounding box is generated based on this coarse segmentation of the pelvic bone. The segmentation network then segments the pelvic bone, prostate, and background within this bounding box at a higher resolution. The localization CNN and the segmentation CNN have identical architectures. The difference lies in the required input shape: the localization network resizes the image to size (81, 68, 96), while the segmentation CNN resizes the pelvic image (as output from the localization network) to size (94, 138, 253). The number of convolutional filters also differs: the segmentation network has 20 convolutional filters in the first layer, while the localization network has only eight. The size and number of filters in subsequent layers are scaled as shown in the flow charts herein. This method finds the corners of the first segmentation and extracts a square with a margin around it to generate a bounding box after the localization network. The input image to the localization network is small because it has been downsampled to a lower resolution, but the segmentation operates on a cropped version of the CT image at the original resolution.

[0229] Figure 6A shows an example of Method 1. Figures 6B and 6C show a comparison between Method 1 and Method 3.

[0230] 7A-7E illustrate an example architecture 700 of a first CNN network described herein. The CNN module architecture (localization network) is used to identify a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of a subject (the VOI is later processed by a second machine learning module (e.g., a second CNN module) for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region). ii. Determination of prostate volume and other tissue volumes (single segmentation machine)

[0231] 7A-7J show an example architecture 750 of the second CNN network described herein. As previously mentioned, the second CNN operates on the VOI to identify the prostate volume as well as various other tissue volumes. The other tissue volumes may include the left / right pelvic bones, sacrum, bladder, gluteal muscles, and rectum. Identifying multiple tissue volumes improves the accuracy of the classification technique, as opposed to performing a binary classification (e.g., where a voxel is simply identified as prostate or background).

[0232] FIG. 8A shows the left pelvic bone (blue 806), right pelvic bone (yellow 802) and sacrum (red 804) in 2D, and FIG. 8B shows the same in 3D.

[0233] FIG. 9A shows the left pelvic bone (red 902), the right pelvic bone (yellow 906) and the prostate gland (blue 904).

[0234] FIG. 9B shows the segmentation of the left and right pelvic bones (two light green regions 952 and 956), the prostate (dark green 958), the bladder (greenish-beige 960), the rectum (brown 962), and the gluteal muscles (blue 964 and red 966).

[0235] FIG. 10 shows a low-quality CT image with segmentation in action. iii. Segmentation Module Architecture Data Input / Output

[0236] In the example architecture 1100 shown in FIG. 11, image segmentation as described herein is performed by a segmentation service module 1104, which receives requests from a client-facing module (Pioneer Web 1102) to process images (e.g., perform segmentation as described herein).

[0237] The specific example architecture 1100 shown in Figure 11 is used in one embodiment of the image segmentation and analysis systems and methods described herein, specifically a cloud-based software application called Pioneer. Pioneer is a software device for assisting in the assessment and characterization of prostate cancer in the prostate using SPECT / CT image data from MIP-1404. The software uses artificial intelligence to automatically segment the image data into distinct anatomical regions and then analyze volumetric regions of interest (ROIs). Pioneer extracts quantitative data from the ROIs to assist in determining the presence or absence of clinically significant prostate cancer.

[0238] The architecture and modular configuration shown in FIG. 11 and other architectures and modular configurations described herein with respect to Pioneer may be adapted for use with other imaging modalities and / or other radiopharmaceuticals. For example, various radiopharmaceuticals suitable for use with SPECT imaging are described herein, below, in Section M, "Imaging Agents." To image the uptake of a radiopharmaceutical in a subject, various 3D functional imaging modalities may be used in combination with 3D anatomical imaging modalities, such as, but not limited to, CT imaging, and may be analyzed by adapted versions of the techniques described herein with respect to Pioneer. For example, various nuclear medicine imaging modalities, such as PET imaging, may be used to image the uptake of a radiopharmaceutical. PET imaging may be performed in combination with CT imaging, as with SPECT imaging, to obtain an image set including a PET image and a CT image (CT / PET image). Thus, the techniques described herein with respect to Pioneer, and CT / SPECT images, may also be adapted for use with CT / PET images. Various radiopharmaceuticals suitable for use with PET imaging are also described herein, below, in Section M, "Imaging Agents."

[0239] Turning again to Figure 11, as shown in Figure 11, Pioneer is implemented as a cloud-based service using Amazon Web Services, where Simple Notification Service (SNS) and Simple Queue Service (SQS) messages are utilized to handle requests. A segmentation service 1104 listens for requests on a dedicated SQS queue. Events raised in Pioneer Web 1102 are published to a dedicated SNS topic "Pioneer Events" 1106. To enable parallel asynchronous processing, a fan-out pattern may be used, where SNS messages are sent to the topic, then replicated and pushed to multiple Amazon SQS queues, 1108a, 1108b, of which the segmentation service is a consumer.

[0240] The segmentation service 1104 may download input data (e.g., image data) from a source uniform resource locator (URL) provided by a requestor (e.g., Pioneer Web 1102) that links to a dataset in a local file system. In certain embodiments, the input data includes pre-processed 3D anatomical images, such as SPECT scans, CT images, and 3D functional images. Image metadata may also be included along with the 3D anatomical and functional images.

[0241] In certain embodiments, the CT and SPECT images have resolutions that meet the specific requirements established for the system. For example, CT images with a resolution range of 1.0-2.2 mm in the x and y directions and a resolution range of 1.0-5.0 mm in the z (slice) direction may be used. In certain embodiments, SPECT images with a resolution range of 2.9-4.8 mm in all directions are used.

[0242] The segmentation service 1104 performs image segmentation to identify the prostate volume and any other tissue volumes in a particular CT image, as described herein, and then provides segmented output data identifying the prostate volume within the particular CT image for storage and / or further processing (e.g., display in a GUI, calculation of uptake metrics, etc.). In certain embodiments, a request issued to the segmentation service 1104 includes a target URL that specifies a storage location of the segmented output data. Thus, the segmentation service 1104 may upload the segmented output data to the target URL included in the request.

[0243] The segmentation output data may include data such as a segmentation mask set that identifies each voxel of the 3D anatomical image (e.g., a CT image) as corresponding to a particular tissue region or background, as determined by the second machine learning module described herein. For each particular tissue region, the segmentation mask set may include, or be used to determine, a corresponding segmentation mask that identifies voxels of the CT image classified as belonging to that particular tissue region. For example, the segmentation mask set may be stored in a .tiled.png format, with voxels labeled with different numerical indicators that identify different particular tissue regions or backgrounds. An example set of indicators for various tissue regions is shown below, with voxels identified as background labeled with 0: {'prostate': 1, 'gluteus_maximus_left': 2, 'gluteus_maximus_right': 3, 'rectum': 4, 'urinary_bladder': 5, 'sacrum_and_coccyx': 6, 'hip_bone_left': 7, 'hip_bone_right': 8}

[0244] In one particular embodiment, the segmentation mask set identifies the prostate volume and the reference volume (eg, the left gluteal muscle) and labels all other voxels as background.

[0245] In certain embodiments, the segmentation output data also includes segmentation mask metadata, which may be used to store information about the particular image that was processed, such as the number of voxels in each direction (e.g., number of rows, columns, and slices) and the spacing of the voxels.

[0246] In certain embodiments, the segmentation service 1104 also performs quantification using segmentation results, such as the identified prostate volume and reference volume, and the intensities of voxels in the 3D functional image (e.g., a SPECT image). This quantification is discussed in more detail herein, for example, in the subsection entitled "Uptake Metrics" below. For example, as described herein, the segmentation service 1104 may identify a maximum intensity voxel of the prostate along with a background value for use in calculating a target-to-background ratio (TBR) value for the subject. The maximum intensity voxel of the prostate is the voxel identified as having the maximum intensity within a set of voxels in the 3D functional image corresponding to the identified prostate volume. In certain embodiments, as described herein, the voxels of the 3D functional image corresponding to the identified prostate volume are corrected for crosstalk (also referred to as bleed) from the subject's bladder, and the maximum intensity voxel of the prostate is identified after correcting for bladder crosstalk. The background value used to determine the TBR value is an average value across the intensities of multiple voxels in the 3D functional image that correspond to a reference volume, such as a volume corresponding to the subject's left gluteus muscle within the 3D anatomical image. As described herein, generally, all voxels that correspond to an identified reference volume are used to calculate the background value.

[0247] Data corresponding to the quantification performed by the segmentation service 1104 may be included in the segmentation output data. For example, the results of the quantification may be included in a quantification results dictionary, as shown in the following example .json format: { “prostate_max”: 988.0, “prostate_max_pos”: { “z”: 92, “y”: 74, "x": 67 }, “background”: 9.056859205776174, }

[0248] The keys of the dictionary of quantification results in the above example are: "prostate_max" stores the intensity of the most intense voxel of the prostate. "prostate_max_pos" stores the location of the most intense voxel of the prostate in the SPECT image. "background" stores the value of the background. An example segmentation service architecture including the composition and interaction of CNN modules

[0249]

[0013] Figure 12 shows an example architecture 1200 of modules for performing CNN-based image segmentation as described herein. The example architecture 1200 includes a first machine learning module (referred to as "Localization Machine" in Figure 12) 1204 for identifying an initial VOI, and a second machine learning module (referred to as "SingleSegMachine" (base) in Figure 12, which is short for "base single segmentation machine") 1208 for identifying prostate volumes, other tissue volumes, and reference volumes as described herein. The inputs and outputs of the first and second machine learning modules are also shown, along with several additional modules, including an auxiliary machine learning module (referred to as SingleSegMachine(aux) in Figure 12, which is short for "auxiliary single segmentation machine") 1212 and a module 1214 for combining the output of the second machine learning module (base single segmentation machine) with the output of any auxiliary machine learning module (auxiliary single segmentation machine) to perform post-processing. The output produced by the modular configuration shown in FIG. 12 includes a set of segmentation masks 1216 that identify various tissue volumes in a 3D anatomical image (eg, a CT image), and segmentation metadata 1218.

[0250] As described herein, the first machine learning module 1204 receives the CT image 1202 as input and generates a bounding box that identifies an initial VOI corresponding to the subject's pelvic region. The second machine learning module 1208 receives the initial VOI along with the CT image 1202 as input. The second machine learning module 1208 may add a crop margin to the initial VOI (e.g., add a margin around the initial bounding box to expand the bounding box) and provide crop end points to the auxiliary machine learning module. As described herein, the second machine learning module 1208 identifies a prostate volume within the initial VOI. The second machine learning module 1208 may also identify other tissue volumes corresponding to other tissue regions within the subject, such as the left gluteal muscle, bladder, and left and right hip bones. The tissue volume corresponding to the subject's left gluteal muscle may be used as a reference volume for calculating a background value, for example, when determining a TBR value for the subject, as described herein. The tissue volumes corresponding to the subject's bladder may be used to correct intensity values ​​of voxels in the SPECT image for bladder crosstalk, for example, as described in "Correcting for Bladder Intensity Bleed-Through" in Section B herein. The identified tissue volumes corresponding to the subject's left and right hip bones may be used for post-processing.

[0251] In certain embodiments, as described herein, performance may be improved by using, in addition to the second machine learning module 1208, one or more auxiliary machine learning modules 1212 (e.g., as shown at 1208 and 1212 in FIG. 12 ) that perform image segmentation in a manner similar to the second machine learning module to identify the prostate volume and other tissue volumes. These auxiliary tissue volumes identified by the auxiliary machine learning modules may be merged by module 1214 with the underlying tissue volumes (prostate volume and any other tissue volumes) identified using the second machine learning module 1208.

[0252] In certain embodiments, the first and second machine learning modules and any auxiliary machine learning modules are implemented as trained CNN modules. Each trained CNN module may be represented (e.g., by computer code) as a directory comprising an associated trained neural network with a model structure and weights. A neural network library such as Keras may be used to represent the trained CNN modules in this manner. The dictionary representing a particular CNN module may include metadata describing any preprocessing performed on images before the images are provided as input to the particular CNN module. Metadata describing how the neural network model was constructed and trained may also be included.

[0253] Thus, a particular CNN module may perform steps such as (1) loading and preprocessing an image given an image filename and image metadata, (2) providing the preprocessed image (e.g., as included in a directory representing the particular CNN module) to an associated trained neural network to obtain raw prediction outputs (e.g., a map including, for each of one or more voxels of a CT image, the likelihood (e.g., probability) that the voxel belongs to a particular category as determined by the trained neural network), and (3) post-processing the raw prediction outputs. As described herein, the first machine learning module 1204 may post-process the raw prediction outputs generated by its associated trained neural network to determine crop end points of a bounding box corresponding to an initial VOI. The second machine learning module 1208 may post-process the raw prediction output generated by its associated trained neural network to determine a segmentation mask set that labels the voxels of the CT image with a value (e.g., a numerical value) that indicates the particular tissue volume to which the voxel corresponds or a value that identifies the voxel as background, as determined by the second machine learning module.

[0254] Figure 13 shows an example model structure for an implementation of the CNN module of the first machine learning module (the localization machine of Figure 12) 1204. Figure 14 shows an example model structure for an implementation of the CNN module of the second machine learning module (the single segmentation machine of Figure 12) 1208. Example First Machine Learning Module (Localization Machine) Implementation

[0255] As described herein, the initial VOI identified by the first machine learning module is a "bounding box" that identifies the region of the CT image corresponding to the pelvic region of the subject, including, for example, the pelvic bones of the subject along with tissue regions such as the prostate, bladder, rectum, left and right gluteal muscles, etc. The second machine learning module, as well as any auxiliary machine learning modules, are given the bounding box identification (e.g., crop endpoints) to limit their processing to a small, specific target region of the CT image, as opposed to having to operate on and process the entire CT image. This improves the performance and efficiency of these machine learning modules that perform computationally intensive fine segmentation by reducing the amount of data they must process.

[0256] In an example implementation of the first machine learning module according to Method 3 as described herein, the localization machine 1204 of FIG. 12 is referred to as a localization CNN, which extracts a bounding box (initial VOI identification) from a coarse segmentation of the CT image performed by its associated trained neural network (CNN).

[0257] In certain embodiments, the localized CNN preprocesses the CT image to prepare it for segmentation. A configuration file may be used to define the preprocessing steps and parameters. Preprocessing may include cropping the CT image to remove one or more regions corresponding to surrounding air, normalizing the intensity of voxels in the CT image, and resizing the CT image (e.g., to generate a resized CT image that conforms to a fixed input size expected by the localized CNN). The intensity normalization preprocessing step adjusts the intensity of voxels in the CT image to generate a specific mean and standard deviation of intensity across all voxels in the CT image. For example, the voxel intensities of the CT image may be normalized by subtracting a first fixed value and then dividing by a second fixed value to generate a CT image in which the mean intensity across all voxels is a specific mean value (e.g., 0) and the standard deviation of intensity across all voxels is a specific standard deviation value (e.g., 1). The resizing step may be accomplished by sampling the CT image.

[0258] The localization CNN receives a preprocessed CT image as input and passes it through a series of layers, as shown in FIG. 13. The output of the localization CNN is a coarse segmentation, e.g., represented by a first segmentation mask set, which classifies voxels in the preprocessed CT image as belonging to one of four categories: three categories representing specific tissue regions, such as (i) the sacrum and coccyx, (ii) the left hipbone, and (iii) the right hipbone; and a fourth category, such as (iv) background (e.g., everything else). This coarse segmentation is derived from the raw prediction map generated by the localization CNN. The raw prediction map has the same set and shape as the voxels in the preprocessed CT image received as input, but is received in four channels. Each channel corresponds to a specific classification category, such as (i) the sacrum and coccyx, (ii) the left hipbone, (iii) the right hipbone, and (iv) background. Each channel represents a probability map for the corresponding classification category. That is, each particular channel corresponding to a particular classification category contains a likelihood value for each voxel in the preprocessed CT image that represents the probability that the voxel belongs to that particular category (e.g., represents a physical volume within the tissue region that the category represents, or represents background). Thus, for each voxel, the sum over the likelihood values ​​for that voxel in each channel is 1.

[0259] To determine a first segmentation mask set that classifies each voxel as belonging to a particular category, each voxel in the preprocessed CT image is assigned to the category (e.g., tissue region or background) that has the greatest likelihood value (e.g., probability) for that voxel. The determined first segmentation mask set has the same shape as the preprocessed CT image, and each voxel labeled by this set has a value that identifies the category to which it is classified. For example, a particular voxel may be labeled with a number such as 1, 2, or 3, corresponding to categories (i), (ii), and (iii), as described above, or may be labeled with a value of 0 if classified as a background voxel.

[0260] A bounding box identifying the initial VOI may be determined as the smallest box (e.g., rectangular volume) that contains all voxels labeled as belonging to categories (i)-(iii) from the first segmentation mask set. Coordinates identifying the bounding box (e.g., coordinates of opposite corners of the rectangular volume corresponding to the bounding box) are determined and output as the crop end points. In one particular embodiment, the preprocessed CT image input to the localization CNN is a resized version of the original CT image, and the coordinates identifying the bounding box are transformed into the coordinate system of the original CT image and output as the crop end points. Example Second Machine Learning Module (Segmentation Machine) Implementation

[0261] In the example architecture of FIG. 12, a second machine learning module, also referred to as a base single segmentation machine 1208 (SingleSegMachine (BASE) in FIG. 12), performs high-resolution segmentation of the CT image to identify the prostate volume corresponding to the target prostate gland, along with other tissue volumes such as the left and right gluteal muscles, rectum, bladder, sacrum and coccyx, and left and right hip bones that correspond to specific tissue regions.

[0262] In certain embodiments, the CT image is preprocessed by a second machine learning module in preparation for segmentation. As with the localization machine 1204, a configuration file may be used to define preprocessing steps and parameters. The preprocessing steps may include normalizing the intensity of the CT image, similar to the techniques described above for the localization machine 1204. The CT image may be cropped using crop endpoints that identify an initial VOI (bounding box) and output by the localization machine 1204 to generate a reduced, fixed-size preprocessed CT image input that serves as input to a trained neural network (trained CNN) associated with and implemented in a single segmentation machine 1208. Preprocessing may also include resizing.

[0263] Similar to the localization CNN of the localization machine, the single segmentation machine implements a trained CNN called a segmentation CNN, which receives a preprocessed CT image as input. The segmentation CNN passes the preprocessed CT image through a series of layers to output a second raw prediction map. Similar to the raw prediction map generated by the localization CNN, the second raw prediction map includes multiple channels, each corresponding to a separate, specific category into which each voxel of the input CT image is classified. Each channel represents a probability map for the corresponding classification category and includes, for each voxel of the preprocessed CT image, a likelihood value corresponding to the probability that the voxel belongs to that category (as determined by the segmentation CNN).

[0264] As previously described, the second raw prediction map output by the segmentation CNN includes a channel corresponding to the target prostate gland. The second raw prediction map may include other channels corresponding to various other tissue regions, such as the left gluteal muscle, right gluteal muscle, rectum, bladder, sacrum and coccyx, left hipbone, and right hipbone. The second raw prediction map may also include a background channel.

[0265] Certain variations of high-resolution segmented CNNs provide predictions from multiple levels of the network, resulting in multiple probability maps per category. These multiple probability maps are referred to as auxiliary predictions. The CNN model structure shown in FIG. 13 corresponds to a model structure that does not generate auxiliary predictions, while the CNN model structure shown in FIG. 14 is for a model that generates auxiliary predictions. In certain embodiments, the number of filters in the top layer (e.g., 20 filters in FIG. 13 and 28 filters in FIG. 14) can vary, but the number of filters in subsequent lower layers doubles with each subsequent layer. In certain embodiments, the segmented CNN generates a single probability map and does not generate auxiliary predictions. In certain embodiments, the segmented CNN generates auxiliary predictions.

[0266] In one particular embodiment, when the segmentation CNN generates the auxiliary prediction maps, the likelihood values ​​for each category are included in a second, raw prediction map, and the respective auxiliary probability maps are averaged together so that a single likelihood value for each category is determined for a particular voxel.

[0267] In certain embodiments, each voxel of the preprocessed CT image is assigned the highest probability value to determine a second fine segmentation mask set that classifies each voxel as belonging to a particular category. In certain embodiments, the number of landmarks in the fine segmentation mask set is reduced so that only certain tissue volumes, such as the prostate volume and a reference volume (e.g., the left gluteal muscle volume), are included in the fine segmentation mask set. Auxiliary Single Segmenting Machine

[0268] In certain embodiments, one or more auxiliary machine learning modules are used to generate auxiliary fine segmentation mask sets similar to the fine segmentation mask set generated by the second machine learning module, where these auxiliary fine segmentation mask sets identify the same set of tissue volumes as the fine segmentation mask set generated by the second machine learning module. In this manner, the second machine learning module generates the base fine segmentation mask set, and the one or more auxiliary machine learning modules each generate an auxiliary fine segmentation mask set, thereby resulting in parallel sets of classifications for voxels of the CT image.

[0269] In a particular embodiment, for each category representing a particular tissue region, a corresponding base fine segmentation mask of the base fine segmentation mask set (e.g., identifying a volume in the CT image determined by a base single segmentation machine as corresponding to the particular tissue region) or a corresponding base fine segmentation mask determined using the base fine segmentation mask set is merged with one or more corresponding auxiliary fine segmentation masks (e.g., each identifying a volume in the CT image determined by a supplementary single segmentation machine as corresponding to the same particular tissue region) to generate a merged fine segmentation mask. For example, for a particular category, certain voxels in one or more auxiliary fine segmentation masks may be identified (e.g., labeled) as belonging to a particular category but not to the set of voxels in the base fine segmentation mask identified as belonging to this particular category. These voxels may be added (e.g., by being labeled as such) to the set of voxels identified as belonging to the particular category in the base fine segmentation mask to generate the final merged fine segmentation mask. 12, the base single segmentation machine 1208 generates a base fine segmentation mask, and one or more auxiliary single segmentation machines 1212 each generate an auxiliary segmentation mask. The base fine segmentation mask generated by the base single segmentation machine 1208 is merged by module 1214 with one or more auxiliary fine segmentation masks generated by the auxiliary single segmentation machine 1212 to generate a final fine segmentation mask set 1216 that includes a final fine segmentation mask for each category.

[0270] In certain embodiments, one or more of the final fine segmentation masks are filtered to preserve only the largest connected portions. In certain embodiments, when filtering is performed to preserve only the largest connected components of the prostate segmentation mask (e.g., the final fine segmentation mask that identifies the volume of the CT image corresponding to the prostate of interest), a subset of the connected components of the prostate segmentation mask is taken into consideration. This subset includes only those components that (i) have a center of mass between the center of mass of the left hip bone and the center of mass of the right hip bone in the left-right direction (x-direction) of the axial plane, and (ii) are inside a bounding box defined to exactly contain the left and right hip bones. iv. Intake metrics

[0271] Referring to step 110 of process 100 in FIG. 1 , one or more uptake metrics are determined, from which a diagnosis of a condition (e.g., prostate cancer) or disease classification may be communicated or automatically rendered. Using the 3D functional image and a prostate volume identified within a VOI in the 3D anatomical image, one or more uptake metrics are determined. For example, the amount of radiopharmaceutical in the subject's prostate may be calculated based on intensity values ​​of voxels in the 3D functional image corresponding to the prostate volume identified within the VOI in the 3D anatomical image. This may involve calculating the sum (e.g., weighted sum), average, and / or maximum of the intensities of voxels in the 3D functional image that represent the physical volume occupied by the subject's prostate. In certain embodiments, this includes calculating a normalization value based on intensity values ​​of voxels in the 3D functional image that correspond to a reference volume identified within the 3D anatomical image. For example, normalized values ​​for the intensities identified in the prostate may be normalized using intensity values ​​of one or more voxels corresponding to the gluteus muscles, or another reference volume within the VOI (or, in certain embodiments, elsewhere within the 3D anatomical image). The uptake metric may then be converted to identify the presence or absence of prostate cancer in the subject, and / or to quantify the risk that the subject has prostate cancer, and / or to disease classification (e.g., as part of tracking the disease over time), and may be used by a physician, for example, to advise treatment options and / or to monitor the efficacy of administered therapy. Target to background ratio

[0272] In certain embodiments, the determined one or more uptake metrics include a target to background ratio (TBR) value for the subject. Determining the TBR value includes (i) determining a target intensity value using intensity values ​​of one or more voxels of a 3D functional image (e.g., a PET image or a SPECT image) corresponding to an identified prostate volume within an initial VOI of the 3D anatomical image (e.g., a CT image), and (ii) determining a background intensity value using intensity values ​​of one or more voxels of the 3D functional image corresponding to the identified reference volume. The TBR value is calculated as a ratio of the target intensity value to the background intensity value.

[0273] In particular, in certain embodiments, the target intensity value is the maximum value of the intensities of voxels in a 3D functional image (e.g., a PET image or a SPECT image) corresponding to the identified prostate volume. As described above, the maximum intensity voxel of the prostate corresponds to the voxel identified as having the maximum intensity within the set of voxels in the 3D functional image corresponding to the identified prostate volume. Thus, the target value may be obtained as the intensity of the maximum intensity voxel of the prostate. The background intensity value may be calculated as the average value over the intensities of multiple voxels in the 3D functional image corresponding to the identified reference volume. As described herein, a gluteus muscle, such as the left gluteus muscle, or a portion thereof may be used as the identified reference volume.

[0274] For example, the TBR value can be calculated from a SPECT / CT image of a subject recorded after administering a radiopharmaceutical such as 1404 to the subject. The SPECT image corresponds to a 3D functional image, and the CT image corresponds to an anatomical image. The image segmentation techniques described herein can be used to identify a prostate volume within a CT image, along with a reference volume corresponding to the subject's left gluteal muscle. The prostate segmentation mask and the left gluteal muscle segmentation mask can be used to identify the prostate volume and the left gluteal muscle reference volume, respectively.

[0275] The prostate volume segmentation mask identifies voxels of the CT image that are classified as belonging to the target prostate by the machine learning segmentation techniques described herein. In a specific embodiment, voxels of the prostate segmentation mask are mapped to corresponding voxels of the SPECT image to identify voxels of the SPECT image that correspond to the identified prostate volume within the CT image. Because the prostate segmentation mask identifies voxels of the CT image, it may have a different resolution from that of the SPECT image (e.g., because the SPECT image and the CT image may have different resolutions). Interpolation (e.g., bilinear interpolation) and / or sampling may be used to match the resolution of the prostate segmentation mask to the resolution of the SPECT image so that each voxel of the map of the prostate segmentation mask maps to a specific corresponding voxel of the SPECT image. In this way, a SPECT prostate mask is obtained that identifies voxels of the SPECT image that correspond to the identified prostate volume within the CT image. A SPECT left gluteal muscle mask that identifies voxels of the SPECT image that correspond to the identified left gluteal muscle reference volume in the CT image may be obtained in a similar manner.

[0276] The SPECT left gluteal muscle mask may be used to determine a background intensity value. Specifically, the intensities of voxels in the SPECT image identified by the SPECT left gluteal muscle mask are extracted and divided into quartiles. The background value is calculated as the average value across the intensities of the extracted left gluteal muscle voxels that fall within the first and third quartiles. Other techniques for calculating the background value may also be used, such as calculating the overall average or median across all extracted left gluteal muscle voxel intensities. The aforementioned technique of dividing the extracted left gluteal muscle voxel intensities into quartiles has been found to be more robust to outliers than calculating the overall average (e.g., because the intensities are discrete and many have the same value) and more accurate than calculating the overall median. The determined background intensity value may be output, for example, under the key "background" and stored in a results dictionary.

[0277] The target intensity value may be calculated using a SPECT prostate mask. The maximum intensity voxel of the prostate may be identified as the voxel in the SPECT prostate mask and has the maximum SPECT image intensity. The target intensity value is determined as the intensity of the maximum intensity voxel of the prostate. Both the target intensity value and the location of the maximum intensity voxel of the prostate may be stored. For example, the resulting dictionary may store the target intensity value and the location of the maximum intensity voxel of the prostate under keys such as "prostate_max" and "prostate_max_position," respectively. In certain embodiments, the intensity of voxels of the SPECT image corresponding to the identified prostate volume may be corrected for bladder "crosstalk" or "bleed" before determining the target intensity value, as described in Section B below, e.g., so that the maximum intensity stored is the maximum corrected intensity, and so that the maximum intensity voxel of the prostate stored is the voxel in the SPECT prostate mask having the maximum corrected intensity.

[0278] In certain embodiments, the subject's prostate cancer status can be determined by comparing the TBR value determined for the subject with one or more thresholds. Specifically, the determined TBR value can be compared to a particular threshold (e.g., a cutoff threshold) to distinguish between patients with clinically significant prostate cancer (e.g., assigned a clinically significant status) and patients without clinically significant prostate cancer (e.g., assigned a clinically insignificant status). Example 4 below provides an exemplary approach for determining a TBR threshold based on TBR values ​​calculated from reference images to achieve a desired sensitivity and specificity. B. Bladder Strength Correction for Bleed-Through

[0279] FIG. 15 is a schematic diagram illustrating contrast agent intensity crosstalk from the bladder to the subject's prostate. In this diagram, reference 1502 is the bladder, 1506 is the crosstalk / bleed intensity from the bladder, and 1504 is the prostate. Certain contrast agents, including PSMA binding agents, have high uptake in the bladder, which can affect the identification of diseased tissue (e.g., prostate cancer). For example, uptake of a radionuclide-labeled PSMA binding agent by the bladder can result in scattering in the 3D functional image, reducing the accuracy of the contrast agent intensity measured in the prostate gland, which is located near the bladder. Training a second CNN for detailed segmentation of both the subject's prostate and bladder can automatically and accurately account for "bleed-through" or "crosstalk" effects and / or other effects resulting from contrast agent uptake by the bladder. Furthermore, by training a second CNN to identify reference regions within the 3D anatomical image, such as the gluteus muscle, it is possible to more accurately weight / normalize the contrast agent intensity measurements, improving the accuracy and diagnostic value of uptake measurements in the subject's prostate. i. Bladder distension

[0280] In one particular embodiment, bladder crosstalk correction involves a step in which the identified bladder volume is dilated (e.g., by morphological dilation) using two iterations. This dilation can be used to prevent high intensities too close to the segmented bladder used to determine the target intensity value when calculating the TBR value (e.g., selected as the maximum intensity), as well as to stabilize the bladder suppression technique described below. ii. Bladder inhibition calculation

[0281] In one specific embodiment, a bladder suppression method is used to remove intensity bleed from the bladder into other regions of the functional image. The amount of suppression, i.e., the intensity bleed to be removed from a particular voxel of the functional image, depends on the distance from that voxel to the core bladder region (corresponding to the region of the bladder with high intensity voxels).

[0282] In certain embodiments, bladder inhibition is performed if the maximum functional image intensity within a volume of the functional image identified as corresponding to the bladder (e.g., corresponding to a bladder volume identified within the 3D anatomical image, e.g., identified by a bladder mask) is greater than a determined background intensity value multiplied by a particular number. As described herein, the background intensity value may be determined based on the intensity of voxels of the 3D functional image corresponding to a reference volume identified within the 3D anatomical image, e.g., a gluteus muscle volume. For example, bladder inhibition may be performed if the maximum functional image intensity within a region corresponding to the identified bladder volume is 15 times the determined background value (e.g., a ratio of bladder to background of at least 15).

[0283] In certain embodiments, bladder inhibition is calculated from and applied to a portion of the 3D functional image that is within a bladder inhibition bounding box of a particular size around the identified bladder volume. For example, a bladder inhibition bounding box may be determined that contains the bladder with a margin of a predetermined size in a particular direction (e.g., 40 mm vertically) and an equal number of voxels in the other direction.

[0284] For example, after masking out a region of the 3D functional image corresponding to the prostate gland, a core bladder region may be determined as a region of the 3D functional image within a bladder inhibition bounding box that includes voxels with intensities within a certain percentage of the maximum intensity within the bounding box (e.g., not including voxels of the masked-out prostate region). For example, the core bladder region may be determined as a region that includes voxels with intensities greater than or equal to 50% of the maximum intensity within the bladder inhibition bounding box. The core bladder region may include high-intensity regions that were not included in the original bladder mask.

[0285] In a specific embodiment, one or more bladder intensity bleed functions are determined to correct the intensity of voxels in the 3D functional image for bladder crosstalk by performing bladder suppression. For example, the 3D functional image may be cropped using a bladder suppression bounding box, and the determined background intensity value may be subtracted from the intensity of voxels within the cropped image region. Sample intensities are then collected to determine how much the bladder intensity (e.g., intensity resulting from the uptake of a radiopharmaceutical inside the subject's bladder) decreases as one moves away from the bladder. Samples are collected starting at the top, front, right, and left extremities of the core bladder region, and then straight up, front, right, or left, one voxel at a time, respectively. When the edge of the cropped image region is encountered, the intensity extrapolated from, for example, two or more previous samples by linear extrapolation may be used as the sample.

[0286] The intensity samples yield four curves (e.g., sets of sampled intensity data points) of intensity decline from the bladder core, each of which may be fitted with a template function to establish four bladder intensity bleed functions that model bladder intensity change as a function of distance from the core bladder region. Prior to further analysis, such as fitting, outlier removal may be performed, for example, to remove the curves that are furthest from the others. A template function, such as an nth-order polynomial (e.g., a quintic polynomial), is fitted to the data points in the remaining three curves to yield a function that models bladder intensity bleed as a function of distance to the core bladder. The bladder intensity bleed functions describe how much to subtract from the original intensity, depending on the distance to the core bladder region, to obtain a corrected intensity.

[0287] In certain embodiments, to reduce the risk that the bladder intensity bleed function underestimates the bladder directional bleed (bleed can vary somewhat between different directions), the distance is multiplied by a dilation factor (e.g., 1.2, in the range of 1 to 2) before fitting to the sampled intensity data points, resulting in a stretched fitted bladder intensity bleed function. The risk of underestimating bladder intensity bleed can also be reduced by multiplying the sampled intensity data points by a conversion factor. The conversion factor may be a variable conversion factor whose value depends on the intensity ratio of the bladder to background. The conversion factor used may, for example, have a particular value when the bladder to background ratio is sufficiently large and increase (e.g., linearly) as the bladder to background ratio decreases. For example, when the bladder to background ratio is sufficiently large, the sample intensity may be multiplied by a factor of 1.2, which increases linearly from 0 for a bladder to background ratio of 15 for lower bladder to background ratios. This approach of multiplying the sampled intensity data points by a conversion factor also improves the robustness of the bladder inhibition approach.

[0288] In certain embodiments, the extent of bladder inhibition, i.e., the furthest distance from the bladder where bladder inhibition applies, is based on where the decay rate of the function is sufficiently small. If the decay rate is not sufficiently small, this distance is chosen to be the length of the intensity sample vector. In certain embodiments, bladder inhibition at all distances is guaranteed to be non-negative.

[0289] Once the bladder intensity bleed function is determined, it can be evaluated at various voxel locations of the 3D functional image, such as voxel locations corresponding to locations within the subject's prostate gland, to determine a bladder intensity bleed value for each particular voxel location. Accordingly, the intensity of a particular voxel can be corrected for bladder crosstalk by subtracting the bladder intensity bleed value determined at the particular voxel location from the intensity of the particular voxel. Intensities of voxels at various locations within the 3D functional image, such as locations corresponding to an identified prostate volume (e.g., identified within the 3D anatomical image), can be corrected for bladder crosstalk in this manner. iii. Corrected prostate maximum intensity and location

[0290] In certain embodiments, the bladder suppression techniques described herein are used to correct intensity values ​​of voxels of a 3D functional image corresponding to a prostate volume identified within a 3D anatomical image, and an uptake metric, such as a TBR value, is determined using the corrected intensity values. C. Visualization of image data and calculated uptake metrics

[0291] Figure 16A shows a GUI interface window listing two subjects (patients), allowing the user to select the desired subject and process or view the subject's image data.

[0292] 16B shows an exemplary GUI with graphical control elements for processing and navigating object data, which appears after a user selects the object "John Doe." The user clicks selectable button elements to complete processing of image data for the selected object.

[0293] In FIG. 16C, after processing of the image data of the selected subject was completed, the user clicked another selectable button to view the processed image data and calculate uptake metrics.

[0294] 17A-17E show a 2D viewer, which shows a set of 2D cross-sectional views of CT image data overlaid with SPECT images and graphics representing identified tissue volumes (specifically, the prostate and pelvic bones). The user can scan through each slice of the cross-section as shown.

[0295] The CT and SPECT images are rendered as selectable layers that can be toggled on and off. Graphics representing identified tissue volumes are also rendered as selectable segmentation layers. In Figure 18A, the user toggles off the SPECT image layer so that only the CT and segmentation layers are displayed. In Figures 18B-18F, the user scans through each cross section.

[0296] In Figure 19A, the user toggles the SPECT image layer on and hides the segmentation layer. In Figures 19B-19D, the user again scans through each cross-section, this time with only the CT and SPECT image layers visible.

[0297] 20A-20C show a rotatable and sliceable 3D viewer for viewing CT images overlaid with SPECT image data, the CT images including a graphical representation of soft tissue.

[0298] 21A-21B also show a rotatable and sliceable 3D viewer, but only displays a graphical representation of the bones in the CT image.

[0299] Figures 22A-22F show how a user can use this 3D viewer to inspect the image data and the automatic identification of the prostate and other tissue regions. The user flips through the slices to view the pelvic region in Figure 22B and switches the segmentation layer to view graphics representing the prostate volume and pelvic bone volumes as colored regions identified and superimposed on the CT image. The user can slice and rotate the image to observe the SPECT image intensity bright spot in the prostate region (purple volume).

[0300] The viewer shown in Figures 23A and 23B is with a black background to improve contrast and mimic a radiologist scan.

[0301] FIG. 24 shows a report (eg, an automatically generated report) using various intake metrics. D. User Interface, Quality Control, and Reporting

[0302] In certain embodiments, the systems and methods described herein are implemented as part of a GUI-based workflow that allows a user, such as a physician (e.g., a radiologist, technician), to upload patient (subject) images and initiate an automated analysis using the techniques described herein, in which a prostate volume is identified in a 3D anatomical image and used to determine an uptake metric using corresponding voxels in a 3D functional image. The user may then view the results of the automated analysis, including the determined uptake metric and any prognostic value determined therefrom. The user may be guided through a quality control workflow to approve or disapprove the results of the automated analysis and, if the results are approved, generate a report for the patient. This quality control workflow allows the user to manually adjust or update the results of the automated analysis, for example, by interacting with the GUI, and generate a report based on the manually updated results.

[0303] Figure 25 illustrates an example workflow 2500 used to analyze SPECT / CT images in certain embodiments. As shown in Figure 25, a user may upload 2504 SPECT / CT images that conform to a particular accepted standard format 2502 (specifically, the DICOM standard in the example of Figure 25). Figure 26A illustrates an example GUI window 2600a of a web-based portal that allows a user to upload images in 2504. In an updated view 2600b of the GUI window of Figure 26A, shown in Figure 26B, several images have been selected for upload and verified to conform to the DICOM standard. One image was identified as not conforming to the DICOM standard and indicated in 2602b that it could not be uploaded.

[0304] Returning to Figure 25, at another step 2506, the user may view a list of patients for whom images have been uploaded. Figure 27A shows a view 2700a of a GUI window listing patients by anonymized numeric identifier. Figure 27B shows another view 2700b of the GUI window shown in Figure 27A, with a row in the patient list corresponding to a particular patient highlighted for selection. Figure 27C shows another view 2700c of this GUI, where, when the row corresponding to this particular patient is selected, a menu listing the tests performed on this particular patient is displayed, including selectable buttons that allow the user to review image data related to the tests or generate reports.

[0305] In certain embodiments, user review of the image data, along with the results of any automated analysis performed using the image data, is a prerequisite for generating a report. User review of the image data and automated analysis may be required to verify the accuracy of the image segmentation. For example, as shown in FIG. 25, after the user views the list of patients and selects a patient in 2506, the user then reviews the patient's image data and the results of the automated processing as described herein in the next step 2508.

[0306] In particular, in review step 2508 of workflow 2500 shown in Figure 25, a user uses a GUI-based viewer to examine SPECT / CT image data for a selected patient. The user may select the SPECT / CT image data as a set of 2D slices or a 3D rendering for viewing or examination. In the view of GUI viewer 2800 shown in Figure 28A, the SPECT / CT image data is displayed as a set of 2D slices 2802a. In the view of GUI-based viewer 2800 shown in Figure 28B, the SPECT / CT image data is displayed as a 3D rendering 2802b.

[0307] As shown in FIGS. 28A and 28B , the GUI viewer 2800 displays the SPECT image and the CT image as selectable layers superimposed on one another. A user may select one layer at a time to view the SPECT image and / or the CT image alone, or may select both the CT and SPECT layers to view the SPECT image superimposed on the CT image. The user may, for example, adjust the opacity of the SPECT image to emphasize / de-emphasize features of the SPECT image superimposed on the CT image. The user may also view a segmentation layer showing the locations of various tissue regions identified within the CT image by the second machine learning module as described herein. In this manner, a user may confirm the image segmentation performed by the second machine learning module, for example, by visual inspection of the CT image layer and the segmentation layer.

[0308] A user may observe or confirm one or more uptake metrics determined by the automated image analysis techniques described herein. For example, in the GUI view 2900 shown in FIG. 29A, a panel 2902 of the GUI displays an automatically determined TBR value 2904 for a patient. The panel also displays a determined TBR-based classification 2906 indicating that the TBR value is associated with clinically significant prostate cancer.

[0309] In certain embodiments, to assist the user in verifying the determined uptake metric, a graphical element indicating the location of the voxel of the identified prostate volume is displayed within the GUI. For example, as described herein, when a TBR value is calculated as the ratio of a target intensity value to a background intensity value, the maximum intensity of a voxel in the SPECT image corresponding to the identified prostate volume is identified. Thus, the GUI may display a graphical element indicating the location of the voxel with the maximum intensity corresponding to the identified prostate volume in the SPECT image compared to other voxels corresponding to the identified prostate volume in the SPECT image. In this manner, the position of the maximum SPECT intensity voxel corresponding to a location within the prostate volume is displayed to the user. The user may then visually verify that this maximum SPECT intensity voxel is actually within the target prostate by, for example, examining the relationship of the graphical element compared to the CT image. For example, as shown in FIG. 29A, a set of crosshair graphical elements 2950, ​​2952, and 2954 identifying the location of the maximum SPECT intensity voxel are superimposed on a 2D slice shown in the image viewer.

[0310] Returning to FIG. 25 , in another step 2510, a user may select to generate a report summarizing the analysis performed on the patient using the uploaded SPECT / CT images and may be guided through a quality control workflow. For example, as shown in FIG. 29A , a user may select (e.g., click) a generate report button 2908 on the GUI. Once the user selects the generate report button 2908, a quality control graphical widget 2910 is displayed, as shown in FIG. 29B . The quality control graphical widget 2910 may guide the user to verify whether various acceptance criteria are met. For example, the quality control graphical widget 2910 guides the user to verify that image requirements are met and that the target and background values ​​used to determine the TBR value are accurate. The user may approve the results of the automated analysis by selecting button 2912 or disapprove by selecting button 2914.

[0311] As shown in Figure 25, after user approval 2512 of the quality control confirmation, the results 2514 of the automated assessment are used to generate a report 2532 about the patient. As shown in Figure 29C, the user may be presented with a widget 2916 requesting confirmation of quality control approval before generating the report. Figure 29D shows an example report 2900d. The report may include an identification of the quality control criteria 2918, along with an identification of the user who approved the quality control and signed the report 2920.

[0312] As shown in FIG. 25 , if a user disapproves of the automatic determination of an intake metric (e.g., a TBR) at 2516, the quality control widget may initiate a guided assessment 2518 workflow. In the guided assessment workflow, the user may manually update the values ​​used to determine one or more intake metrics through manual interaction with the GUI. The user may also provide input, which may be received by the quality control widget, indicating that images related to the patient cannot be used to accurately determine the intake metric. For example, following user input corresponding to a quality control disapproval, the user may be presented with a GUI element such as that shown in FIG. 29E. The GUI element 2900 e shown in FIG. 29E allows the user to select that a TBR value cannot be determined for the patient by selecting button 2922, or to manually update the target intensity value and background intensity value used to calculate the TBR value by selecting button 2924.

[0313] For example, if a user determines (e.g., by visual inspection of the image) that the image is of too low quality to be used to accurately determine a TBR value, the user may select button 2922 in GUI element 2900e to identify the case as unscored. As shown in Figure 25, following receiving the user's identification that the case is unscored at 2520, a report 2532 is generated that marks the case as unscored and identifies the case as having failed quality control at 2530. Figure 29F shows example graphics and text 2900f that may be included in a report identifying the case as having failed quality control.

[0314] In certain embodiments, a user may decide to update the target intensity value and / or background intensity value used to determine the TBR value by manual interaction with the GUI. When user input is received at 2522 indicating that the user wishes to update the target intensity value and / or background intensity value, the user is guided to manually set the target value at 2524 and / or manually set the background value at 2526. FIG. 29G shows a view 2900g of a manual input graphical widget displayed to the user, allowing the user to set the target value and / or background value. The user may click button 2926 to manually set the target intensity value by interaction with the GUI. When the user selects button 2926, a voxel selection graphical element is presented, allowing the user to select a voxel in the SPECT image to be used as the maximum intensity voxel in calculating the TBR value. For example, the user may be provided with a movable crosshair, such as the crosshair shown in FIG. 29A, that allows positioning in the SPECT and CT images. The user may move the crosshair and / or click a location within the viewer to select a particular voxel to use as the maximum intensity voxel. As shown in Figure 29H, the updated manual input graphical widget view 2900h shows the intensity value 2930 of the selected maximum intensity voxel to be used as the updated target intensity value for calculating the updated TBR value.

[0315] The user may select button 2928 to set a background value to use in calculating the TBR value. When the user selects button 2928, a voxel selection element is presented, and the user may be guided to select multiple voxels that they identify as belonging to the left gluteal muscle (e.g., based on visual inspection of the CT image layer and / or segmentation layer displayed in the GUI viewer). Because the background intensity value for calculating the TBR is an average across the intensities of multiple voxels in the SPECT image corresponding to physical locations within the left gluteal muscle, the user may be guided to select a sufficient number of voxels to ensure that an accurate background intensity value is determined. For example, as shown in manual input graphical widget view 2900h in FIG. 29H, multiple samples may be displayed along with updated values ​​of background intensity 2932 as the user is selecting voxels to use in determining an updated background intensity value. In certain embodiments, the user is required to select at least a predetermined number (e.g., 100) of samples, and the displayed background intensity value and number of samples 2932 include a visual indication (e.g., a color change) to inform the user that a sufficient number of samples have been selected.

[0316] 25, once the user has completed manual entry of the updated target value and / or updated background intensity value, the updated values ​​are used to calculate an updated TBR value and stored as semi-automated assessment 2528. A report 2532 may then be generated using the results of the semi-automated assessment. E. Example Cloud-Based Architecture and Service Configurations

[0317] The systems and methods described herein are implemented as a cloud-based application in certain embodiments. The cloud-based application may use multiple modules to perform various functionality, such as a client-facing module that receives input from a user and provides an interface for presenting image data and results. The client-facing module may then interface with other modules, such as a segmentation services module to perform automated image segmentation to identify prostate volume and calculate uptake metrics as described herein.

[0318] FIG. 30 illustrates an example microservice network architecture 3000 for implementing the systems and methods described herein as a cloud-based application. This particular example microservice network is used for an example cloud-based application called Pioneer, as described herein. In the example microservice network architecture 3000 of FIG. 30, microservice Pioneer Web 3016 is a client-facing module that provides an interface for clients 3026 to interact with stored images and analysis results (e.g., by serving code instructions such as JavaScript and HTML). As shown in FIG. 30, Pioneer Web 3016 also communicates with other microservices. Audit service 3014 logs events and stores event logs in log file storage database 3012. Cognito 3020 remembers and authenticates users. Auth service 3022 tracks user populations and customer product registrations. Registration by Signature service 3024 is a front-end service that enables users to participate in, for example, the use of a cloud-based system. The slice box 3004 stores image files in the standardized DICOM format, including 3D anatomical images such as CT images and 3D functional images such as SPECT and PET images. The images are stored in the image storage database 3002. The image service 3006 reads images (e.g., DICOM files) and stores the image and image metadata in a specific (e.g., convenient, e.g., standardized) format. The segmentation service 3010 performs automated image segmentation and uptake metric determination as described herein. The segmentation service 3010 retrieves image data prepared by the image service 3006 from the data storage database 3008, performs image segmentation and uptake metric determination, and stores the results in the data storage database 3008.

[0319] The architecture shown in FIG. 30 can be used to implement the applications and platforms described herein in various data centers, including publicly available data centers. The data center provides infrastructure in the form of servers and networks, providing services such as networking, messaging, authentication, logging, and storage. The architecture 3000 for the application uses a set of limited-scope functional units called microservices. Each microservice handles an isolated set of tasks, such as image storage, risk index calculation, medical image type identification, and other tasks. Services (e.g., microservices) can communicate with each other using standard protocols such as Hypertext Transfer Protocol (HTTP). Organizing an application into a network of microservices, as shown in architecture 3000 of FIG. 30, allows each part of the platform to be scaled independently to meet high demands and ensure minimal downtime. In certain embodiments, such an architecture allows for the improvement or replacement of components without affecting other parts of the application or the platform that includes the application, among other things.

[0320] FIG. 31 is a block flow diagram illustrating an example data flow 3100 for performing automatic segmentation and analysis of CT / SPECT images using a microservices network such as that shown in FIG. 30. Images uploaded by a user are stored in a slice box 3110. A user may initiate processing of a study (e.g., patient images) to begin automated image segmentation and analysis, for example, by interacting with Pioneer Web 3104a via client 3102a. Study processing may also be initiated automatically (e.g., following image upload, e.g., at regular time intervals). As shown in FIG. 31, once study processing is initiated, Pioneer Web 3104a interacts with Image Service 3106 to begin preparation (e.g., preprocessing, e.g., formatting) of image data for automatic segmentation and analysis. Image Service 3106 stores the preprocessed images in database S3 3112. Pioneer Web 3104a initiates image segmentation and analysis by the segmentation service (e.g., by sending an SQS message). The segmentation service 3108 ingests image data, including 3D anatomical images and 3D functional images, such as CT / SPECT images, from S3 3112. The segmentation service 3108 performs image segmentation to identify tissue volumes, including the prostate volume, within the 3D anatomical image, as described herein, and calculates one or more uptake metrics using the identified tissue volumes and the 3D functional image. The results of the automated image segmentation and uptake metric determination are stored in S3 3112. The segmentation service posts a callback to Pioneer Web 3104a indicating whether the results were successful or an error / exception occurred. Upon receiving the callback, Pioneer Web 3104a posts a callback to the channel Pubsub 3118. A plurality of Pioneer web services 3104a and 3104b may receive status updates for the tests and notify clients 3102a and 3102b of the status updates, thereby interacting with clients 3102a and 3102b.

[0321] Figure 32 illustrates the flow of data 3200 between microservices, showing how a client can request the status of a test and be notified of test status updates in a manner similar to that described above with reference to Figure 31. As shown in Figure 32, a client 3202 can send a test ID to a Pioneer web service instance 3204, which receives status updates via a Pub / Sub channel 3212. Pioneer web can receive callbacks from any segmentation service 3208 regarding completed computations (e.g., completed image segmentation and intake metric analysis as described herein). The status regarding the completed computations for a test can be stored in a database 3210, and the updated status regarding the test is sent to the Pub / Sub channel 3212. All Pioneer web instances (e.g., 3204 and 3206) can subscribe to status updates and provide status updates (e.g., notifications of completed computations for a test) to interacting clients 3202.

[0322] In certain embodiments, such as those described above with respect to Figures 30 and 31, an image service module that may be included in an implementation of the systems and methods described herein performs pre-processing and formatting of image data to provide appropriately formatted image data to a segmentation service that performs, for example, automated image segmentation and intake metric determination.

[0323] The image services module may preprocess different images from various modalities, including 3D functional images and 3D anatomical images, to standardize and format the images. For example, preprocessing a 3D functional image (e.g., a nuclear medicine image such as a SPECT image) may include performing basic compliance checks and interpreting voxel intensity values ​​as specified in a standard format (e.g., a DICOM PS3 NM Image IOD). Frames of the 3D functional image may be arranged in a specific order corresponding to a direction along the subject, such as from head to foot. The position may be adjusted to represent the outer corner of the first voxel. Attributes from the 3D functional image that may be required for further processing may also be extracted.

[0324] Preprocessing of a 3D anatomical image, such as a CT image, may include performing basic compliance checks and interpreting voxel intensity values ​​as specified in a standard format (e.g., DICOM PS3 NM Image IOD). Slices of the 3D anatomical image may be arranged in a specific order corresponding to a direction along the subject, such as from head to foot. Cropping boundaries may also be determined to remove one or more regions of the 3D anatomical image corresponding to air around the imaged patient. The position may be adjusted to represent the outer corners of the first voxel. Attributes from the 3D anatomical image that may be required for further processing may also be extracted.

[0325] FIG. 33 shows architecture 3300, which illustrates how a cloud-based application implements the image segmentation and analysis techniques described herein and can be combined with other systems for performing other types of image analysis to provide a cloud-based platform that can be used by clients / users for multiple image analysis applications. Architecture 3300 shown in FIG. 33 includes a set of microservices 3320 shared between two or more applications. The left panel 3310 and the right panel 3330 show the microservices in the two applications. Microservices network 3330 implements a version of the Pioneer cloud-based application described herein to provide automated analysis of CT / SPECT images, calculation of uptake metrics such as TBR, and report generation. The web-based application implemented by microservices network 3310 shown in the left panel, called aBSI, analyzes whole-body scans obtained with a gamma camera to calculate an automated bone scan index (BSI). Further details regarding aBSI and automated BSI determination are provided in U.S. Patent Application No. 15 / 794,220, filed October 10, 2017, the contents of which are incorporated herein by reference in their entirety. F. Identification of Other Target Volumes of Interest and Calculation of Uptake Metrics

[0326] With respect to FIG. 34 , the techniques described herein may be used to identify other target volumes (e.g., lung, lymph node, bone, liver). Other image types may be used. Various radiopharmaceuticals may be used to generate 3D functional images. For example, in one particular embodiment, the contrast agent used for SPECT images is 1404, and in one particular embodiment, the contrast agent used for PET images is PyL. In one embodiment, imaging is performed over the subject's entire body to detect disease (e.g., tumors) in tissues such as lymph nodes. Another metric related to the total number and / or size of tumors and / or metastasis and / or amount of diseased tissue (e.g., cancer) is automatically determined using the methods described herein. For example, in process 3400, a 3D anatomical image is received in step 3402, and a corresponding 3D functional image is received in step 3404. At 3406, the first CNN may be used to efficiently identify one or more subregions of the initial 3D anatomical image for further segmentation by the second CNN at 3408, and at 3410, the corresponding 3D functional image is analyzed within the identified one or more subregions to quantify overall contrast agent uptake and / or provide calculated metrics representative of the level and / or extent of disease of interest. [Example]

[0327] Example 1 G. Example 1: Automated Detection and Quantification of Prostate PSMA Uptake from SPECT / CT Imaging In this example, 99mTc MIP-1404, a small molecule inhibitor of prostate-specific membrane antigen (PSMA), was used to detect clinically significant disease in prostate cancer. The goal of this example was to develop a deep learning model for automated detection and quantification of MIP-1404 uptake in the prostate in SPECT / CT images.

[0328] A deep learning algorithm based on convolutional neural networks was developed to automatically segment the prostate and pelvic bones from CT images. The algorithm was designed to process high- and low-dose CT images and whole- and partial-body views without manual interaction. Training materials consisted of 100 diagnostic CT images (all male) with manual full and separate segmentations of relevant anatomical regions. The algorithm was validated in a phase 2 trial of MIP-1404, including 102 high-risk prostate cancer patients, all of whom underwent PSMA imaging before radical prostatectomy. All validation scans were quantified manually using the OsiriX medical image viewer (Pixmeo SARL) by measuring maximum uptake in a circular ROI placed on the slice and region with the highest visually determined uptake value within the prostate. The automated algorithm uses the volumetric segmentation of the automated algorithm to measure uptake in all voxels of the prostate, recording the maximum uptake. Pearson's correlation coefficient was used to assess agreement between manual and automated quantification of prostate uptake.

[0329] The algorithm based on this training material had 2.7 million parameters and was optimized using Adam (gradient descent mutation). In the test set, 1404 images from 34 patients (33%, 34 / 102) were excluded due to excessive CT artifacts, incomplete data, and / or data formatting issues. The computation time for evaluable patients (N = 68) was 13 seconds (per case) on commodity hardware. The automated maximum uptake values ​​correlated significantly with manually obtained values ​​in the prostate (correlation coefficient = 0.95, 95% confidence interval = [0.91, 0.97]; gradient = 0.89, 95% confidence interval = [0.80, 0.98]; p < 0.0001). The algorithm was deterministic, fully automated, and 100% reproducible.

[0330] This example demonstrates the feasibility of objective and automated measurement of MIP-1404 uptake in the prostate. Example 2 H. Example 2: Technical and Clinical Performance of Automated Segmentation and Uptake Metric Determination

[0331] Example 2 demonstrates the technical and clinical performance evaluation of one embodiment of the automated image segmentation and uptake metric determination technique described herein, implemented by an exemplary cloud-based application called Pioneer. To establish the hypothesis for generating the algorithm for Pioneer, SPECT and CT image data of MIP-1404 from two clinical studies involving healthy volunteers (Phase 1 study: MIP-1404-1301) and prostate cancer patients with available histopathological diagnosis data after radical prostatectomy (Phase 2 study: MIP-1404-201) were used. Data comparing the receiver operating characteristic plots (ROC), sensitivity, specificity, and positive and negative predictive values ​​generated by Pioneer with those obtained by traditional manual interpretation indicated that Pioneer may be an effective tool to assist radiologists in interpreting MIP-1404 images, thereby facilitating the evaluation of prostate cancer patients. i. Technical performance

[0332] Analytical validation and motivation of the algorithm's segmentation and quantification performance criteria were predefined as detailed in OTHA-2262 and OTHA-2263, respectively. A total of 61 1404 SPECT / CT images and medical standards read by nuclear medicine physicians were used as benchmarks to assess analytical performance. All manual readings were performed independently and blinded to the technical performance acceptance criteria.

[0333] Performance testing of prostate segmentation revealed a mean Dice of 0.77 and a standard deviation of 0.012. The lower limit of the one-sided 95% confidence interval was 0.75, which is greater than the default threshold (0.70) in OTHA-2262. Performance testing of background (left gluteus maximus) segmentation revealed a mean Dice of 0.94 and a standard deviation of 0.002. The lower limit of the one-sided 95% confidence interval was 0.94, which is greater than the default threshold (0.80) in OTHA-2262.

[0334] Of the total 61 target locations predicted by the automated Pioneer software, only 4 (6.6%) were classified as incomplete by human experts (nuclear medicine readers), a result superior to the OTHA-2263 rule (10% failures). ii.Clinical results

[0335] In a retrospective ad hoc analysis of clinical outcomes, 14 healthy volunteers (Phase 1 study MIP-1404-1301) and 105 subjects with prostate cancer (Phase 2 study MIP-1404-201) were combined into a single population for analysis using Pioneer. Subjects in MIP-1404-201 who had received prior therapy for prostate cancer were excluded.

[0336] Images with CT artifacts or images that could not be reconstructed for automated analysis were also excluded.A total of 75 images were evaluated: 61 subjects with prostate cancer and 14 healthy volunteers.

[0337] Pioneer's automated assessment was evaluated against the gold standard of histopathological diagnosis for patients diagnosed with prostate cancer who underwent radical prostatectomy after SPECT / CT imaging with MIP-1404. Healthy volunteers were assumed to be free of prostate cancer based on normal PSA and pelvic MRI at the time of testing. Spearman's rho was used to assess correlation with Gleason score. The area under the receiver operating characteristic (ROC) curve was used to determine the algorithm's performance in detecting prostate cancer in the prostate. Sensitivity and specificity were determined from the optimal threshold / cutoff value from the ROC curve.

[0338] Automated quantitative assessment of 1404 images correlated with Gleason score (rho: 0.54; p<0.0001). The ROC curve demonstrated an AUC of 0.80 (95% CI: 67-94). The optimal threshold for a binary TBR-based outcome in distinguishing clinically significant prostate cancer from clinically insignificant cancer or normal prostate was determined to be 25. Using this threshold, Pioneer distinguished clinically significant prostate cancer from clinically insignificant prostate cancer or normal prostate with 75% sensitivity and 80% specificity using 1404 SPECT / CT images from the MIP-1404-1301 and MIP-1404-201 trials.

[0339] Thus, Pioneer improves on manual reading by (i) providing more objective and more reproducible reading performance across all diagnostic endpoints, and (ii) point estimates for important diagnostic parameters, e.g., sensitivity and specificity, shown to be consistently >70% using histopathology as the gold standard. Example 3 I. Example 3: Exemplary Cloud-Based Software for Automated and Semi-Automated Analysis of 1404 CT / SPECT Images

[0340] Example 3 is an example of a cloud-based software platform called Pioneer that implements one embodiment of the automated image analysis methodology described herein. Pioneer is a cloud-based software platform implemented in accordance with regulatory and data security standards that allows users to upload 1404 SPECT / CT image data, view them using a 2D or 3D medical image viewer accessed through the user's regular internet browser, and review and export TBR values. The software also provides a quality control workflow that allows users to assess the quality of the analysis, with the option to reject and / or adjust the automated analysis.

[0341] Therefore, the software employs artificial intelligence algorithms to automatically identify and analyze regions of interest (ROIs) under user supervision. Pioneer extracts image data from the ROIs to provide an objective analysis (target-to-background ratio (TBR)) based on prostate and background uptake. Because the signal in the prostate is often obscured by signals from the bladder (MIP-1404 excreted in urine), the software also has the built-in ability to segment the bladder and suppress bladder-related signals, so that prostate signals can be measured more accurately.

[0342] Pioneer's nonclinical performance data includes a verification and validation (V&V) assessment, which includes the definition of test methods. Predefined acceptance criteria were designed to ensure performance at least equivalent to the state of the art (manual assessment). Validation included software unit testing, integration testing, and software system testing, with functional testing of all software requirements. A validation process was performed to ensure the system met user requirement specifications. V&V test results demonstrated that Pioneer met its intended use, user requirements, and software requirements. Example 4 J. Example 4: Selection of TBR Thresholds for Clinically Significant Findings

[0343] Example 4 is an example illustrating how TBR thresholds for classifying a patient's prostate cancer status into clinically significant and clinically insignificant categories can be determined.

[0344] To select an appropriate threshold, two datasets of SPECT / CT images were combined. The first dataset included images of healthy individuals from a phase 1 trial of the 1404 drug. This dataset initially contained 14 images. Segmentation of the prostate within the images was performed using the method described herein, and two images in which prostate segmentation clearly failed were excluded, leaving 12 images. The second dataset included images of individuals with prostate cancer from a phase 2 trial of the 1404 drug. The images were classified based on the Gleason classification of the histopathological diagnosis from the subject's radical prostatectomy. A Gleason score of 7 or greater was considered clinically significant, while a Gleason score of 6 or less was considered clinically insignificant. The dataset initially contained 65 images (63 clinically significant and 2 clinically insignificant); one image that did not cover the entire pelvic region and one image in which prostate segmentation clearly failed were excluded, leaving 63 images (61 clinically significant and 2 clinically insignificant).

[0345] In summary, 14 images without clinically significant pathology and 61 images with clinically significant pathology were used.

[0346] To calculate the TBR values ​​for the images, a software package (version 1.0.0rcIII of the ctseg package) implementing one embodiment of the automated image segmentation and uptake metric determination technique described herein was used.

[0347] Figure 35A shows a swarm plot of clinically insignificant images (≤6 on the x-axis) and clinically significant images (≥7 on the x-axis). A threshold of 25 gives a sensitivity of 0.77 (the lower limit of the Jefferies one-sided 95% confidence interval is 0.67) and a specificity of 0.71 (the lower limit of the Jefferies one-sided 95% confidence interval is 0.50) on the test data. Figure 35B shows the ROC curves based on varying the TBR threshold. The TBR threshold point of 25 is marked as 3502 (red circle marker).

[0348] As shown in Figures 35A and 35B, a threshold of 25 provided a value for distinguishing patients with clinically significant prostate cancer from those without. However, because there are few data points in the immediate vicinity of the threshold, the sensitivity and specificity estimates are not robust to small random variations in the data. To obtain more robust estimates, the R package "scdensity" was used to estimate a smooth unimodal density for the distribution of the logarithm of the TBR values ​​for clinically insignificant and significant images, respectively. From this, sensitivity and specificity estimates were calculated for different TBR thresholds; see Table 1. [Table 1]

[0349] Therefore, based on the analysis described in this example, a TBR threshold of 25 was selected based on the desired specificity and sensitivity. These preliminary studies based on Phase 1 and Phase 2 data demonstrated an improvement in the diagnostic accuracy of SPECT / CT for MIP-1404 using automated methods versus manual reading. Example 5 K. Example 5: Analysis of Images from a Phase 3 Study Using AI

[0350] This example demonstrates the use of one embodiment of the artificial intelligence (AI)-based image analysis system and method described herein in the context of a Phase 3, multi-center, multi-viewer, open-label study. In particular, this example is titled "A Phase 3 Study to Evaluate the Safety and Efficacy of 99m This paper describes the results obtained from the MIP-1404-3302 study (hereinafter referred to as "Study 3302"), entitled "TC-MIP-1404 SPECT / CT Imaging and PIONEER as a combination product to detect clinically significant prostate cancer in men with biopsy-proven low-grade prostate cancer (proSPECT-A1)." In Study 3302, physicians used a combination of PIONEER and TC-MIP-1404 SPECT / CT Imaging as a contrast agent (for SPECT imaging). 99m A cloud-based implementation of the image analysis technology described herein (referred to as "PIONEER") was used to help identify subjects with and without clinically significant prostate cancer based on the analysis and interpretation of SPECT / CT images obtained using TC-MIP-1404. As used herein, "1404 SPECT / CT images" and "1404 SPECT imaging / CT imaging" refer to, respectively, SPECT imaging and CT imaging. 99m A phrase referring to a SPECT / CT image obtained using TC-MIP-1404 or the imaging process for obtaining such an image.

[0351] Study 3302 is a Phase 3 Study to Evaluate the Safety and Efficacy Phase 3 study MIP-1404-33 titled "1404 SPECT / CT Imaging to Detect Clinically Significant Prostate Cancer in Men with Biopsy-Proven Low-Grade Prostate Cancer Candidates for Active Surveillance" The study is based on the 3301 trial. The 3301 trial also assesses the use of 1404 SPECT / CT imaging to distinguish subjects with and without clinically significant prostate cancer. However, in the 3301 trial, physicians will perform the diagnosis using traditional manual methods (described in more detail herein) without the assistance of PIONEER.

[0352] This example summarizes the results obtained by the two methods and compares the results of the AI-assisted diagnosis performed in the 3302 trial with the conventional manual diagnosis in the 3301 trial.

[0353] Briefly, Studies 3301 and 3302 utilized a common dataset of 1,404 SPECT / CT images obtained from subjects for whom histopathology results were also available. In both Studies 3301 and 3302, each subject's SPECT / CT images were analyzed to assign a presence / absence, or positive / negative, status of clinically significant prostate cancer. Results from the analysis of the 1,404 SPECT / CT images were compared with ground truth histopathology results to assess the sensitivity and specificity of the 1,404 SPECT / CT images to identify clinically significant prostate cancer.

[0354] In the 3301 study, 1404 SPECT / CT images were analyzed using a traditional manual approach, in which physicians used standard imaging review software to scroll through slices of two-dimensional CT and SPECT images to manually position markers, identify regions of interest, and calculate TBR values. In contrast, in the 3302 study, physicians performed an AI-assisted review using an embodiment of the system and method described herein. Specifically, the AI-assisted approach in the 3302 study used a workflow similar to that shown in FIG. 25 , in which physicians performed image analysis with the assistance of PIONEER software. For each image, the PIONEER software fully automatically performed 3D segmentation of the prostate, calculated an initial TBR value, and assigned subjects an early prostate cancer status based on comparison of the initial TBR value with a predetermined threshold. The results of this automated analysis were presented to the physician for review using a slightly modified version of the GUI implemented by the PIONEER software and described above with respect to FIGS. 26A-29H. The physician could use the GUI to accept or reject the results of the fully automated analysis, update the target and / or background values, and recalculate the TBR in a semi-automated manner.

[0355] As described in more detail below, the results of the AI-assisted method used in Study 3302 were found to be more accurate than the traditional manual reading method used in Study 3301. Moreover, by reducing or substantially eliminating reliance on subjective human judgment, the AI-assisted method in Study 3302 rapidly produced highly consistent and reproducible results, with physicians only approving the fully automated analysis in the vast majority of cases. i.3301 study, study population, imaging, and manual reading protocol.

[0356] Study 3301 was a multicenter, multiviewer, open-label study. Study 3301 began in December 2015. Study 3301 completed enrollment and review in December 2017. An analysis of the results is included herein.

[0357] The objective of Study 3301 is to evaluate 1404 CT / SPECT scans for the detection of clinically significant prostate cancer in men who have undergone a diagnostic transrectal ultrasound (TRUS)-guided biopsy, with histopathological evidence of a combined Gleason score of 3+4 or less, and / or who are candidates for active surveillance.

[0358] Two populations of subjects were enrolled. The first population, designated "Population A," consisted of men with biopsy-proven low- to intermediate-grade prostate cancer (i.e., biopsy-assigned a combined Gleason score of 3+3 or 3+4) who were candidates for active surveillance but decided to undergo radical prostatectomy (RP). The second population, designated "Population B," consisted of men with biopsy-proven very low-risk prostate cancer who were scheduled to undergo periodic repeat biopsies as part of routine active surveillance.

[0359] The study objectives were to evaluate (1) the specificity of 1404 SPECT / CT imaging for identifying subjects without clinically significant prostate cancer (e.g., a combined Gleason score of 3+4 or less) and (2) the sensitivity of 1404 SPECT / CT imaging for identifying subjects with clinically significant prostate cancer (e.g., a combined Gleason score of greater than 3+4).

[0360] All enrolled subjects received 20±3 mCi (740±111 MBq) 99m A single dose of TC-MIP-1404 was injected, and SPECT / CT imaging and whole-body 2D imaging of 1404 were performed 3 to 6 hours after injection. According to the criteria for treatment protocol, subjects underwent optional RP surgery (Cohort A) or prostate biopsy (Cohort B) within 42 days of study drug administration, after which specimens were histologically evaluated. Histopathological examination was performed on the prostate specimen (Cohort A) or biopsy material (Cohort B) to establish each patient's ground truth prostate cancer status. The primary pathologist was blinded to all clinical data, including imaging results.

[0361] The images were analyzed using a traditional manual reading technique. Image analysis and diagnosis performed in this manner will hereinafter also be referred to as "standalone." Standalone analysis was performed using traditional SPECT / CT imaging workstation software, such as MIMvista from MIM Software Inc., commonly found in most hospitals. FIG. 38A shows a screenshot of the MIMvista GUI as used in a traditional standalone workflow. In the standalone workflow, the image viewer (e.g., a doctor or technician) viewed the CT and SPECT images as 2D slices. As shown in FIG. 38A, sets of CT slices 3802a, 3802b, and 3802c are shown along with corresponding SPECT image slices 3804a, 3804b, and 3804c. The viewer (e.g., manually) positioned the prostate within the CT image slice (e.g., 3802a). The viewer then manually scrolled through the slices to identify the slice with the greatest uptake in the SPECT image. Once the viewer identified this maximum uptake slice, they (manually) placed a fixed-size circular marker 3806 in the corresponding slice in the CT image to identify the region of interest, as shown in Figure 38B. The maximum uptake in this region of interest (i.e., within the circle) was then recorded (traditional software such as MIMvista only displays this number along with other values ​​such as the average uptake).

[0362] Moving to FIG. 38C, the viewer performed a similar procedure to determine the background value. The viewer placed a circular marker on the obturator muscle near the prostate to identify the region of interest 3808 and recorded the average uptake in this region as the background value. The TBR was then calculated as the ratio of the maximum uptake value to the background value. Notably, in this conventional approach, automatic 3D segmentation was not performed. Instead, the viewer inspected the SPECT / CT image slice by slice to ultimately identify the two-dimensional region of interest within the CT image slice. Additionally, the obturator muscle was used as the reference tissue region, as opposed to using the gluteus muscle as the reference tissue for determining the background value, as is done using the PIONEER system.

[0363] In Study 3301, three independent, blinded physician readers analyzed 1404 SPECT / CT images and assigned each subject a clinically significant / insignificant prostate cancer status based on the determined TBR values. The primary histopathological assessment, along with the results of the image-based analysis compared with the histopathological results, was used as ground truth to assess the diagnostic performance of the 1404 SPECT / CT images. ii.3302 study

[0364] The 3302 study builds on the 3301 study, using the same patient population and imaging data, but assesses the performance of the AI-assisted image analysis methods described herein to detect clinically significant prostate cancer. Specifically, the 3302 study is a predefined retrospective study based on analysis of datasets (SPECT / CT images and histopathology) derived from the 3301 study. As in the 3301 study and as described herein, the primary histopathology evaluation was used as ground truth for performance assessment. The two primary endpoints of the 3302 study are: As a combination product for detecting clinically significant prostate cancer 99mThe specificity of Tc-MIP-1404 and PIONEER when compared with histopathological diagnosis after either radical prostatectomy (Group A) or prostate biopsy (Group B) As a combination product for detecting clinically significant prostate cancer 99m The sensitivity of Tc-MIP-1404 and PIONEER when compared with histopathological diagnosis after either radical prostatectomy (Group A) or prostate biopsy (Group B).

[0365] The specific workflows and analytical protocols used, and the results of comparing the performance of the AI-assisted approach for Study 3302 with the sole manual approach for Study 3301, are described herein. iii. AI-assisted image analysis workflow

[0366] Analysis of images with PIONEER assistance in the 3302 study was performed using a version of the quality control and reporting workflow described herein (e.g., with respect to FIG. 25). FIG. 39 shows a block flow diagram of the particular workflow 3900 used in the 3302 study. As shown, a technician performed a group of steps 3902 to upload and initiate the initial automated analysis by the PIONEER software, after which a physician reviewed the analysis and performed the steps of the quality control and reporting workflow 3904.

[0367] The technician was responsible for uploading the SPECT / CT image data 1404 and ensuring that imaging requirements were met. The automated analysis then began, resulting in one of three results presented by the analysis GUI window, as shown in FIGS. 40A-40C. FIG. 40A shows a screenshot of the analysis GUI window used to indicate that the automated analysis was successfully completed and the images were ready for review (e.g., by a physician). FIG. 40B shows a screenshot of the analysis GUI window indicating that the automated analysis was not completed due to an analysis error; in this case, manual guidance performed by a physician was used to complete the analysis (which was also considered ready for review). FIG. 40C shows a screenshot of the analysis GUI window used to indicate that the automated analysis was not completed due to incomplete data, requiring correction of imaging issues and / or re-uploading of the data by a technician.

[0368] The physician followed the workflow of Figure 39 to review the images using a GUI similar to that described herein. Figures 41-44 show screenshots of various windows of the GUI that the physician uses to analyze the images and perform a diagnosis. Figure 41 is a screenshot showing a list of patients to be reviewed by the physician. The physician viewer selected a particular patient to review their data and assign a prostate cancer status.

[0369] The physician viewer selected a patient and reviewed the results of the automated analysis. Figures 42A-42E show screenshots of the analysis GUI view used by the physician to review the results of the automated analysis performed by the PIONEER software. The analysis GUI implemented in the PIONEER software used in the 3302 study is similar to that shown and described herein in Figures 29A-29H. Figure 42A shows a screenshot of the analysis GUI view presented to the physician. The screenshot in Figure 42A is similar to the view shown in Figure 29A, displaying a 2D slice of SPECT image data overlaid on the CT image data and a segmentation mask identifying the prostate. Similar to Figure 29A, the GUI view in Figure 42A also shows the automatically determined TBR value and the assignment of a prostate cancer status (e.g., "clinically significant") based on a comparison of the TBR value to a threshold value as described herein. FIG. 42B shows another screenshot of the viewer shown in FIG. 42A, in which the physician has switched to a 3D volumetric view of the SPECT / CT image data and segmentation mask.

[0370] As shown in Figure 42A and in more detail in Figure 42C, the analysis GUI used in Study 3302 also includes a likelihood severity scale that assigns a subject's result to one of four categories of clinical significance. Specifically, in addition to the binary classification of clinically significant or not, a subject's prostate cancer status was assigned to one of four categories based on a comparison of the calculated TBR value to three thresholds. On the likelihood severity scale, TBR values ​​below a first threshold were assigned a "very unlikely" status, TBR values ​​at or above the first threshold but below a second threshold were assigned a "unlikely" status, TBR values ​​between the second and third thresholds were assigned a "likely" status, and values ​​above the third threshold were assigned a "very likely" status.

[0371] The three TBR thresholds used to assign subjects to categories on the likelihood severity scale were determined using the ROC analysis described in Example 4 with reference to Figures 35A and 35B. Each of the three thresholds was selected to result in a particular pair of sensitivity and specificity values. Table 2 below shows the same data as Table 1, but was used to display specificity and sensitivity data for additional TBR thresholds. Figures 45A and 45B replot the data shown in Figures 35A and 35B, but three TBR thresholds were used to assign subjects to the indicated likelihood severity scale categories. As shown in Table 2 and Figures 45A and 45B, a first, lower threshold of 12 was selected to result in a target sensitivity of 0.95; a second, intermediate threshold of 25 was selected to result in a sensitivity of 0.75 and a specificity of 0.75; and a third, higher threshold of 45 was selected to result in a specificity of 0.9. Table 3 below shows the lower bounds of the Jefferies one-sided 95% confidence intervals (CI) for each of these three thresholds. [Table 2] [Table 3]

[0372] Figures 45C and 45D show how these thresholds and corresponding categories on the likelihood severity scale relate to the total Gleason scores (ranging from 2 to 10) of subjects in the Phase 1 and Phase 2 populations used to determine the thresholds. The results shown in Figures 45C and 45D show the likelihood severity scale using an indicated four-category labeling scheme, which was later updated to the four equivalent categories shown in Figures 42A and 42C. The "negative" (referred to as "very unlikely" on the scale shown in Figure 42C) category 4501 corresponds to TBR values ​​below 12. The "probably negative" (referred to as "unlikely" on the scale shown in Figure 42C) category 4502 corresponds to TBR values ​​ranging from 12 to 25. As shown in Figures 45A and 45B, patients assigned to the "negative" 4501 and "probably negative" 4502 categories based on TBR values ​​typically have a total Gleason score below 7. The "probably positive" (referred to as "highly likely" on the scale shown in Figure 42C) category 4503 corresponds to TBR values ​​ranging from 25 to 45 and captures total Gleason scores of 7 and 8. Patients with TBR values ​​above 45 are assigned to the "positive" (referred to as "very likely" on the scale shown in Figure 42C) category 4504 and will typically have a total Gleason score of 9 or 10.

[0373] As previously described herein, the physician was given the option to accept the automated assessment or to override the fully automated assessment and update the target (maximum uptake) and background values ​​used for the TBR calculation. Figure 42D shows a screenshot of the GUI view used by the physician to update the target and background values ​​used to calculate the TBR. As shown in Figure 42E, the physician was also given the option to update the clinical significance classification.

[0374] As shown in Figure 39, once the physician completed the quality control workflow, a report was generated at 3903. In the report generated by fully automated assessment shown in Figure 43A, the physician only approved the TBR calculation and clinical significance status determined by the PIONEER software's automated analysis. In the report generated by semi-automated analysis shown in Figure 43B, the physician, assisted by the software, updated the target and background values. Figure 44 shows a report generated for an unassessable case. iv.3302 test results

[0375] Study 3302 used three independent new readers (different from those who performed the data analysis in Study 3301) to analyze 1404 SPECT / CT images using the workflow described herein to identify patients with or without clinically significant prostate cancer status. Thirteen of the 464 cases in Study 3301 were excluded due to missed pathology (three cases) or technical issues (10 cases), leaving a total of 451 cases for analysis in Study 3302. The first reader, referred to as "AI Viewer 1," analyzed 451 cases, the second reader, referred to as "AI Viewer 2," analyzed 450 cases, and the third reader, referred to as "AI Viewer 3," analyzed 441 cases.

[0376] As shown in Figure 46, the AI-assisted analysis approach described herein enabled physicians to rapidly assess the clinical significance and severity of each patient's prostate cancer, primarily based on the fully automated assessment generated by the PIONEER software. The histogram in Figure 46 shows the number of cases each reader was able to process within a specific time frame. Readers using an embodiment of the PIONEER software averaged 3.5 minutes per case in Study 3302. In contrast, with the manual reading approach in Study 3301, for each case, a technician spent approximately 5-10 minutes manually placing the regions of interest in the prostate and background (obturator membrane) as described above, followed by an additional 5-10 minutes for the radiologist to evaluate and document this in the case report. Thus, what would have taken as much as 20 minutes per case was reduced to an average of 3.5 minutes with the AI-assisted approach, resulting in a significant improvement in efficiency.

[0377] Furthermore, as shown in Figure 47, the AI-assisted approach to image analysis and prostate cancer status diagnosis in Study 3302 yielded highly consistent and reproducible results. Consistency in results refers to the similarity of results across separate viewers. Reproducibility refers to intraviewer variability, i.e., the degree of similarity between results generated by a single viewer performing repeated analyses.

[0378] The set of graphs in Figure 47 demonstrates the consistency and reproducibility of the results obtained by the AI-assisted approach in the 3302 study. In particular, as shown in histogram 4702 in the figure, the vast majority of cases, 69% to 85%, were processed fully automatically, with the physician viewers simply approving the binary classification performed by the PIONEER system. The remaining cases were processed semi-automatically, with the physician intervening to update the maximum intake target value.

[0379] The primary reason for the viewer to intervene to select a new, maximum-intensity image voxel for the target value was due to bladder intensity outflow into the prostate region in the SPECT images, which is believed to be primarily caused by Compton scattering. The PIONEER system included a bladder outflow correction function as described herein. However, in certain cases (particularly in the mild patient population with small prostate uptake examined in Studies 3301 and 3302), the viewer objected to the automated analysis (e.g., believing the automated selection of the target voxel to be inaccurate) and used assistance from the PIONEER system to update the selection of the target voxel.

[0380] The high acceptance rate of fully automated results is noteworthy because fully automated diagnosis eliminates intra- and inter-reader variability (allowing for completely consistent and reproducible results).

[0381] The scatter plots on the right side of Figure 4710a, Figure 4710b, and Figure 4710c (collectively Figure 4710) and Figure 4720a, Figure 4720b, and Figure 4720c (collectively Figure 4720) show variability across readers. Each plot compares the TBR results obtained by a specific AI-assisted reader with those obtained by an internal reference reader (who also used PIONEER software and followed the same protocol as the three readers in the 3302 study). Each point represents a case, with the y-axis value giving the TBR value calculated by the internal reader and the x-axis value giving the TBR value calculated by the specific reader. Scatter plot 4700 shows a case performed by fully automated assessment, eliminating inter-reader variability as evidenced by a perfect correlation coefficient (r=1), allowing for completely consistent results. A case analyzed using the semi-automated method described herein is shown in plot 4720. The scatter plot and large correlation coefficient (better than r = 0.95) indicate that highly consistent results were obtained even when viewers intervened to update their selection of the target voxel. Thus, although the placement of this target voxel was subjective, the PIONEER system aided in enabling highly consistent results.

[0382] The performance of the three AI-assisted viewers who analyzed the images in Study 3302, along with the performance of the three manual viewers from Study 3301, are shown in Figures 48A and 48B. Figure 48A shows the receiver operating characteristic (ROC) curves for each of the three AI-assisted viewers from Study 3302 and the three manual viewers from Study 3301. The ROC curves show how sensitivity and specificity change when the TBR threshold used to perform a binary positive / negative classification of a patient's clinically significant prostate cancer is varied. As described herein, the histopathology diagnosis was used as the ground truth for analyzing sensitivity and specificity in both Study 3301 and Study 3302. Table 4 below shows how the SPECT / CT imaging results of 1404 were compared with the histopathology diagnosis to determine whether the imaging result was a true positive (TP), true negative (TN), false positive (FP), or false negative (FN) for each specific case. [Table 4]

[0383] A positive prostate histopathology diagnosis was defined as clinically significant disease reported in an electronic case report form after MIP-1404 SPECT / CT imaging in (i) at least one prostate lobe (for Cohort A) or (ii) prostate biopsy (for Cohort B). For Cohort A, laboratory assessment of the primary histopathology diagnosis of clinically significant disease was Any Gleason grade disease greater than 3+4 within the prostate gland, Any Gleason grade 3+4 disease within the prostate, with grade 4 greater than or equal to 10%; Supported by at least one of the following findings: stage T3 (pT3 according to the prostate cancer staging system) or any disease more severe inside the prostate.

[0384] If a subject in Cohort B subsequently underwent RP, then the histopathology results obtained from the prostate specimen were used as the reference standard, with the subject remaining in Cohort B for subgroup analyses.

[0385] A positive imaging assessment indicates the presence of prostate cancer as adjudicated by the imaging reader (ie, based on evaluation of 1404 SPECT / CT imaging data).

[0386] Specificity was defined as the proportion of true negatives (TN) among subjects with histopathologically defined clinically insignificant prostate cancer (i.e., subjects with a combined Gleason score of 3+4 or less). Specificity was calculated as follows: Specificity=TN / (FP+TN)

[0387] Sensitivity is the proportion of true positives (TP) among subjects with clinically significant prostate cancer as defined by histopathology (i.e., subjects with a combined Gleason score greater than 3 + 4). Specificity was calculated as follows: Sensitivity=TP / (TP+FN)

[0388] Varying the TBR threshold resulted in different rates of TP, FP, FN, and TN, and therefore different sensitivities and specificities. The ROC curves shown in Figure 48A plot the resulting changes in sensitivity and specificity for each reader as the TBR threshold was varied. As shown in the figure, all AI-assisted reader curves were above the manual reader curves, indicating that the AI-assisted approach improved performance. Furthermore, as shown in Figure 48B, area under the curve (AUC) values ​​were calculated for each reader, and the performance of the AI-assisted reader was improved compared to that of the manual reader (p-values ​​of p<0.05 for five of nine possible comparisons between one of three manual readers and one of three AI-assisted readers). Additionally, for a particular TBR threshold, comparing the TBR values ​​calculated by the AI-assisted approach for the 3302 study with this TBR threshold provided high specificity in achieving a binary classification of positive or negative for clinically significant prostate cancer. In particular, specificities of 95%, 96%, and 96% were found for patients with very low, low, and intermediate risk for prostate cancer, respectively, with a sensitivity of 21%.

[0389] Figures 49A-49C compare the performance of the internal reference reader (the three AI-assisted readers of Study 3302 compared to this person in Figure 47) with the performance of each of the manual readers of Study 3301. As evidenced by the data shown in Figure 47, the performance of the internal reference reader is highly correlated with and representative of the three AI-assisted readers of Study 3302. Figures 49A-49C compare individual ROC models for binary classification (positive / negative for clinically significant disease based on comparison of TBR values ​​to a threshold) between the AI-assisted internal reference reader (referred to in the figures as "PIONEER Read") and each manual reader of Study 3301. In each case, the improvement in the AUC metric for the AI-assisted reader compared to the manual reader was statistically significant (p-values ​​of 0.01, 0.02, and 0.045).

[0390] As shown in Figures 49A-49C, the AUC for the AI-assisted internal reference reader was 0.66 (95% CI: 0.61-0.71). Logistic regression indicated an optimal TBR threshold of 14.4, with a sensitivity of 65% (95% CI: 61%-70%) and a specificity of 62% (95% CI: 58%-67%). Automated performance was significantly superior to manual reader 1 (AUC 0.63, p = 0.0445), manual reader 2 (AUC 0.62, p = 0.0201), and manual reader 3 (AUC 0.62, p = 0.0122). Additionally, when a multivariable model including baseline covariates (taking into account patient age, weight, and PSA score) was used, the performance of the AI-assisted approach in Study 3302 was compared to the manual approach in Study 3301. For multivariable models, the AI-assisted internal norm reader (AUC 0.80) demonstrated higher accuracy in predicting clinically significant disease than the manual reader (AUC 0.77) (p=0.0014).

[0391] Thus, this example demonstrates that the combination of physician-assisted PIONEER assessment with 1404 imaging data represents a significant improvement over manual reviewers in specificity and sensitivity when assessing the clinical significance of prostate cancer. In addition, PIONEER-assisted physicians are highly efficient and produce highly reproducible results when analyzing imaging data, along with useful information regarding the clinical aggressiveness of the diagnosis. Example 6 L. Example 6: Training and Validation of Convolutional Neural Networks Implemented by a First Machine Learning Module (Localization Machine) and a Second Machine Learning Module (Segmentation Machine)

[0392] Example 6 is an example illustrating the training and validation of a CNN module used to segment CT images to identify various tissue volumes, including prostate volumes, according to aspects and embodiments described herein. In this example, the neural network was defined and trained using the Keras framework with the Tensorflow backend. i. Training and validation data

[0393] The training and validation data included CT images combined with radiologist-corrected semi-automated segmentation to outline all or some of the following body parts: (i) prostate, (ii) bladder, (iii) rectum, (iv) left gluteus maximus, (v) right gluteus maximus, (vi) left hipbone, (vii) right hipbone, and (viii) sacrum and coccyx.

[0394] To train the localization CNN, 90 high-quality CT images were used with segmentations of all the above body parts. These CT images and 10 images of the same type were used to train the segmentation CNN for high-resolution segmentation of the left gluteus maximus muscle. To train the segmentation CNN for high-resolution segmentation of the prostate, an additional dataset of 73 lower-quality images with more anatomical variation (due to disease) in the prostate and bladder was used. These additional images were matched only with the prostate and bladder segmentations. For these 73 images, provisional landmarks were generated for the rectum, left and right hip bones, and sacrum and coccyx. The provisional landmarks were predicted landmarks generated using a pre-trained network (trained on 100 high-quality CT images). The provisional landmarks were merged with manual landmarks before training. ii. Training structure

[0395] Each training run was defined by a configuration file named config.json, which contained parameters for (i) which dataset to use, (ii) data preprocessing, (iii) partitioning of data into training and validation sets, (iv) model structure, and (v) training hyperparameters.

[0396] This configuration file is used to ensure that when a trained model is used for inference, the same image preprocessing is done during inference as during training. iii. Pretreatment

[0397] The configuration file defines the preprocessing steps applied during training. Several preprocessing steps were included. For localization training, a crop preprocessing step was performed to remove ambient air and expose a bounding box defined by the pelvic bone, bladder, rectum, and prostate for segmentation. A constant intensity normalization step was performed by subtracting a fixed value and then dividing by another fixed value; these values ​​were chosen so that, on average, the intensity of voxels in the image had a mean intensity of 0 and a standard deviation of 1. The images were also reshaped to a fixed size by resampling. For localization, the resolution in each direction was reduced by a factor of 4 for images with a median size in each direction. For high-resolution segmentation, the resolution remained unchanged for images with a median size in each dimension. Other preprocessing steps included one-hot encoding of segmentation labels, reweighting of segmentation labels as defined in Section iv, "Optimization," below, and data augmentation as defined in Section iv, "Optimization," below.

[0398] Some preprocessing steps are performed offline to speed up training, resulting in a preprocessed dataset. The config.json pertaining to training only defines the preprocessing steps that are done online.

[0399] For localization training, the preprocessing steps for offline preprocessing are stored in a separate file meta_data / prep_config.json in the directory of the trained CNN. iv. Optimization

[0400] The loss function used for training is voxel-wise classifier cross-entropy, weighted so that all voxels that belong together to one class (in the true labeling) have the same weight as all voxels that belong to any other class (when the label frequency is equal to the median frequency across the dataset). This approach balances the classes, which is important for prostate segmentation because the prostate is very small compared to, for example, the background.

[0401] The loss is optimized using mini-batch gradient descent with the Adam optimizer for approximately 2000-3000 epochs. The batch size is 1. A batch size of 1 means that batch normalization becomes instance normalization (since the batch is only one example). Instance normalization is also enforced during inference to improve performance.

[0402] The learning rate is set to a very low value (e.g., 1×10) for the first few epochs. -5 ) learning rate schedule, followed by a high learning rate (e.g., 1 × 10 -2 ) at each step. The learning rate was halved after 250 epochs, then halved again after the following 500 epochs, and then halved again after the following 1000 epochs. Dropout is used to reduce overfitting. The dropout rate varies between 0.2 and 0.5 for different networks (but is the same at all positions when applied to a given network, as shown, for example, in Figures 13 and 14). The appropriate dropout rate will be based on experience with multiple training runs and how dropout affects the difference in performance on the training and validation sets.

[0403] Training techniques that use training image data augmented by adding random distortions prevent neural networks from focusing on fine details in the image. Neural networks (e.g., localization CNNs and segmentation CNNs) trained in this way are able to treat (e.g., perform effective localization and / or segmentation) images with image artifacts as if the artifacts were not present or common in the training data.

[0404] Random distortions were added to augment the training data, including additive or multiplicative noise and smoothed salt noise. These random distortions were applied separately with a certain probability. These random distortions were scheduled not to be used for the first few hundred epochs so that the network could first learn to handle artifact-free images, and then the noise level was gradually increased. Intensity distortions were only applied to the 100 high-quality CT images and not to the 73 low-quality images, as they already contained artifacts.

[0405] The localization CNN is trained to handle (e.g., take as input) whole-body images as well as images of different body parts. For this purpose, augmentation with random cropping of the images (always preserving the entire bounding box of the pelvis) is used. v. Computational Resources

[0406] Localization training was performed on an Nvidia GeForce GTX 1050 and took several hours to train.

[0407] High-resolution segmentation training was performed on an Nvidia GeForce GTX 1080 Ti and took 2-3 days to train. vi. Model selection

[0408] To optimize the neural network structure and select hyperparameters for training, we need to obtain a metric to measure performance. The primary metric for evaluating training progress is the Sorensen-Dice score (hereafter referred to as the "Dice score"), either a weighted score (as described in Section iv "Optimization" above) or a score for an individual body part (prostate or left gluteus maximus).

[0409] When aggregating results across multiple images, in most cases the average of the evaluated Dice scores is used (e.g., as a metric). In one particular case, the frequency of images with Dice scores below a certain level is used.

[0410] For each localization training run, the model with the best average weighted Dice score for the training images was selected. For segmentation training used to train a segmentation neural network to segment the prostate, the Dice score for the prostate was used as the basis for selection. For segmentation training used to train a segmentation neural network to segment the left gluteus maximus, the Dice score for the left gluteus maximus was the basis for selection. For segmentation training using auxiliary prediction, the Dice score was based on prediction from the main output (the output that is also present when there is no auxiliary prediction).

[0411] When training the localization CNN, 30% of the 90 CT images were set aside for validation so that the performance of the localization CNN could be evaluated using images on which the localization CNN was not trained. 23 of the low-quality CT images were set aside for validation so that only 50 of the low-quality images were used to train the segmentation CNN for high-resolution prostate segmentation (for a total of 150 training CT images).

[0412] When choosing between different localization training images, a metric corresponding to the crop accuracy was used. The crop accuracy was evaluated based on several metrics / aspects. In particular, one metric was to minimize the number of images among the 90 training and validation images that required a crop margin greater than 0.1 to encompass the entire pelvic bone. Another metric was to minimize the error in the distance to the bounding box walls (assessed by examining the error box plot). Another metric was to minimize the error in the distance to the bounding box walls (assessed by examining the error box plot) in 102 low-quality CT images (without available ground truth segmentation). The 2D projection of the coarse segmentation onto them and the superimposed final bounding box should show good agreement between the segmentation and the anatomical structures, especially covering the appropriate region.

[0413] When choosing between different high-resolution segmentation training, the following metrics / aspects were taken into consideration: (i) Dice score on training and validation data, (ii) precision and recall of the prostate on training and validation data, (iii) overlap between the ground truth bladder and predicted prostate, and (iv) segmentation examples superimposed on the CT image in a CT image viewer that allows scrolling through slices in sagittal, axial, and coronal planes.

[0414] Thus, this example provides an example technique that can be used to train the neural network models used in the localization machine learning module (first machine learning module) and segmentation machine learning modules (second machine learning module and any auxiliary segmentation machine learning modules) described in this specification. M. Contrast agents

[0415] In certain embodiments, the 3D functional image is a nuclear medicine image using a contrast agent containing a radiopharmaceutical. The nuclear medicine image is obtained after administering the radiopharmaceutical to a patient and provides information about the distribution of the radiopharmaceutical within the patient. A radiopharmaceutical is a compound containing a radionuclide.

[0416] Nuclear medicine images (e.g., PET scans, SPECT scans, whole-body scans, combined PET-CT images, combined SPECT-CT images) detect radiation emitted from radionuclides in radiopharmaceuticals to form images. The distribution of a particular radiopharmaceutical within a patient can be determined by biological mechanisms such as blood flow or perfusion, as well as specific enzyme or receptor binding interactions. Different radiopharmaceuticals may be designed to utilize different biological mechanisms and / or specific specialized enzyme or receptor binding interactions, and thus, when administered to a patient, selectively concentrate within specific types of tissue and / or regions within the patient. Regions within the patient where the concentration of the radiopharmaceutical is higher than other regions emit more radiation, and these regions appear brighter in a nuclear medicine image. Therefore, intensity variations within a nuclear medicine image can be used to map the distribution of the radiopharmaceutical within the patient. This mapped distribution of the radiopharmaceutical within the patient can be used, for example, to infer the presence of cancerous tissue within various regions of the patient's body.

[0417] For example, technetium-99m methylenediphosphonate ( 99m When administered to a patient, the radiopharmaceutical (Tc MDP) selectively accumulates within the patient's skeletal region, particularly at sites with abnormal bone formation associated with malignant bone lesions. The selective concentration of the radiopharmaceutical at these sites produces identifiable hot spots (localized areas of high intensity in nuclear medicine images). Therefore, by identifying such hot spots within a patient's whole-body scan, the presence of malignant bone lesions associated with metastatic prostate cancer can be inferred. Risk indices that correlate with a patient's overall survival and other prognostic metrics, such as disease status, progression, and response, can be provided to patients. 99mIt can be calculated based on an automated analysis of intensity changes in whole-body scans obtained after administration of Tc MDP. 99m Other radiopharmaceuticals may also be used in a manner similar to Tc MDP.

[0418] In certain embodiments, the particular radiopharmaceutical used will depend on the particular nuclear medicine imaging modality being used. For example: 18 Sodium fluoride (NaF) is also 99m Similar to Tc MDP, it accumulates in bone lesions but can be used in conjunction with PET imaging. In certain embodiments, PET imaging may utilize the radioactive vitamin choline, which is readily taken up by prostate cancer cells.

[0419] In certain embodiments, radiopharmaceuticals may be used that selectively bind to specific proteins or target receptors, and thereby increase their expression, particularly in cancer tissues. Such proteins or target receptors include, but are not limited to, tumor antigens such as CEA expressed in colorectal cancer, Her2 / neu expressed in multiple cancers, BRCA1 and BRCA2 expressed in breast and ovarian cancers, and TRP-1 and TRP-2 expressed in melanoma.

[0420] For example, human prostate-specific membrane antigen (PSMA) is upregulated in prostate cancer, including metastatic disease. PSMA is expressed by virtually all prostate cancers, and its expression is further increased in poorly differentiated metastatic cancers and hormone-refractory cancers. Thus, radiopharmaceuticals corresponding to PSMA-binding agents (e.g., compounds with high affinity for PSMA) labeled with one or more radionuclides can be used to obtain nuclear medicine images of a patient, from which the presence and / or status of prostate cancer within various regions of the patient (e.g., including, but not limited to, the skeletal region) can be assessed. In certain embodiments, nuclear medicine images obtained using PSMA-binding agents are used to identify the presence of cancerous tissue within the prostate when the prostate cancer is localized. In certain embodiments, nuclear medicine images obtained using radiopharmaceuticals containing PSMA-binding agents are used to identify the presence of cancerous tissue within various regions of not only the prostate but also other organs and tissue regions, including the lungs, lymph nodes, and bones, as is appropriate when the disease is metastatic.

[0421] Specifically, when administered to a patient, the radionuclide-labeled PSMA-binding agent selectively accumulates within cancer tissue based on its affinity for PSMA. The selective concentration of the radionuclide-labeled PSMA-binding agent at specific sites within the patient 99m Similar to that described above with respect to the Tc MDP, this produces detectable hot spots in nuclear medicine images. Because the PSMA-binding agent concentrates within various cancerous tissues and regions of the body that express PSMA, localized cancer within a patient's prostate and / or metastatic cancer in various regions of the patient's body can be detected and assessed. As described below, risk indices correlating with patient overall survival and other prognostic metrics indicative of disease state, progression, cure, etc., can be calculated based on automated analysis of intensity changes in nuclear medicine images obtained after administering the PSMA-binding agent radiopharmaceutical to the patient.

[0422] Various radionuclide-labeled PSMA binding agents can be used as radiopharmaceutical imaging agents for nuclear medicine imaging to detect and evaluate prostate cancer. In certain embodiments, specific radionuclide-labeled PSMA binding agents are used depending on factors such as the specific imaging modality (e.g., PET, e.g., SPECT) and the specific region (e.g., organ) of the patient to be imaged. For example, certain radionuclide-labeled PSMA binding agents are suitable for PET imaging, while others are suitable for SPECT imaging. For example, certain radionuclide-labeled PSMA binding agents facilitate imaging of the patient's prostate and are primarily used when the disease is localized, while other binding agents facilitate imaging of organs and regions throughout the patient's body and are effective in evaluating metastatic prostate cancer.

[0423] Various PSMA binding agents and radionuclide-labeled versions of these binding agents are described in U.S. Pat. No. 8,778,305, U.S. Pat. No. 8,211,401, and U.S. Pat. No. 8,962,799, each of which is incorporated by reference herein in its entirety. i. PET imaging of radionuclide-labeled PSMA binders

[0424] In certain embodiments, the radionuclide-labeled PSMA binding agent is a radionuclide-labeled PSMA binding agent suitable for PET imaging.

[0425] In certain embodiments, the radionuclide-labeled PSMA binding agent is 18 F]DCFPyL (also referred to as PyL™, DCFPyL- 18 (also called F): [ka] or a pharmaceutically acceptable salt thereof.

[0426] In certain embodiments, the radionuclide-labeled PSMA binding agent is 18F]DCFBC: [ka] or a pharmaceutically acceptable salt thereof.

[0427] In certain embodiments, the radionuclide-labeled PSMA binding agent is 68 Ga-PSMA-HBED-CC( 68 Also called Ga-PSMA-11): [ka] or a pharmaceutically acceptable salt thereof.

[0428] In certain embodiments, the radionuclide-labeled PSMA binding agent is PSMA-617: [ka] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA binding agent is 68 Ga-PSMA-617, which 68 In certain embodiments, the radionuclide-labeled PSMA-binding agent is: 177 Lu-PSMA-617, which is 177 Lu-labeled PSMA...

Claims

[Claim 1] Devices, systems, methods, etc.

Images