System and method for platform-neutral whole-body image segmentation

Abstract

Presented herein are systems and methods for automatically analyzing three-dimensional (3D) medical images of an individual in order to automatically identify specific 3D volumes within the 3D images that correspond to specific anatomical regions (e.g., organs and / or tissues). Notably, the image analysis methods described herein are not limited to a single specific organ or part of the body. Instead, they are relatively stable and widely applicable, thereby providing consistent, effective, and accurate detection of anatomical regions (including soft tissue organs) throughout the body. In certain embodiments, accurate identification of one or more such volumes is used to automatically determine quantitative measurements representing the absorption of radiopharmaceuticals in specific organs and / or tissue regions. These absorption measurements can be used to assess the disease state of an individual, determine the prognosis of an individual, and / or determine the efficacy of a treatment modality.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 789,155, filed January 7, 2019, U.S. Provisional Application No. 62 / 837,941, filed April 24, 2019, U.S. Provisional Application No. 62 / 863,608, filed June 19, 2019, U.S. Provisional Application No. 62 / 870,210, filed July 3, 2019, U.S. Provisional Application No. 62 / 907,158, filed September 27, 2019, and U.S. Provisional Application No. 62 / 934,305, filed November 12, 2019, the contents of each of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present invention generally relates to methods, systems, and architectures for automated analysis of medical images. More particularly, in certain embodiments, the present invention relates to automated identification of one or more specific regions of interest (e.g., corresponding to specific organs or tissues) within an image of an individual. Background Art

[0004] Targeted image analysis involves the use of radiolabeled small molecules that bind to specific receptors, enzymes, and proteins in the body that change during the course of the disease. After being administered to a patient, these molecules circulate in the blood until they find their intended target. The bound radiopharmaceutical remains at the site of the disease, while the remainder of the agent is cleared from the body. The radioactive portion of the molecule acts as a beacon, allowing images depicting the location and concentration of the disease to be obtained using common nuclear medicine cameras found in most hospitals around the world (known as single-photon emission computerized tomography (SPECT) or positron emission tomography (PET) cameras). Physicians can then use this information to determine the presence and extent of the patient's disease. Physicians can use this information to provide patients with a recommended course of treatment and track the progression of the disease.

[0005] There are a variety of software-based analysis techniques that can be used by radiologists or physicians to analyze and enhance PET and SPECT images. There are also a large number of radiopharmaceuticals that can be used to image specific types of cancer. Imaging agents used in the art include, but are not limited to, 18 F-NaF, 11C-choline, 2-deoxy-2[18F]fluoro-d-glucose (FDG), and the like. For example, the small molecule diagnostic agent 1404 targets the extracellular domain of prostate-specific membrane antigen (PSMA), a protein amplified on the surface of >95% of prostate cancer cells and a recognized target for detecting primary and metastatic prostate cancer. 1404 is labeled with technetium-99m, a widely available, relatively inexpensive gamma-emitting isotope that facilitates efficient production and possesses attractive spectral characteristics for nuclear medicine imaging applications.

[0006] Another example radiopharmaceutical is PyL TM (Also known as [ 18 F]DCFPyL and 18F-PyL), which are clinical-stage fluorinated PSMA-targeted PET imaging agents for prostate cancer. A proof-of-concept study published in the April 2016 issue of the Journal of Molecular Imaging and Biology demonstrated that PyL TM PET imaging of the tumor revealed high levels of PyL in sites of putative metastatic disease and primary tumors. TM absorption, thus demonstrating the potential for high sensitivity and specificity in detecting prostate cancer.

[0007] An oncologist can use images from a targeted PET or SPECT study of a patient as input to their assessment of whether the patient has a particular disease (e.g., prostate cancer), what stage of the disease is evident, what the proposed course of treatment (if any) would be, whether surgical intervention is indicated, and the likely prognosis. The oncologist can use a radiologist's report in this assessment. A radiologist's report is a technical assessment of the PET or SPECT images prepared by a radiologist for the physician requesting the imaging study and includes, for example, the type of study performed, the clinical history, comparisons between the images, the techniques used to perform the study, the radiologist's observations and findings, and the radiologist's overall impression and recommendations based on the results of the imaging study. The signed radiologist's report is sent to the physician who ordered the study for review, followed by a discussion between the physician and the patient regarding the results and treatment recommendations.

[0008] Thus, the process involves having a radiologist perform an imaging study on a patient; analyzing the images obtained; creating a radiologist report; forwarding the report to the requesting physician; having the physician formulate an evaluation and treatment recommendation; and having the physician communicate the results, recommendations, and risks to the patient. The process may also involve repeating an imaging study due to an inconclusive result or ordering additional tests based on the initial result.

[0009] If the imaging studies show that the patient has a particular disease or condition (eg, cancer), the physician will discuss different treatment options, including surgery, and the risks of doing nothing or taking a cautious wait or active surveillance approach rather than surgery.

[0010] There are limitations associated with this process, both from the physician's perspective and from the patient's perspective. While the radiologist's report is certainly helpful, the physician must ultimately rely on his or her experience in formulating an assessment and recommendation for his or her patient. In addition, the patient must have a great deal of trust in his or her physician. The physician can show the patient his or her PET / SPECT images and can inform the patient of the numerical risks associated with different treatment options or the likelihood of a particular prognosis, but the patient may have difficulty understanding this information. In addition, the patient's family will likely have questions, especially if cancer is diagnosed but the patient chooses not to undergo surgery. The patient and / or their family members may search online for additional information and may develop misunderstandings about the risks of the diagnosed condition. A difficult ordeal can become even more traumatic.

[0011] Therefore, there remains a need for systems and methods for improving the automated analysis of medical imaging studies and communicating those results, diagnoses, prognoses, treatment recommendations, and associated risks to patients. In particular, there is a need for an image analysis system that continuously, efficiently, and accurately detects anatomical regions (including soft tissue organs) throughout the body. Summary of the Invention

[0012] Presented herein are systems and methods for automatically analyzing three-dimensional (3D) medical images of an individual to automatically identify specific 3D volumes within the 3D images that correspond to specific anatomical regions (e.g., organs and / or tissues). Notably, the image analysis methods described herein are not limited to a single specific organ or part of the body. Rather, they are relatively stable and broadly applicable, thereby providing consistent, efficient, and accurate detection of anatomical regions (including tissues and / or organs) throughout the body. In certain embodiments, accurate identification of one or more such volumes is used to automatically determine quantitative metrics representing the absorption of radiopharmaceuticals in specific organs and / or tissue regions. These absorption metrics can be used to assess the disease state of an individual, determine the prognosis of an individual, and / or determine the efficacy of a treatment modality.

[0013] The ability of the methods described herein to process 3D images is a significant advantage over certain other image analyses that only identify 2D regions within 2D images. For example, one method for cancer detection, the Bone Scan Indicator (BSI) software from EXINIDiagnostics AB, detects regions suspected of bone cancer (see also U.S. Patent Application No. 8,855,387, published on October 7, 2014). However, BSI analysis is performed on 2D scintigraphic images, as opposed to 3D images.

[0014] Furthermore, as the options for imaging agents available to physicians for detecting cancer continue to increase, there remains a need for software that is useful in analyzing images from any of the various imaging agents using multiple detection modalities (SPECT / CT, PET / CT, and the like). Functional images such as SPECT and PET provide detailed and specific information about biological processes in the body, but their potential is only realized when combined with detailed anatomical maps so that function can be localized to individual organs and structures. While CT and MRI provide detailed anatomical information, conventional (e.g., manual) identification of organs and structures is difficult, subjective, and time-consuming, making certain assessments infeasible without computer support. Therefore, the systems and methods described herein facilitate image analysis specifically related to cancer detection, diagnosis, staging, and the like by providing a platform-neutral image analysis method that allows accurate and robust image segmentation applicable to a variety of imaging modalities.

[0015] For example, the whole body segmentation method described herein allows for the automated analysis of a combination of anatomical and functional images to accurately identify and grade cancerous lesions within an individual. Specifically, in the administration of radiopharmaceuticals, such as PSMA binders (e.g., PyL TM ), a PET / CT composite image of the individual can be obtained. The machine learning-based automatic segmentation method described herein is used to identify a target volume of interest (VOI) within the CT image of the PET / CT composite that represents a target tissue region where a cancerous lesion may be found. For example, a bone VOI corresponding to a graphical representation of one or more bones of the individual can be identified. Once the bone VOI is identified in the anatomical, CT, image, it can be mapped to the PET image to identify the corresponding bone volume therein. The corresponding bone volume in the PET image is then analyzed to detect one or more localized areas of relatively high intensity (referred to as hot spots). These hot spots physically correspond to localized areas of increased accumulation of radiopharmaceuticals and, therefore, to potential cancerous lesions.

[0016] In certain embodiments, the ability to accurately and rapidly perform whole-body segmentation via the methods described herein is utilized to provide a useful and uniform scale on which to evaluate and / or measure the level of radiopharmaceutical uptake in physical lesions corresponding to detected hotspots (e.g., thereby grading the expression level of a particular biomolecule (e.g., PSMA)). Specifically, in addition to detecting target VOIs corresponding to specific target tissue regions where cancerous lesions may occur, additional target VOIs corresponding to reference tissue regions are detected. These reference VOIs are additionally mapped to the PET image to identify corresponding reference volumes therein. Intensity measurements (e.g., average, peak, maximum, etc.) within these reference volumes are then calculated and used as reference points against which the intensity of individual detected hotspots is evaluated and scaled to index values.

[0017] For example, in certain embodiments, aortic and liver VOIs corresponding to representations of a portion of the aorta and liver, respectively, are identified within the anatomical image and mapped to the functional image to identify corresponding reference volumes therein. The intensity level of each of these reference volumes is determined (e.g., as the mean, median, peak, etc., of the voxel intensities within each reference volume) and proportionally assigned a corresponding index level. Subsequently, for each specific individual hotspot, the hotspot intensity level is determined (e.g., similarly, as the mean, median, peak, etc., of the voxel intensities within the detected hotspot). A corresponding individual hotspot index value is then determined based on the individual hotspot intensity level, the aortic reference intensity level, and the liver reference intensity level. This method provides for evaluating and measuring standardized ratios of absorption associated with hotspots between different images. For example, this allows for comparison of multiple images acquired for a single individual at different time points, as well as comparisons between images of different individuals.

[0018] In certain embodiments, individual hotspot indices are used to calculate an overall index for analyzing the presence of cancerous lesions in individuals and / or specific target tissue regions. The overall index can serve as an indicator of disease severity and / or risk, for example by reflecting the total lesion volume within a specific target tissue region, using weighting based on the hotspot index values ​​of the individual detected hotspots to account for radiopharmaceutical uptake within the lesions. For example, the PSMA-weighted total lesion volume within a bone volume can be calculated as the weighted sum of the volumes of the individual detected hotspots weighted by their corresponding index values. This index can reflect the level and / or aggressiveness of cancer metastasis into the bone.

[0019] The volume corresponding to other tissue regions can be through similarly identifying and being used to measure overall index value.For example, lymph node volume also can be used for assessing the order of severity of cancer metastasis.Also can differentiate and analyze the volume corresponding to the tissue region of initial discovery localized disease to detect pathological changes.For example, prostate volume can be used for assessing the order of severity of prostate cancer in its initial stage.Similarly, the breast volume representing breast tissue region and the lung volume representing lung tissue region can be used for assessing localized breast cancer and lung cancer respectively.

[0020] Thus, by providing whole-body segmentation and automatic image-based analysis of cancerous lesions in various target tissue regions of interest throughout an individual's body, the AI-based systems and methods described herein allow for analysis of various cancers at different stages. The technology can be used to identify and stage localized disease in tissue regions of interest (e.g., prostate), for example, for initial screening and monitoring, as well as to monitor areas of cancer metastasis, such as bones and lymph nodes, as the disease progresses. Similarly, the methods described herein provide a complete set of tools for automatic image-based detection of cancer and tracking disease evolution, progression, and response to treatment.

[0021] In one aspect, the present invention relates to a method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a specific target tissue region, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of an individual obtained using an anatomical imaging modality (e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound), wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the individual; and (b) for each of a plurality of target tissue regions, performing, by the processor, a computational analysis using one or more machine learning modules (e.g., wherein at least one of the one or more machine learning modules is a convolutional neural network). and (d) storing and / or providing the 3D segmentation map for display and / or further processing.

[0022] In certain embodiments, the 3D anatomical image is a full body image.

[0023] In some embodiments, step (c) includes digitally stitching together multiple 3D segmentation masks to form a 3D segmentation map {e.g., by creating an initially empty image volume (e.g., initializing all voxel values ​​to zero) and then inserting labels from each segmentation mask into the image volume [e.g., by mapping labeled (e.g., as representing a specific target tissue region as determined by the machine learning module) voxels of the input image to one or more machine learning modules to voxels of the image volume (e.g., so as to match voxels of the image volume with voxels of the input image representing the same physical location, thereby correctly labeling the voxels of the image volume)]}.

[0024] In some embodiments, step (b) includes, for at least one specific target tissue region: determining, using a first module (e.g., a localization module) (e.g., a first machine learning module) an initial VOI within the 3D anatomical image, the initial VOI corresponding to an anatomical region (e.g., a group of related tissues, such as the pelvic region, the thoracic region, the head and / or neck region, and the like) that contains the specific target tissue region (e.g., wherein the initial VOI excludes more voxels in the 3D anatomical image than it contains; e.g., wherein the initial VOI includes less than 25% of the voxels in the 3D anatomical image; e.g., wherein a majority of the voxels within the initial VOI represent a physical volume within the anatomical region); and identifying, using a second module (e.g., a segmentation module) (e.g., a second machine learning module), a target VOI within the initial VOI that corresponds to the specific target tissue region.

[0025] In some embodiments, the second module is a CNN module that implements CNN.

[0026] In certain embodiments, the first module is a CNN module that implements a CNN to perform coarse segmentation to automatically identify an initial VOI corresponding to an anatomical region containing a specific target tissue region [e.g., by automatically identifying a graphical representation of a set of related tissues within the anatomical image (e.g., and then determining the initial VOI as a rectangular region (e.g., a rectangular column or rectangular box) that completely surrounds the identified graphical representation of the set of related tissues)] [e.g., for each of one or more specific tissue regions expected to be within the anatomical region, by automatically identifying a corresponding VOI in the anatomical image (e.g., via coarse segmentation), for example, and determining the initial VOI as a rectangular region that completely surrounds all identified VOIs corresponding to the specific tissue regions].

[0027] In some embodiments, a first module receives as input a subsampled [e.g., by a factor of two or more than two (e.g., four) along one or more dimensions] version of an anatomical image and uses the subsampled version of the anatomical image to identify an initial VOI [e.g., and wherein a second module receives a full-resolution version of the anatomical image cropped to the initial VOI (e.g., such that the first module operates on a lower-resolution image representing a larger physical volume than the second module, while the second module operates on an image of higher resolution but representing a smaller physical volume)].

[0028] In some embodiments, the first module is a first CNN module and the second module is a second CNN module, and wherein the first CNN module includes additional filters to account for a rate of change as the image size increases relative to the second CNN module.

[0029] In some embodiments, the 3D anatomical image is a whole-body image and step (b) includes: automatically determining a plurality of initial VOIs within the 3D anatomical image using one or more localization modules implementing machine learning techniques (e.g., wherein each localization module is a CNN module implementing a CNN), each initial VOI corresponding to a specific anatomical region (e.g., a group of related tissues, such as the pelvic region, the chest region, the head and / or neck region, the spinal region, the upper body region, the lower body region, etc.) and wherein a relevant subset of the target VOIs is located; and automatically identifying, for each initial VOI, using one or more segmentation modules implementing machine learning techniques (e.g., wherein each segmentation module is a CNN module implementing a CNN).

[0030] In some embodiments, the plurality of initial VOIs include one or more members selected from the group consisting of: a pelvic region initial VOI corresponding to the individual's pelvic region (e.g., wherein a subset of target VOIs within the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: left ilium and / or right ilium, sacrum, and coccyx); a spinal region initial VOI corresponding to the individual's spine (e.g., wherein a subset of target VOIs within the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: thoracic spine); a left upper body region initial VOI corresponding to the left side of the individual's upper body (e.g., wherein the target VOI subset located in the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: one or more left ribs, a left scapula, and a left clavicle); and a right upper body region initial VOI corresponding to the right side of the individual's upper body (e.g., wherein the target VOI subset located in the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: one or more right ribs, a right scapula, and a right clavicle).

[0031] In certain embodiments, the method includes: (e) receiving, by a processor, a 3D functional image of an individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)] [e.g., wherein the 3D functional image includes a plurality of voxels each representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within one or more of a target tissue region of the individual]; and (f) identifying, by the processor, using a 3D segmentation map, within the 3D functional image one or more 3D volumes each corresponding to the identified target VOI (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image).

[0032] In certain embodiments, the method includes: (g) determining, by a processor, the individual's cancer status [(e.g., prostate cancer status; e.g., metastatic cancer status (e.g., metastatic cancer, including, for example, metastatic prostate cancer, breast cancer, lung cancer, colon cancer, skin cancer, etc.))] (e.g., using stereo pixel intensities and one or more identified 3D volumes within the functional image) (e.g., based on detected lesions) [e.g., the likelihood that the individual has and / or has prostate cancer and / or a particular stage of prostate cancer (e.g., metastatic prostate cancer)] [e.g., the likelihood that the individual has and / or has metastatic cancer (including, for example, metastatic prostate cancer, breast cancer, lung cancer, colon cancer, skin cancer, etc.)].

[0033] In some embodiments, the method includes repeatedly performing steps (a) to (g) for a plurality of anatomical and corresponding functional images acquired at different time points to determine the individual's cancer status at each time point, thereby tracking the cancer status over time (e.g., to evaluate disease progression and / or treatment efficacy).

[0034] In some embodiments, the method includes (e.g., prior to step (g)): (h) automatically adjusting, by a processor, the stereoscopic pixel intensity of the 3D functional image to correct for background absorption (e.g., absorption by a radioactive drug) in one or more background tissue regions (e.g., correcting for absorption of a radioactive drug that normally occurs in one or more background tissue regions and does not necessarily indicate the presence of a cancerous lesion).

[0035] In some embodiments, the target tissue region includes one or more background tissue regions and step (h) includes: using the 3D segmentation map to identify one or more 3D background tissue volumes within the 3D functional image, each corresponding to a specific background tissue region (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image); and adjusting the stereo pixel intensity of the 3D functional image based on the stereo pixel intensity within the 3D background tissue volume (e.g., by using the stereo pixel intensity within the 3D background tissue volume to estimate the contribution to the intensity outside the 3D background tissue volume).

[0036] In certain embodiments, the one or more background tissue regions include one or more members selected from the group consisting of: bladder (eg, urinary bladder), kidney, duodenum, small intestine, spleen, liver, pancreas, stomach, adrenal gland, rectum, and testicle.

[0037] In certain embodiments, the method includes: (i) automatically detecting, by a processor, one or more hot spots within a 3D functional image, wherein the hot spots are determined to represent a lesion based on voxel intensity within the 3D functional image [e.g., based on a comparison of the intensity within the 3D functional image to a threshold (e.g., where the 3D functional image is a 3D PET image and the threshold is a particular standard uptake value (SUV) level)] (e.g., and additionally based on one or more 3D volumes identified within the 3D functional image).

[0038] In some embodiments, step (i) comprises using one or more thresholds [e.g., comparing voxel intensities of the 3D functional image with one or more thresholds (e.g., where the one or more thresholds comprise a plurality of region-specific thresholds, each for a specific subset of one or more 3D volumes identified within the 3D functional image)].

[0039] In some embodiments, step (i) comprises applying one or more filters to the 3D functional image [e.g., as in a blob detection technique; e.g., wherein the one or more filters comprise one or more Gaussian filters (e.g., a high-pass Gaussian filter and a low-pass Gaussian filter, as in a Difference of Gaussians method); e.g., wherein the one or more filters comprise a Laplacian filter (e.g., as in a Laplacian of a Gaussian technique); e.g., wherein different filter kernels are used for different subsets of the one or more 3D volumes identified within the 3D functional image].

[0040] In some embodiments, step (i) includes using a combination of two or more techniques to identify hotspots in different subsets of one or more 3D volumes identified within the 3D functional image [e.g., using a first filtering method (e.g., as in paragraph

[0038] ) for a first subset of one or more 3D volumes and using a second filtering method for a second subset; e.g., using a thresholding method (e.g., as in paragraph

[0037] ) for a first subset of one or more 3D volumes and using a filtering method (e.g., as in paragraph

[0038] ) for a second subset of one or more 3D volumes].

[0041] In certain embodiments, step (i) includes: detecting an initial set of hotspots, and for at least a portion of the hotspots in the initial set, classifying each of at least a portion of the detected hotspots as a cancerous lesion or a non-cancerous lesion (e.g., as noise) [e.g., using a machine learning module; e.g., based on the shape and / or location of the hotspot (e.g., in combination with anatomical knowledge; e.g., wherein the location includes an identification of a specific target tissue region corresponding to the 3D volume in which the hotspot is located and / or the relative location of the hotspot within the specific target tissue region); e.g., and removing hotspots classified as non-cancerous lesions from the initial set, thereby obtaining a final set of hotspots determined to represent lesions].

[0042] In certain embodiments, the target tissue region includes one or more background tissue regions and the method includes: using a 3D segmentation map to identify one or more 3D background tissue volumes within a 3D functional image (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image) that each corresponds to a specific background tissue region (e.g., background tissue regions where significant radiopharmaceutical absorption occurs under normal circumstances and does not necessarily indicate the presence of a cancerous lesion); and excluding 3D voxels within the 3D background tissue from the voxels used to automatically detect one or more hotspots at step (i).

[0043] In certain embodiments, a method comprises using one or more detected hotspots (eg, the intensity of one or more detected hotspots) to determine the cancer status of an individual.

[0044] In certain embodiments, the target tissue region includes one or more reference tissue regions and the method includes: using the 3D segmentation map to identify, by a processor, within the 3D functional image, one or more 3D reference volumes, each corresponding to a specific reference tissue region; determining, by the processor, one or more reference intensity values, each associated with a specific 3D reference volume of the one or more 3D reference volumes and corresponding to an intensity measurement value (e.g., an average / mean, a median, a maximum, etc.) within the specific 3D reference volume; determining, by the processor, one or more individual hotspot intensity values, each associated with a specific hotspot of at least a portion of the detected one or more hotspots and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) of the specific hotspot; and determining, by the processor, one or more individual hotspot index values ​​using the one or more individual hotspot intensity values ​​and the one or more reference intensity values ​​[e.g., wherein each individual hotspot index value is associated with a specific hotspot of at least a portion of the detected one or more hotspots and is determined using (i) the individual hotspot intensity value associated with the specific hotspot and (ii) the one or more reference intensity values ​​(e.g., based on a comparison therebetween)].

[0045] In some embodiments, determining one or more individual hotspot index values ​​includes mapping each of the one or more reference intensity values ​​to a corresponding reference index value in a certain proportion, and for each individual hotspot intensity value, using the reference intensity value and the corresponding reference index value to interpolate the corresponding individual hotspot index value.

[0046] In certain embodiments, the reference tissue region includes one or more members selected from the group consisting of: liver, aorta (eg, a thoracic aorta portion; eg, an abdominal aorta portion), and parotid gland.

[0047] In some embodiments, the first reference intensity value (i) is a blood reference intensity value associated with a 3D reference volume corresponding to a portion of the aorta and (ii) is mapped to a first reference index value; the second reference intensity value (i) is a liver reference intensity value associated with a 3D reference volume corresponding to the liver and (ii) is mapped to a second reference index value; and the second reference intensity value is greater than the first reference intensity value and the second reference index value is greater than the first reference index value.

[0048] In certain embodiments, the reference intensity values ​​include a maximum reference intensity value that maps to a maximum reference index value, and hotspots having associated hotspot intensity values ​​greater than the maximum reference intensity value are assigned hotspot index values ​​equal to the maximum reference index value.

[0049] In certain embodiments, the method includes determining, by a processor, an overall index value indicative of a cancer status of the individual using at least a portion of the one or more hotspot index values.

[0050] In some embodiments, the overall index value is determined as a weighted sum of at least a portion of the individual hotspot index values ​​[e.g., located within a 3D volume of a functional image corresponding to a skeletal region of the individual; e.g., located within a 3D volume of a functional image corresponding to a lymphatic region of the individual] [e.g., where the sum of the individual hotspot index values ​​is weighted by a size measurement (e.g., 3D volume; e.g., mean diameter) of the associated hotspots].

[0051] In certain embodiments, the overall index value is associated with a specific target tissue region identified within the anatomical image corresponding to the specific target VOI; and the overall index value is determined using the hotspot index values ​​of a subset of hotspots located within a specific 3D volume in the 3D functional image corresponding to the specific identified target VOI [e.g., and wherein the overall index value is calculated as a weighted sum of the hotspot index values ​​of the subset of hotspots (e.g., wherein the weighted sum is normalized by an estimated volume of the specific target tissue region (e.g., calculated as the volume of the specific 3D volume in the functional image and / or the volume of the specific target VOI))].

[0052] In certain embodiments, the specific target tissue region is selected from the group consisting of a skeletal region including one or more bones of the individual, a lymphatic region, and a prostate region.

[0053] In certain embodiments, the 3D anatomical image is an x-ray computed tomography (CT) image and the 3D functional image is a 3D single photon emission computed tomography (SPECT) image.

[0054] In certain embodiments, the 3D anatomical images are x-ray computed tomography (CT) images and the 3D functional images are 3D positron emission tomography (PET) images.

[0055] In certain embodiments, a 3D PET image of a subject is obtained after administering a radiopharmaceutical comprising a prostate-specific membrane antigen (PSMA) binding agent to the subject.

[0056] In certain embodiments, the radiopharmaceutical comprises [18F]DCFPyL.

[0057] In some embodiments, step (b) includes cropping the 3D anatomical image, by the processor, to remove voxels representing air [e.g., creating a cropped anatomical image, and using the cropped anatomical image to identify one or more target VOIs (e.g., using the cropped anatomical image as input for one or more machine learning modules, as opposed to the original sized 3D anatomical image)].

[0058] In some embodiments, the target tissue region includes one or more members selected from the group consisting of: left hip, right hip, sacrum and coccyx region, left clavicle, right clavicle, left rib, right rib, left scapula, right scapula, sternum, lumbar spine, thoracic spine, skull, cervical spine, left femur, right femur, left humerus, right humerus, prostate, bladder, rectum, left gluteus maximus, right gluteus maximus, aorta (e.g., thoracic aorta portion; e.g., abdominal aorta portion), left kidney, right kidney, liver, left lung, right lung, spleen, ventricle, left adrenal gland, right adrenal gland, gall bladder, brain, pancreas, heart, mandible, left bronchus, right bronchus, trachea, left common iliac artery, right common iliac artery and parotid gland.

[0059] In some embodiments, one or more of the target tissue areas include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left femur, right femur, left fibula, right fibula, left hip, right hip, left humerus, right humerus, mandible, left patella, right patella, left radius, right radius, left tibia, right tibia, left ulna, right ulna, left ribs (e.g., the first left rib, the second left rib, the third left rib, the fourth left rib, the fifth left rib, the sixth left rib, the seventh left rib, the eighth left rib, the ninth left rib, the tenth left rib, the eleventh left rib, the twelfth left rib), right ribs (e.g., the first right rib, the second right rib, the third right rib, the fourth right rib, the fifth right rib, the sixth right rib, the seventh right rib, the eighth right rib, the ninth right rib, the tenth right rib, the eleventh right rib, the twelfth right rib), sacrum and coccyx (e.g., the combined sacrum and coccyx areas; e.g., a single the sacrum and coccyx alone to distinguish between the two), left scapula, right scapula, skull, sternum, spinal region [e.g., cervical region, including one or more (e.g., up to all) cervical vertebrae; e.g., lumbar region, including one or more (e.g., up to all) lumbar vertebrae; e.g., thoracic region, including one or more (e.g., up to all) thoracic vertebrae], and individual vertebrae [e.g., individual cervical vertebrae (e.g., the first cervical vertebra, the second cervical vertebra, the third cervical vertebra, the fourth cervical vertebra, the fifth cervical vertebra, the sixth cervical vertebra, the seventh cervical vertebra); e.g., individual lumbar vertebrae (e.g., the first lumbar vertebra, the second lumbar vertebra, the third lumbar vertebra, the fourth lumbar vertebra, the fifth lumbar vertebra, the sixth lumbar vertebra); e.g., individual thoracic vertebrae (e.g., the first thoracic vertebra, the second thoracic vertebra, the third thoracic vertebra, the fourth thoracic vertebra, the fifth thoracic vertebra, the sixth thoracic vertebra, the seventh thoracic vertebra, the eighth thoracic vertebra, the ninth thoracic vertebra, the tenth thoracic vertebra, the eleventh thoracic vertebra, the twelfth thoracic vertebra)].

[0060] In certain embodiments, one or more of the target tissue regions include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left hip, right hip, left ribs (e.g., the first left rib, the second left rib, the third left rib, the fourth left rib, the fifth left rib, the sixth left rib, the seventh left rib, the eighth left rib, the ninth left rib, the tenth left rib, the eleventh left rib, the twelfth left rib), right ribs (e.g., the first right rib, the second right rib, the third right rib, the fourth right rib, the fifth right rib, the sixth right rib, the seventh right rib, the eighth right rib, the ninth right rib, the tenth right rib, the eleventh right rib, the twelfth right rib), sacrum, and coccyx (e.g., the sacrum and coccyx regions combined; e.g., the sacrum and coccyx regions alone). the scapula and coccyx to distinguish between the two), left scapula, right scapula, sternum, spinal region [e.g., cervical region, including one or more (e.g., up to all) cervical vertebrae; e.g., lumbar region, including one or more (e.g., up to all) lumbar vertebrae; e.g., thoracic region, including one or more (e.g., up to all) thoracic vertebrae], and individual vertebrae [e.g., individual lumbar vertebrae (e.g., the first lumbar vertebra, the second lumbar vertebra, the third lumbar vertebra, the fourth lumbar vertebra, the fifth lumbar vertebra); e.g., individual thoracic vertebrae (e.g., the first thoracic vertebra, the second thoracic vertebra, the third thoracic vertebra, the fourth thoracic vertebra, the fifth thoracic vertebra, the sixth thoracic vertebra, the seventh thoracic vertebra, the eighth thoracic vertebra, the ninth thoracic vertebra, the tenth thoracic vertebra, the eleventh thoracic vertebra, the twelfth thoracic vertebra)].

[0061] In some embodiments, one or more of the target tissue regions include one or more soft tissue regions (e.g., organs) selected from the group consisting of: left adrenal gland, right adrenal gland, aorta (e.g., thoracic aorta portion; e.g., abdominal aorta portion), brain, left bronchus, right bronchus, gallbladder, left gluteus maximus, right gluteus maximus, heart, left common crotch artery, right common crotch artery, left kidney, right kidney, liver, left lung, right lung, pancreas, prostate, rectum, spleen, trachea, bladder, ventricle, and parotid gland.

[0062] In certain embodiments, the one or more target tissue regions include one or more soft tissue regions (eg, organs) selected from the group consisting of: gallbladder, left kidney, right kidney, liver, left lung, right lung, prostate, and bladder.

[0063] In some embodiments, the target tissue region includes one or more bone regions, each of which corresponds to a specific bone [e.g., where the target tissue region includes at least a portion (e.g., at most all) of the bones listed in paragraphs

[0058] or

[0059] ]; and one or more soft tissue regions, each of which corresponds to a specific soft tissue region [e.g., where the target tissue region includes at least a portion (e.g., at most all) of the soft tissue regions listed in paragraphs

[0060] or

[0061] ].

[0064] In another aspect, the present invention relates to a method for automatically processing 3D images to automatically identify cancerous lesions in an individual, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of an individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the individual; (b) for each of a plurality of target tissue regions, automatically identifying, by the processor, a corresponding target volume of interest (VOI) within the 3D anatomical image using one or more machine learning modules (e.g., wherein at least one of the one or more machine learning modules is a convolutional neural network (CNN) module); (c) determining, by the processor, a 3D segmentation map representing a plurality of 3D segmentation masks, each 3D segmentation mask representing a particular identified target VOI (e.g., automatically digitally stitching the plurality of 3D segmentation masks together to form the 3D segmentation map); (d) receiving, by the processor, a 3D segmentation map representing a plurality of 3D segmentation masks, each 3D segmentation mask representing a particular identified target VOI (e.g., automatically digitally stitching the plurality of 3D segmentation masks together to form the 3D segmentation map); (e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET))] of a 3D functional image of the individual [e.g., wherein the 3D functional image includes a plurality of voxels each representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within one or more of the target tissue regions of the individual]; (e) identifying, using the 3D segmentation map, within the 3D functional image one or more 3D volumes each corresponding to an identified target VOI (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image); and (f) automatically detecting, by the processor, within at least a portion of the one or more 3D volumes identified within the 3D functional image, one or more hot spots based on voxel intensity within the 3D functional image [e.g., based on a comparison of the intensity within the 3D functional image to a threshold value (e.g., wherein the 3D functional image is a 3D functional image] PET images and the threshold is a specific standard uptake value (SUV) level)] determined to represent lesions.

[0065] In certain embodiments, the method includes using, by a processor, one or more detected hotspots (eg, the intensity of the one or more detected hotspots) to determine a cancer status of the individual.

[0066] In certain embodiments, the target tissue region includes one or more reference tissue regions and wherein the method comprises: using the 3D segmentation map to identify, by a processor, within the 3D functional image, one or more 3D reference volumes, each corresponding to a particular reference tissue region; determining one or more reference intensity values, each associated with a particular 3D reference volume of the one or more 3D reference volumes and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) within the particular 3D reference volume; determining, by the processor, one or more individual hotspot intensity values, each associated with a particular hotspot of at least a portion of the detected one or more hotspots and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) of the particular hotspot; and determining, by the processor, one or more individual hotspot index values ​​using the one or more individual hotspot intensity values ​​and the one or more reference intensity values ​​[e.g., wherein each individual hotspot index value is associated with a particular hotspot of at least a portion of the detected one or more hotspots and is determined using (i) the individual hotspot intensity value associated with the particular hotspot and (ii) the one or more reference intensity values ​​(e.g., based on a comparison therebetween)].

[0067] In certain embodiments, the reference tissue region includes one or more members selected from the group consisting of: liver, aorta (eg, a thoracic aorta portion; eg, an abdominal aorta portion), and parotid gland.

[0068] In certain embodiments, the method includes determining, by a processor, an overall index value indicative of a cancer status of the individual using at least a portion of the one or more hotspot index values.

[0069] In some embodiments, the overall index value is determined as a weighted sum of at least a portion of the individual hotspot index values ​​[e.g., located within a 3D volume of a functional image corresponding to a skeletal region of the individual; e.g., located within a 3D volume of a functional image corresponding to a lymphatic region of the individual] [e.g., where the sum of the individual hotspot index values ​​is weighted by a size measurement (e.g., 3D volume; e.g., mean diameter) of the associated hotspots].

[0070] In certain embodiments, the 3D anatomical images are x-ray computed tomography (CT) images and the 3D functional images are 3D positron emission tomography (PET) images.

[0071] In certain embodiments, a 3D PET image of a subject is obtained after administering a radiopharmaceutical comprising a prostate-specific membrane antigen (PSMA) binding agent to the subject.

[0072] In certain embodiments, the radiopharmaceutical comprises [18F]DCFPyL.

[0073] In another aspect, the present invention relates to a method for automatically processing 3D images to identify and measure the uptake of a radiopharmaceutical in a cancerous lesion (e.g., a cancer metastasis) in an individual having or at risk of cancer (e.g., prostate cancer, breast cancer, lung cancer; e.g., metastatic cancer, e.g., metastatic prostate cancer, metastatic breast cancer, metastatic lung cancer), the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the individual; and (b) processing, by the processor of the computing device, a 3D anatomical image of the individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound]; a processor that automatically identifies within the 3D anatomical image, using one or more machine learning modules (e.g., wherein at least one of the machine learning modules is a convolutional neural network (CNN) module): a first bone volume comprising a graphical representation of one or more bones of the individual; a first aorta volume comprising a graphical representation of at least a portion of the individual's aorta; and a first liver volume comprising a graphical representation of the individual's liver; and (c) determining, by the processor, a 3D segmentation map representing a plurality of 3D segmentation masks, the 3D segmentation masks including a bone mask representing the identified first bone volume, an aorta mask representing the identified first aorta volume, and a liver mask representing the identified first liver volume. (d) receiving, by the processor, a 3D functional image of the individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)] [e.g., wherein the 3D functional image comprises a plurality of voxels, each representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within one or more bones, aorta portions, and / or liver of the individual]; (e) automatically identifying within the 3D functional image using the 3D segmentation map (e.g., by segmenting the 3D functional image into segments representing a specific physical volume within the individual); the 3D segmentation mask of the 3D segmentation map being mapped to the 3D functional image): a second bone volume corresponding to the first identified bone volume within the 3D anatomical image; a second aortic volume corresponding to the first aortic volume identified within the 3D anatomical image; and a second liver volume corresponding to the first liver volume identified within the 3D anatomical image; (f) automatically detecting, by the processor, one or more hot spots within the second bone volume, the hot spots determined to represent a lesion based on voxel intensity within the second bone volume (e.g., the one or more hot spots correspond to localized areas of relatively high intensity, e.g., identified based on a comparison of voxel intensity located within the second bone volume to a threshold);and (g) for each of the one or more detected hotspots, determining, by the processor, an individual hotspot index value (e.g., a measurement indicating radiopharmaceutical absorption in the lesion represented by the detected hotspot) by: determining an aorta reference intensity level based on intensity measurements (e.g., average, maximum, median, etc.) of voxels within the second aorta volume; determining a liver reference intensity level based on intensity measurements (e.g., average, maximum, median, etc.) of voxels within the second liver volume; and for each individual detected hotspot: determining a corresponding individual hotspot intensity level based on intensity measurements (e.g., average, maximum, median, etc.) of the voxels of the detected hotspot; and determining a corresponding individual hotspot index level based on the individual hotspot intensity level, the aorta reference intensity level, and the liver reference intensity level.

[0074] In certain embodiments, the method includes determining, by a processor, an overall index value indicative of a cancer status of the individual based on individual hotspot index values ​​for at least a portion of the one or more detected hotspots.

[0075] In certain embodiments, the subject has or is at risk for prostate cancer.

[0076] In certain embodiments, step (b) includes automatically identifying a first prostate volume within the 3D anatomical image that includes a graphical representation of the individual's prostate, the 3D segmented map determined at step (c) further includes a prostate mask representing the identified first prostate volume, step (e) includes automatically identifying a second prostate volume within the 3D functional image that corresponds to the first identified prostate volume within the 3D anatomical image, step (f) includes automatically detecting one or more hot spots in the second prostate volume, and the method further includes: by the processor, (i) determining an overall bone index value indicative of lesion content (e.g., and severity) in one or more bones of the individual based on individual hot spot index values ​​for at least a portion of the one or more detected hot spots located in the second bone volume; and (ii) determining an overall prostate index value indicative of lesion content (e.g., and severity) in the prostate of the individual based on individual hot spot index values ​​for at least a portion of the one or more detected hot spots located in the second prostate volume.

[0077] In certain embodiments, the subject has or is at risk for metastatic cancer (eg, metastatic prostate cancer, metastatic breast cancer, metastatic lung cancer, and other metastatic bone cancers).

[0078] In another aspect, the present invention relates to a method for automatically processing a set of whole-body 3D anatomical images of different sizes to automatically identify multiple 3D volumes corresponding to specific target tissue regions within each 3D anatomical image, the method comprising: (a) receiving, by a processor of a computing device, a set of 3D anatomical images of one or more individuals acquired using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical images include a graphical representation of tissue (e.g., soft tissue and / or bone) within each of the one or more individuals, wherein the set of 3D anatomical images has a mean x-dimension, a mean y-dimension, and a mean z-dimension, at least one of which (the mean x-dimension, the mean y-dimension, or the mean z-dimension) has a standard deviation of at least 3% of the corresponding mean [e.g., the mean x-dimension, the mean y-dimension, or the mean z-dimension] the 3D anatomical image set; and (b) automatically determining, by the processor, at least one initial VOI corresponding to a specific anatomical region (e.g., a group of related tissues, such as the pelvic region and the spinal column region) including one or more specific related target tissue regions (e.g., one or more specific bones, such as one or more specific organs) within each image of the 3D anatomical image set, thereby identifying at least one initial VOI of the corresponding 3D anatomical image; and (c) for each initial VOI, automatically identifying, by the processor, one or more segmentation modules, each of which implements a CNN for each of the one or more specific target tissue regions related to the specific anatomical region corresponding to the initial VOI, a corresponding target VOI (e.g., a graphical representation of the specific target tissue region).

[0079] In another aspect, the present invention relates to a system for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a specific target tissue region, the system comprising: a processor of a computing device; and a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to: (a) receive a 3D anatomical image of an individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical image includes a graphical representation of tissue (e.g., soft tissue and / or bone) within the individual; b) for each of a plurality of target tissue regions, automatically identifying a corresponding target volume of interest (VOI) within the 3D anatomical image using one or more machine learning modules (e.g., wherein at least one of the one or more machine learning modules is a convolutional neural network (CNN) module); (c) determining a 3D segmentation map representing a plurality of 3D segmentation masks, each 3D segmentation mask representing a particular identified target VOI (e.g., automatically digitally stitching the plurality of 3D segmentation masks together to form the 3D segmentation map); and (d) storing and / or providing the 3D segmentation map for display and / or further processing.

[0080] In certain embodiments, the 3D anatomical image is a full body image.

[0081] In certain embodiments, at step (c), the instructions cause the processor to digitally stitch together the multiple 3D segmentation masks to form a 3D segmentation map {e.g., by creating an initially empty image volume (e.g., initializing all voxel values ​​to zero) and then inserting labels from each segmentation mask into the image volume [e.g., by mapping labeled (e.g., as representing a specific target tissue region as determined by the machine learning module) voxels of the input image to one or more machine learning modules to voxels of the image volume (e.g., so as to match voxels of the image volume with voxels of the input image representing the same physical location, thereby correctly labeling the voxels of the image volume)]}.

[0082] In certain embodiments, at step (b), for at least one specific target tissue region, the instructions cause the processor to: determine, using a first module (e.g., a localization module) (e.g., a first machine learning module) an initial VOI within the 3D anatomical image, the initial VOI corresponding to an anatomical region (e.g., a group of related tissues, such as the pelvic region, the thoracic region, the head and / or neck region, and the like) that contains the specific target tissue region (e.g., wherein the initial VOI excludes more voxels than it contains in the 3D anatomical image; e.g., wherein the initial VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the initial VOI represent a physical volume within the anatomical region); and identify, using a second module (e.g., a segmentation module) (e.g., a second machine learning module), a target VOI within the initial VOI that corresponds to the specific target tissue region.

[0083] In some embodiments, the second module is a CNN module that implements CNN.

[0084] In certain embodiments, the first module is a CNN module that implements a CNN to perform coarse segmentation to automatically identify an initial VOI corresponding to an anatomical region containing a specific target tissue region [e.g., by automatically identifying a graphical representation of a set of related tissues within the anatomical image (e.g., and then determining the initial VOI as a rectangular region (e.g., a rectangular column or rectangular box) that completely surrounds the identified graphical representation of the set of related tissues)] [e.g., for each of one or more specific tissue regions expected to be within the anatomical region, by automatically identifying a corresponding VOI in the anatomical image (e.g., via coarse segmentation), for example, and determining the initial VOI as a rectangular region that completely surrounds all identified VOIs corresponding to the specific tissue regions].

[0085] In some embodiments, a first module receives as input a subsampled [e.g., by a factor of two or more than two (e.g., four) along one or more dimensions] version of an anatomical image and uses the subsampled version of the anatomical image to identify an initial VOI [e.g., and wherein a second module receives a full-resolution version of the anatomical image cropped to the initial VOI (e.g., such that the first module operates on a lower-resolution image representing a larger physical volume than the second module, while the second module operates on an image of higher resolution but representing a smaller physical volume)].

[0086] In some embodiments, the first module is a first CNN module and the second module is a second CNN module, and wherein the first CNN module includes additional filters to account for a rate of change as the image size increases relative to the second CNN module.

[0087] In certain embodiments, the 3D anatomical image is a whole-body image and wherein at step (b), the instructions cause the processor to: automatically determine, using one or more localization modules implementing machine learning techniques (e.g., wherein each localization module is a CNN module implementing a CNN), a plurality of initial VOIs within the 3D anatomical image, each initial VOI corresponding to a particular anatomical region (e.g., a group of related tissues, such as a pelvic region, a thoracic region, a head and / or neck region, a spinal region, an upper body region, a lower body region, etc.) and wherein a relevant subset of the target VOIs is located; and for each initial VOI, automatically identify, using one or more segmentation modules implementing machine learning techniques (e.g., wherein each segmentation module is a CNN module implementing a CNN), a relevant subset of the target VOIs.

[0088] In some embodiments, the plurality of initial VOIs include one or more members selected from the group consisting of: a pelvic region initial VOI corresponding to the individual's pelvic region (e.g., wherein a subset of target VOIs within the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: left ilium and / or right ilium, sacrum, and coccyx); a spinal region initial VOI corresponding to the individual's spine (e.g., wherein a subset of target VOIs within the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: thoracic spine); a left upper body region initial VOI corresponding to the left side of the individual's upper body (e.g., wherein the target VOI subset located in the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: one or more left ribs, a left scapula, and a left clavicle); and a right upper body region initial VOI corresponding to the right side of the individual's upper body (e.g., wherein the target VOI subset located in the pelvic region initial VOI includes one or more target VOIs corresponding to target tissue regions selected from the group consisting of: one or more right ribs, a right scapula, and a right clavicle).

[0089] In certain embodiments, the instructions cause the processor to: (e) receive a 3D functional image of an individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)] [e.g., wherein the 3D functional image includes a plurality of voxels each representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within one or more of a target tissue region of the individual]; and (f) identify, within the 3D functional image, one or more 3D volumes each corresponding to an identified target VOI using a 3D segmentation map (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image).

[0090] In certain embodiments, the instructions cause the processor to: (g) determine the individual's cancer status [(e.g., prostate cancer status; e.g., metastatic cancer status (e.g., metastatic cancer, including, for example, metastatic prostate cancer, breast cancer, lung cancer, colon cancer, skin cancer, etc.))] (e.g., using stereo pixel intensities and one or more identified 3D volumes within the functional image) (e.g., based on detected lesions) [e.g., the likelihood that the individual has and / or has prostate cancer and / or a particular stage of prostate cancer (e.g., metastatic prostate cancer)] [e.g., the likelihood that the individual has and / or has metastatic cancer (including, for example, metastatic prostate cancer, breast cancer, lung cancer, colon cancer, skin cancer, etc.)].

[0091] In some embodiments, the instructions cause the processor to repeatedly perform steps (a) through (g) on ​​a plurality of anatomical and corresponding functional images acquired at different time points to determine the individual's cancer status at each time point, thereby tracking the cancer status over time (e.g., to evaluate disease progression and / or treatment efficacy).

[0092] In some embodiments, the instructions cause the processor (e.g., prior to step (g)): (h) automatically adjust the stereo pixel intensity of the 3D functional image to correct for background absorption (e.g., absorption by a radiopharmaceutical) in one or more background tissue regions (e.g., to correct for absorption by a radiopharmaceutical that normally occurs in one or more background tissue regions and is not necessarily indicative of the presence of a cancerous lesion).

[0093] In certain embodiments, the target tissue region includes one or more background tissue regions and wherein at step (h), the instructions cause the processor to: use the 3D segmentation map to identify one or more 3D background tissue volumes within the 3D functional image, each corresponding to a specific background tissue region (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image); and adjust the stereo pixel intensity of the 3D functional image based on the stereo pixel intensity within the 3D background tissue volume (e.g., by using the stereo pixel intensity within the 3D background tissue volume to estimate the contribution to the intensity outside the 3D background tissue volume).

[0094] In certain embodiments, the one or more background tissue regions include one or more members selected from the group consisting of: bladder (eg, bladder), kidney, duodenum, small intestine, spleen, liver, pancreas, stomach, adrenal gland, rectum, and testicle.

[0095] In certain embodiments, the instructions cause the processor to: (i) automatically detect one or more hot spots within a 3D functional image, wherein the hot spots are determined to represent a lesion based on the volumetric pixel intensity within the 3D functional image [e.g., based on a comparison of the intensity within the 3D functional image to a threshold (e.g., where the 3D functional image is a 3D PET image and the threshold is a particular standard uptake value (SUV) level)] (e.g., and also based on one or more 3D volumes identified within the 3D functional image).

[0096] In some embodiments, at step (i), the instructions cause the processor to use one or more thresholds [e.g., comparing the voxel intensities of the 3D functional image with the one or more thresholds (e.g., where the one or more thresholds include a plurality of region-specific thresholds, each for a specific subset of one or more 3D volumes identified within the 3D functional image)].

[0097] In some embodiments, at step (i), the instructions cause the processor to apply one or more filters to the 3D functional image [e.g., as in a speckle detection technique; e.g., wherein the one or more filters include one or more Gaussian filters (e.g., a high-pass Gaussian filter and a low-pass Gaussian filter, as in a Difference of Gaussian method); e.g., wherein the one or more filters include a Laplacian filter (e.g., as in a Laplacian of Gaussian technique); e.g., wherein different filter kernels are used for different subsets of the one or more 3D volumes identified within the 3D functional image].

[0098] In some embodiments, at step (i), the instructions cause the processor to use a combination of two or more techniques to identify hotspots in different subsets of one or more 3D volumes identified within the 3D functional image [e.g., using a first filtering method (e.g., as in paragraph

[0096] ) for a first subset of one or more 3D volumes and using a second filtering method for a second subset; e.g., using a thresholding method (e.g., as in paragraph

[0095] ) for a first subset of one or more 3D volumes and using a filtering method (e.g., as in paragraph

[0096] ) for a second subset of one or more 3D volumes].

[0099] In certain embodiments, at step (i), the instructions cause the processor to detect an initial set of hotspots, and for at least a portion of the hotspots of the initial set, classify each of at least a portion of the detected hotspots as a cancerous lesion or a non-cancerous lesion (e.g., as noise) [e.g., using a machine learning module; e.g., based on the shape and / or location of the hotspot (e.g., in combination with anatomical knowledge; e.g., where the location includes an identification of a specific target tissue region corresponding to the 3D volume in which the hotspot is located and / or a relative location of the hotspot within the specific target tissue region); e.g., and removing hotspots classified as non-cancerous lesions from the initial set, thereby obtaining a final set of hotspots determined to represent lesions].

[0100] In certain embodiments, the target tissue region includes one or more background tissue regions and the instructions cause the processor to: use the 3D segmentation map to identify one or more 3D background tissue volumes within the 3D functional image that each corresponds to a specific background tissue region (e.g., background tissue regions where significant radiopharmaceutical absorption occurs under normal circumstances and does not necessarily indicate the presence of cancerous lesions) (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image); and exclude 3D voxels within the 3D background tissue from the voxels used to automatically detect the one or more hotspots at step (i).

[0101] In certain embodiments, the instructions cause the processor to use the one or more detected hotspots (eg, the intensity of the one or more detected hotspots) to determine the cancer status of the individual.

[0102] In certain embodiments, the target tissue region includes one or more reference tissue regions and the instructions cause the processor to: use the 3D segmentation map to identify one or more 3D reference volumes within the 3D functional image, each corresponding to a particular reference tissue region; determine one or more reference intensity values, each associated with a particular 3D reference volume of the one or more 3D reference volumes and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) within the particular 3D reference volume; determine one or more individual hotspot intensity values, each associated with a particular hotspot of at least a portion of the detected one or more hotspots and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) of the particular hotspot; and determine one or more individual hotspot index values ​​using the one or more individual hotspot intensity values ​​and the one or more reference intensity values ​​[e.g., wherein each individual hotspot index value is associated with a particular hotspot of at least a portion of the detected one or more hotspots and is determined using (i) the individual hotspot intensity value associated with the particular hotspot and (ii) the one or more reference intensity values ​​(e.g., based on a comparison therebetween)].

[0103] In some embodiments, the instructions cause the processor to determine one or more individual hotspot index values ​​by mapping each of the one or more reference intensity values ​​to a corresponding reference index value at a certain ratio, and for each individual hotspot intensity value, using the reference intensity value and the corresponding reference index value to interpolate the corresponding individual hotspot index value.

[0104] In certain embodiments, the reference tissue region includes one or more members selected from the group consisting of: liver, aorta (eg, a thoracic aorta portion; eg, an abdominal aorta portion), and parotid gland.

[0105] In some embodiments, the first reference intensity value (i) is a blood reference intensity value associated with a 3D reference volume corresponding to a portion of the aorta and (ii) is mapped to a first reference index value; the second reference intensity value (i) is a liver reference intensity value associated with a 3D reference volume corresponding to the liver and (ii) is mapped to a second reference index value; and the second reference intensity value is greater than the first reference intensity value and the second reference index value is greater than the first reference index value.

[0106] In certain embodiments, the reference intensity values ​​include a maximum reference intensity value mapped to a maximum reference index value, and wherein hotspots having associated hotspot intensity values ​​greater than the maximum reference intensity value are assigned a hotspot index value equal to the maximum reference index value.

[0107] In certain embodiments, the instructions cause the processor to use at least a portion of the one or more hotspot index values ​​to determine an overall index value indicative of a cancer status of the individual.

[0108] In some embodiments, the overall index value is determined as a weighted sum of at least a portion of the individual hotspot index values ​​[e.g., located within a 3D volume of a functional image corresponding to a skeletal region of the individual; e.g., located within a 3D volume of a functional image corresponding to a lymphatic region of the individual] [e.g., where the sum of the individual hotspot index values ​​is weighted by a size measurement (e.g., 3D volume; e.g., mean diameter) of the associated hotspots].

[0109] In certain embodiments, the overall index value is associated with a specific target tissue region identified within the anatomical image corresponding to the specific target VOI; and the overall index value is determined using the hotspot index values ​​of a subset of hotspots located within a specific 3D volume in the 3D functional image corresponding to the specific identified target VOI [e.g., and wherein the overall index value is calculated as a weighted sum of the hotspot index values ​​of the subset of hotspots (e.g., wherein the weighted sum is normalized by an estimated volume of the specific target tissue region (e.g., calculated as the volume of the specific 3D volume in the functional image and / or the volume of the specific target VOI))].

[0110] In certain embodiments, the specific target tissue region is selected from the group consisting of a skeletal region including one or more bones of the individual, a lymphatic region, and a prostate region.

[0111] In certain embodiments, the 3D anatomical image is an x-ray computed tomography (CT) image and the 3D functional image is a 3D single photon emission computed tomography (SPECT) image.

[0112] In certain embodiments, the 3D anatomical images are x-ray computed tomography (CT) images and the 3D functional images are 3D positron emission tomography (PET) images.

[0113] In certain embodiments, a 3D PET image of a subject is obtained after administering a radiopharmaceutical comprising a prostate-specific membrane antigen (PSMA) binding agent to the subject.

[0114] In certain embodiments, the radiopharmaceutical comprises [18F]DCFPyL.

[0115] In certain embodiments, at step (b), the instructions cause the processor to crop the 3D anatomical image to remove voxels representing air [e.g., creating a cropped anatomical image, and using the cropped anatomical image to identify one or more target VOIs (e.g., using the cropped anatomical image as input for one or more machine learning modules, as opposed to the original sized 3D anatomical image)].

[0116] In some embodiments, the target tissue area includes one or more members selected from the group consisting of: left hip, right hip, sacrum and coccyx area, left clavicle, right clavicle, left rib, right rib, left scapula, right scapula, sternum, lumbar spine, thoracic spine, skull, cervical spine, left femur, right femur, left humerus, right humerus, prostate, bladder, rectum, left gluteus maximus, right gluteus maximus, aorta (e.g., thoracic aorta portion; e.g., abdominal aorta portion), left kidney, right kidney, liver, left lung, right lung, spleen, ventricle, left adrenal gland, right adrenal gland, gall bladder, brain, pancreas, heart, mandible, left bronchus, right bronchus, trachea, left common iliac artery, right common iliac artery and parotid gland.

[0117] In some embodiments, one or more of the target tissue areas include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left femur, right femur, left fibula, right fibula, left hip, right hip, left humerus, right humerus, mandible, left patella, right patella, left radius, right radius, left tibia, right tibia, left ulna, right ulna, left ribs (e.g., the first left rib, the second left rib, the third left rib, the fourth left rib, the fifth left rib, the sixth left rib, the seventh left rib, the eighth left rib, the ninth left rib, the tenth left rib, the eleventh left rib, the twelfth left rib), right ribs (e.g., the first right rib, the second right rib, the third right rib, the fourth right rib, the fifth right rib, the sixth right rib, the seventh right rib, the eighth right rib, the ninth right rib, the tenth right rib, the eleventh right rib, the twelfth right rib), sacrum and coccyx (e.g., the combined sacrum and coccyx areas; e.g., a single the sacrum and coccyx alone to distinguish between the two), left scapula, right scapula, skull, sternum, spinal region [e.g., cervical region, including one or more (e.g., up to all) cervical vertebrae; e.g., lumbar region, including one or more (e.g., up to all) lumbar vertebrae; e.g., thoracic region, including one or more (e.g., up to all) thoracic vertebrae], and individual vertebrae [e.g., individual cervical vertebrae (e.g., the first cervical vertebra, the second cervical vertebra, the third cervical vertebra, the fourth cervical vertebra, the fifth cervical vertebra, the sixth cervical vertebra, the seventh cervical vertebra); e.g., individual lumbar vertebrae (e.g., the first lumbar vertebra, the second lumbar vertebra, the third lumbar vertebra, the fourth lumbar vertebra, the fifth lumbar vertebra, the sixth lumbar vertebra); e.g., individual thoracic vertebrae (e.g., the first thoracic vertebra, the second thoracic vertebra, the third thoracic vertebra, the fourth thoracic vertebra, the fifth thoracic vertebra, the sixth thoracic vertebra, the seventh thoracic vertebra, the eighth thoracic vertebra, the ninth thoracic vertebra, the tenth thoracic vertebra, the eleventh thoracic vertebra, the twelfth thoracic vertebra)].

[0118] In certain embodiments, one or more of the target tissue regions include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left hip, right hip, left ribs (e.g., the first left rib, the second left rib, the third left rib, the fourth left rib, the fifth left rib, the sixth left rib, the seventh left rib, the eighth left rib, the ninth left rib, the tenth left rib, the eleventh left rib, the twelfth left rib), right ribs (e.g., the first right rib, the second right rib, the third right rib, the fourth right rib, the fifth right rib, the sixth right rib, the seventh right rib, the eighth right rib, the ninth right rib, the tenth right rib, the eleventh right rib, the twelfth right rib), sacrum, and coccyx (e.g., the sacrum and coccyx regions combined; e.g., the sacrum and coccyx regions alone). the scapula and coccyx to distinguish between the two), left scapula, right scapula, sternum, spinal region [e.g., cervical region, including one or more (e.g., up to all) cervical vertebrae; e.g., lumbar region, including one or more (e.g., up to all) lumbar vertebrae; e.g., thoracic region, including one or more (e.g., up to all) thoracic vertebrae], and individual vertebrae [e.g., individual lumbar vertebrae (e.g., the first lumbar vertebra, the second lumbar vertebra, the third lumbar vertebra, the fourth lumbar vertebra, the fifth lumbar vertebra); e.g., individual thoracic vertebrae (e.g., the first thoracic vertebra, the second thoracic vertebra, the third thoracic vertebra, the fourth thoracic vertebra, the fifth thoracic vertebra, the sixth thoracic vertebra, the seventh thoracic vertebra, the eighth thoracic vertebra, the ninth thoracic vertebra, the tenth thoracic vertebra, the eleventh thoracic vertebra, the twelfth thoracic vertebra)].

[0119] In certain embodiments, one or more of the target tissue regions include one or more soft tissue regions (e.g., organs) selected from the group consisting of: left adrenal gland, right adrenal gland, aorta (e.g., thoracic aorta portion; e.g., abdominal aorta portion), brain, left bronchus, right bronchus, gallbladder, left gluteus maximus, right gluteus maximus, heart, left common iliac artery, right common iliac artery, left kidney, right kidney, liver, left lung, right lung, pancreas, prostate, rectum, spleen, trachea, bladder, ventricle, and parotid gland.

[0120] In certain embodiments, the one or more target tissue regions include one or more soft tissue regions (eg, organs) selected from the group consisting of: gallbladder, left kidney, right kidney, liver, left lung, right lung, prostate, and bladder.

[0121] In some embodiments, the target tissue region includes one or more bone regions, each of which corresponds to a specific bone [e.g., wherein the target tissue region includes at least a portion (e.g., at most all) of the bones listed in paragraphs

[0116] or

[0117] ]; and one or more soft tissue regions, each of which corresponds to a specific soft tissue region [e.g., wherein the target tissue region includes at least a portion (e.g., at most all) of the soft tissue regions listed in paragraphs

[0118] or

[0119] ].

[0122] In another aspect, the present invention relates to a system for automatically processing 3D images to automatically identify cancerous lesions in an individual, the system comprising: a processor of a computing device; and a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to: (a) receive a 3D anatomical image of the individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the individual; (b) for each of a plurality of target tissue regions, automatically identify a corresponding target volume of interest (VOI) within the 3D anatomical image using one or more machine learning modules (e.g., wherein at least one of the one or more machine learning modules is a convolutional neural network (CNN) module); (c) determine a 3D segmentation map representing a plurality of 3D segmentation masks, each 3D segmentation mask representing a particular identified target VOI (e.g., automatically digitally stitching the plurality of 3D segmentation masks together to form the 3D segmentation map); d) receiving a 3D functional image of the individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)] [e.g., wherein the 3D functional image includes a plurality of voxels each representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within one or more of the target tissue regions of the individual]; (e) identifying, using the 3D segmentation map, within the 3D functional image one or more 3D volumes each corresponding to an identified target VOI (e.g., by mapping a 3D segmentation mask of the 3D segmentation map to the 3D functional image); and (f) automatically detecting, within at least a portion of the one or more 3D volumes identified within the 3D functional image, one or more hot spots based on voxel intensity within the 3D functional image [e.g., based on a comparison of the intensity within the 3D functional image to a threshold value (e.g., wherein the 3D functional image is a 3D functional image] PET images and the threshold is a specific standard uptake value (SUV) level)] determined to represent lesions.

[0123] In certain embodiments, the instructions cause the processor to use the one or more detected hotspots (eg, the intensity of the one or more detected hotspots) to determine the cancer status of the individual.

[0124] In certain embodiments, the target tissue region includes one or more reference tissue regions and wherein the instructions cause the processor to: use the 3D segmentation map to identify one or more 3D reference volumes within the 3D functional image, each corresponding to a particular reference tissue region; determine one or more reference intensity values, each associated with a particular 3D reference volume of the one or more 3D reference volumes and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) within the particular 3D reference volume; determine one or more individual hotspot intensity values, each associated with a particular hotspot of at least a portion of the detected one or more hotspots and corresponding to an intensity measurement value (e.g., an average, a median, a maximum, etc.) of the particular hotspot; and determine one or more individual hotspot index values ​​using the one or more individual hotspot intensity values ​​and the one or more reference intensity values ​​[e.g., wherein each individual hotspot index value is associated with a particular hotspot of at least a portion of the detected one or more hotspots and is determined using (i) the individual hotspot intensity value associated with the particular hotspot and (ii) the one or more reference intensity values ​​(e.g., based on a comparison therebetween)].

[0125] In certain embodiments, the reference tissue region includes one or more members selected from the group consisting of: liver, aorta (eg, a thoracic aorta portion; eg, an abdominal aorta portion), and parotid gland.

[0126] In certain embodiments, the instructions cause the processor to use at least a portion of the one or more hotspot index values ​​to determine an overall index value indicative of a cancer status of the individual.

[0127] In some embodiments, the overall index value is determined as a weighted sum of at least a portion of the individual hotspot index values ​​[e.g., located within a 3D volume of a functional image corresponding to a skeletal region of the individual; e.g., located within a 3D volume of a functional image corresponding to a lymphatic region of the individual] [e.g., where the sum of the individual hotspot index values ​​is weighted by a size measurement (e.g., 3D volume; e.g., mean diameter) of the associated hotspots].

[0128] In certain embodiments, the 3D anatomical images are x-ray computed tomography (CT) images and the 3D functional images are 3D positron emission tomography (PET) images.

[0129] In certain embodiments, a 3D PET image of a subject is obtained after administering a radiopharmaceutical comprising a prostate-specific membrane antigen (PSMA) binding agent to the subject.

[0130] In certain embodiments, the radiopharmaceutical comprises [18F]DCFPyL.

[0131] In another aspect, the present invention relates to a system for automatically processing 3D images to identify and measure uptake of a radiopharmaceutical in cancerous lesions (e.g., cancer metastases) in an individual having or at risk of cancer (e.g., prostate cancer, breast cancer, lung cancer; e.g., metastatic cancer, e.g., metastatic prostate cancer, metastatic breast cancer, metastatic lung cancer), the system comprising: a processor of a computing device; and a memory having stored thereon instructions, wherein the instructions, when executed by the processor, cause the processor to: (a) receive a 3D anatomical image of the individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D The method comprises: (a) automatically identifying within the 3D anatomical image: a first bone volume comprising a graphical representation of one or more bones of the individual; a first aorta volume comprising a graphical representation of at least a portion of the individual's aorta; and a first liver volume comprising a graphical representation of the individual's liver; and (c) determining a 3D segmentation map representing a plurality of 3D segmentation masks, the 3D segmentation masks including a bone mask representing the identified first bone volume, an aorta mask representing the identified first aorta volume, and a 3D segmentation map representing a plurality of 3D segmentation masks. and a liver mask representing the identified first liver volume; (d) receiving a 3D functional image of the individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)] [e.g., wherein the 3D functional image includes a plurality of voxels each representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within one or more bones, aorta portions, and / or liver of the individual]; (e) automatically identifying (e.g., by) within the 3D functional image using the 3D segmentation map mapping the 3D segmentation mask of the 3D segmentation map to the 3D functional image): a second bone volume corresponding to the first identified bone volume within the 3D anatomical image; a second aortic volume corresponding to the first aortic volume identified within the 3D anatomical image; and a second liver volume corresponding to the first liver volume identified within the 3D anatomical image; (f) automatically detecting one or more hot spots within the second bone volume, the hot spots determined to represent a lesion based on voxel intensity within the second bone volume (e.g., the one or more hot spots correspond to localized areas of relatively high intensity, e.g., identified based on a comparison of voxel intensity located within the second bone volume to a threshold);and (g) determining, for each of the one or more detected hotspots, an individual hotspot index value (e.g., a measurement indicating radiopharmaceutical uptake in the lesion represented by the detected hotspot) by: determining an aorta reference intensity level based on intensity measurements (e.g., average, maximum, median, etc.) of the voxels within the second aorta volume; determining a liver reference intensity level based on intensity measurements (e.g., average, maximum, median, etc.) of the voxels within the second liver volume; and for each individual detected hotspot: determining a corresponding individual hotspot intensity level based on the intensity measurements (e.g., average, maximum, median, etc.) of the voxels of the detected hotspot; and determining a corresponding individual hotspot index level based on the individual hotspot intensity levels, the aorta reference intensity level, and the liver reference intensity level.

[0132] In certain embodiments, the instructions cause the processor to determine an overall index value indicative of the cancer status of the individual based on the individual hotspot index values ​​for at least a portion of the one or more detected hotspots.

[0133] In certain embodiments, the subject has or is at risk for prostate cancer.

[0134] In certain embodiments, the instructions cause the processor to: automatically identify, within the 3D anatomical image, a first prostate volume that includes a graphical representation of the individual's prostate, at step (b); include, within the 3D anatomical image, a prostate mask that represents the identified first prostate volume in the determined 3D segmented image, at step (c); automatically identify, within the 3D functional image, a second prostate volume that corresponds to the first identified prostate volume within the 3D anatomical image, at step (e); automatically detect, within the second prostate volume, one or more hot spots; and (i) determine, based on individual hot spot index values ​​for at least a portion of the one or more detected hot spots located in the second bone volume, an overall bone index value that indicates the content (e.g., and severity) of lesions in one or more bones of the individual; and (ii) determine, based on individual hot spot index values ​​for at least a portion of the one or more detected hot spots located in the second prostate volume, an overall prostate index value that indicates the content (e.g., and severity) of lesions in the individual's prostate.

[0135] In certain embodiments, the subject has or is at risk for metastatic cancer (eg, metastatic prostate cancer, metastatic breast cancer, metastatic lung cancer, and other metastatic bone cancers).

[0136] In another aspect, the present invention relates to a system for automatically processing a set of whole-body 3D anatomical images of varying sizes to automatically identify a plurality of 3D volumes corresponding to specific target tissue regions within each 3D anatomical image, the system comprising: a processor of a computing device; and a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to: (a) receive a set of 3D anatomical images of one or more individuals acquired using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical images include a graphical representation of tissue (e.g., soft tissue and / or bone) within each of the one or more individuals, wherein the set of 3D anatomical images has a mean x-dimension, a mean y-dimension, and a mean z-dimension, at least one of which (the mean x-dimension, the mean y-dimension, or the mean z-dimension) has a standard deviation of at least 3% about the corresponding mean [ For example, at least one of them (average x-dimension, average y-dimension or average z-dimension) has a relative standard deviation of at least 3%, at least 5%, at least 10%, at least 15% or at least 20%]; and (b) using a localization module implementing a CNN to automatically determine at least one initial VOI corresponding to a specific anatomical region (e.g., a group of related tissues, such as the pelvic region and the spinal column region) in each image of the 3D anatomical image set, the specific anatomical region including one or more specific related target tissue regions (e.g., one or more specific bones, such as one or more specific organs), thereby identifying at least one initial VOI corresponding to the 3D anatomical image; and (c) for each initial VOI, using one or more segmentation modules that each implement a CNN for each of the one or more specific target tissue regions associated with the specific anatomical region corresponding to the initial VOI to automatically identify the corresponding target VOI (e.g., a graphical representation of the specific target tissue region).

[0137] With respect to another aspect of the invention, features of the embodiments described with respect to one aspect of the invention may be applied. BRIEF DESCRIPTION OF THE DRAWINGS

[0138] The foregoing and other objects, aspects, features and advantages of the present invention will become more apparent and better understood by referring to the following description in conjunction with the accompanying drawings, in which:

[0139] Figure 1 A diagram illustrating the integration of different imaging modalities with an artificial intelligence (AI) platform according to an illustrative embodiment.

[0140] Figure 2 An image showing a skeleton of an individual with different specific bones identified is shown in accordance with an illustrative embodiment.

[0141] Figure 3 A block diagram illustrating the use of whole body segmentation in different functional imaging modalities for different applications according to an illustrative embodiment.

[0142] Figure 4 A diagram showing a process flow for utilizing an AI module to classify, locate, and quantify different information data sets for cancerous lesions according to an illustrative embodiment.

[0143] Figure 5A is a block flow diagram of a process for automatically processing a 3D image to identify a 3D volume within the 3D image that corresponds to a specific target tissue region, according to an illustrative embodiment.

[0144] Figure 5B is a block flow diagram of a process for automatically processing 3D images to identify 3D volumes and using the 3D volumes to detect hot spots representing cancerous lesions, according to an illustrative embodiment.

[0145] Figure 6A is a block flow diagram of a process for automatically processing whole-body segmented CT images according to an illustrative embodiment.

[0146] Figure 6B For display Figure 6A A collection of CT images showing the processing results of the different steps of the process presented.

[0147] Figure 7 is a set of three images used as training data for a machine learning module for segmenting CT images, according to an illustrative embodiment.

[0148] Figure 8 is a collection of two images in which the identified volume of interest is overlapped.

[0149] Figure 9A To administer PyL TM Finally, a collection of 2D views of the 3D PET / CT images obtained from the individual.

[0150] Figure 9B After background correction Figure 9A The same set of 2D views shown in .

[0151] Figure 9C To administer PyL TM Finally, a collection of 2D views of 3D PET / CT images obtained from an individual.

[0152] Figure 9D After background correction Figure 9C The same set of 2D views shown in .

[0153] Figure 10A To illustrate a method for analyzing PyL TM - Block flow diagram of an example process for PET imaging.

[0154] Figure 10B To administer PyL to an individual TM PET / CT images obtained later.

[0155] Figure 10C Two sets of CT images showing volumes of interest identified via the machine learning approach described herein.

[0156] Figure 10D Three sets of images are shown to illustrate background correction of PET images according to an illustrative embodiment.

[0157] Figure 11A is a view of a 3D CT image with volumes of interest corresponding to specific bones marked with colors, according to an illustrative embodiment.

[0158] Figure 11B for Figure 11A Another view of a 3D CT image with a marked volume of interest is shown in .

[0159] Figure 12 CT image with superimposed PET image showing identified background tissue areas.

[0160] Figure 13A A view showing a CT image with an overlaid PET image of a detected lesion.

[0161] Figure 13B To demonstrate the presence of detected lesions Figure 13A Another view of a CT image with an overlaid PET image.

[0162] Figure 13C To demonstrate the presence of detected lesions Figure 13A Another view of a CT image with an overlaid PET image.

[0163] Figure 13D To demonstrate the presence of detected lesions Figure 13A Another view of a CT image with an overlaid PET image.

[0164] Figure 14A To administer PyL to an individual TM The two view sets of CT images with overlapping PET images are obtained.

[0165] Figure 14B Display and Figure 14A The same view, but processed using the machine learning method described in this article, providing background correction and lesion detection.

[0166] Figure 15A To administer PyL to an individual TMThe two view sets of CT images with overlapping PET images are obtained.

[0167] Figure 15B Display and Figure 15A The same view, but processed using the machine learning method described in this article, providing background correction and lesion detection.

[0168] Figure 16A is a view of a 3D CT image with the identified volumes of interest overlaid.

[0169] Figure 16B for Figure 16A Another view of the 3D CT image shown in , with the identified volumes of interest overlaid.

[0170] Figure 17 is a block diagram of an exemplary cloud computing environment used in certain embodiments.

[0171] Figure 18 is a block diagram of an example computing device and an example mobile computing device used in certain embodiments.

[0172] Figure 19 is a set of three figures showing quantification of PyL in lesions identified in specific tissue regions based on PET / CT image analysis for six different cancer indications, according to an illustrative embodiment. TM - Overall indicator value of PSMA absorption.

[0173] Figure 20 is a graph showing individual lesion PSMA Index (LPI) values ​​determined for lesions detected in three different tissue regions (bone, lymph, and prostate) via PET / CT image analysis, according to an illustrative embodiment.

[0174] Figure 21 is a set of two graphs comparing reference values ​​determined via manual and automatic image analysis of blood pool reference values ​​and liver reference values, according to an illustrative embodiment.

[0175] The features and advantages of the present invention will become more apparent from the detailed description set forth below in conjunction with the drawings, in which like reference characters identify corresponding elements throughout. In the drawings, like reference numerals generally indicate identical, functionally similar, and / or structurally similar elements. DETAILED DESCRIPTION

[0176] It is intended that the claimed systems, architectures, devices, methods, and processes of the present invention encompass variations and adaptations made using information from the embodiments described herein. Adaptations and / or modifications of the systems, architectures, devices, methods, and processes described herein may be performed as contemplated by this description.

[0177] Throughout the embodiments, when articles, apparatus, systems, and structures are described as having, comprising, or including specific components, or when processes and methods are described as having, comprising, or including specific steps, it is contemplated that there are additional articles, apparatus, systems, and structures of the invention that consist essentially of, or consist of, the listed components, and that there are processes and methods of the invention that consist essentially of, or consist of, the listed process steps.

[0178] Should be understood that the order of steps or the order in which a certain action is performed is not important, as long as the present invention remains operable.In addition, two or more than two steps or actions can be performed simultaneously.

[0179] Any reference to any publication herein (e.g., in the Prior Art section) does not admit that the publication serves as prior art with respect to any of the claims presented herein. The Prior Art section is presented for clarity and is not intended as a description of prior art with respect to any claim.

[0180] As indicated, the documents are incorporated herein by reference. In the event of any discrepancy in the meaning of a particular term, the meaning provided in the definitions section above shall prevail.

[0181] Headings are provided for the convenience of the reader - the presence and / or placement of headings is not intended to limit the scope of the subject matter described herein.

[0182] i. Definition

[0183] As used herein, "radionucleus" refers to a moiety that includes a radioactive isotope of at least one element. Suitable exemplary radionuclides include, but are not limited to, those described herein. In some embodiments, the radionuclide is one used in positron emission tomography (PET). In some embodiments, the radionuclide is one used in single photon emission computed tomography (SPECT). In some embodiments, a non-limiting list of radionuclides includes 99m Tc, 111 In, 64 Cu, 67 Ga, 68 Ga, 186 Re、 188 Re、 153 Sm, 177 Lu, 67 Cu, 123 I. 124 I. 125 I. 126 I. 131 I. 11 C.13 N. 15 O. 18 F. 153 Sm, 166 Ho, 177 Lu, 149 Pm, 90 Y. 213 Bi, 103 Pd, 109 Pd, 159 Gd, 140 La, 198 Au, 199 Au, 169 Yb, 175 Yb, 165 Dy, 166 Dy, 105 Rh, 111 Ag, 89 Zr, 225 Ac, 82 Rb, 75 Br, 76 Br, 77 Br, 80 Br, 80m Br, 82 Br, 83 Br, 211 At and 192 Ir.

[0184] As used herein, the term "radiopharmaceutical" refers to a compound that includes a radionuclide. In certain embodiments, radiopharmaceuticals are used for diagnostic and / or therapeutic purposes. In certain embodiments, radiopharmaceuticals include small molecules labeled with one or more radionuclides, antibodies labeled with one or more radionuclides, and antigen-binding portions of antibodies labeled with one or more radionuclides.

[0185] As used herein, "3D" or "three-dimensional" with respect to an "image" means conveying information with respect to three dimensions. A 3D image may be presented as a three-dimensional data set and / or may be a collection of two-dimensional representations or displayed in a three-dimensional representation.

[0186] As used herein, an "image" (e.g., a 3D image of an individual) includes any visual representation, such as a picture, a video frame, a streaming video, and any electronic, digital, or mathematical analog of a picture, a video frame, or a streaming video. In certain embodiments, any device described herein includes a display for displaying an image or any other result generated by a processor. In certain embodiments, any method described herein includes a step of displaying an image or any other result generated by the method.

[0187] As used herein, "subject" means a human or other mammal (eg, rodents (mice, rats, hamsters), pigs, cats, dogs, horses, primates, rabbits, and the like).

[0188] As used herein, "administering" an agent means introducing a substance (e.g., an imaging agent) into a subject. Generally, any route of administration can be utilized, including, for example, parenteral (e.g., intravenous), oral, topical, subcutaneous, peritoneal, intraarterial, inhalation, vaginal, rectal, nasal, introduction into the cerebrospinal fluid, or instillation into a body compartment.

[0189] As used herein, the terms "filter" and "filtering" (as in "filter function" or "filter") refer to a function that operates on a localized portion of an input array (e.g., a multidimensional array) of data (e.g., image data, such as values ​​computed by a CNN layer), referred to herein as a "subpatch," thereby computing a response value for a given subpatch. Generally, the filter is applied in a sliding window manner across the entire array to compute multiple response values ​​for the array. Specifically, for a given multidimensional array, a subpatch of the array may be a rectangular region of the array of a particular size (e.g., having the same number of dimensions as the array). For example, for a 6×3×3 array, a given 3×3×3 subpatch refers to a given 3×3×3 set of adjacent values ​​(e.g., a neighborhood) of the array, such that there are five different 3×3×3 subpatch(es) in the 6×3×3 array (each offset by one position along the first dimension).

[0190] For example, for a given sub-block of an array, a filter function can calculate a response value using the values ​​of the sub-block. The filter function can be applied across the entire array in a sliding window fashion, thereby calculating a response value for each of the plurality of sub-blocks of the array. The calculated response values ​​can be stored in an output array to maintain a positional correspondence between the response values ​​and the sub-blocks of the input array.

[0191] For example, in a first step, starting with a sub-block in a corner of the input array, the filter may compute a first response value and store the first response value in the corresponding corner of the output array. In some embodiments, in a second step, the filter then computes a second response value for a second sub-block that is offset by one position along a particular dimension of the input array. The second response value may be stored in the corresponding position in the output array (i.e., offset by one position along the same dimension of the output array). For the complete input array, the following steps may be repeated along each dimension of the input array: offset the position of the sub-block; compute the response value and store the response value in the corresponding position in the output array. In some embodiments (e.g., the strided filtering method), the sub-block for which the filter computes its response value is offset by more than one position each time along a given dimension, such that response values are not computed for every possible sub-block of the input array.

[0192] As used herein, the term "convolutional neural network (CNN)" refers to a type of artificial neural network in which at least one layer performs one or more filtering functions. As used herein, the term "convolutional layer" refers to a CNN layer that receives an input array as input and computes an output array, where the values of the output array are computed by applying one or more filters to the input array. Specifically, in some embodiments, the convolutional layer receives as input an input array having n + 1 dimensions and produces an output array also having n + 1 dimensions. The first n dimensions of the input and output arrays operating on the filtering layer of the CNN are referred to herein as "spatial dimensions". The (n + 1)th dimension of the input is referred to herein as the "input channel" dimension. The size of the input channel dimension is referred to herein as the "number of input channels". The (n + 1)th dimension of the output is referred to herein as the "output channel" dimension. The size of the output channel dimension is referred to herein as the "number of output channels".

[0193] In some embodiments, the convolutional layer computes response values by applying a filter that operates on a sub-block that is smaller than the input array along the spatial dimensions but extends across all output channel dimensions. For example, an input array of size N×M×L×K0 has three spatial dimensions and K0 output channels. The filter of the convolutional layer may operate on a sub-block having a size of N f ×M f ×L f ×K0, where N f ≤N, M f ≤M and L f ≤L. Generally, the filter of the convolutional layer operates on a sub-block having a size where N f <N, M f <M and / or L f <L. For example, in some embodiments, N f <<N, M f <<M and / or Lf < <L。

[0194] Thus, for each of the one or more filters applied by the convolutional layer, the response value calculated by the given filter is stored in the corresponding output channel. Thus, a convolutional layer that receives an input array having n+1 dimensions computes an output array also having n+1 dimensions, where the (n+1)th dimension represents the output channel corresponding to the one or more filters applied by the convolutional layer. In this manner, the output array calculated by a given convolutional layer can be received as input by a subsequent convolutional layer.

[0195] As used herein, the term "size" with respect to a filter of a convolutional layer refers to the size of the spatial dimension along the sub-block on which the filter operates (e.g., considering the sub-block size along the output channel dimension to be the total number of output channels). As used herein, the term "size" with respect to a convolutional layer (as with "size of a convolutional layer") refers to the size of the filter of the convolutional layer (e.g., each filter of the convolutional layer has the same size). In some embodiments, the filters of the convolutional layer have multiple variable parameters determined via a machine learning training procedure. In some embodiments, the number of parameters of a given filter is equal to the number of values ​​in the sub-block on which the given filter operates. For example, a filter of size N operating on an input array with K0 output channels f ×M f ×L f The filter has N f ×M f ×L f × K0 parameters. In certain embodiments, the filter is implemented as an array, and the response value measured by the filter for a given sub-block is calculated as the dot product between the filter and the given sub-block.

[0196] As used herein, the term "fully convolutional neural network (FCNN)" refers to a CNN in which each layer of the CNN is a convolutional layer.

[0197] As used herein, the term "volume" as used with respect to the input or output of a CNN layer refers to the input array received by the CNN layer or the output array computed thereby.

[0198] As used herein, the term "CNN module" refers to a computer-implemented program that implements a particular CNN in order to determine one or more output values ​​for a given input, such as an image (e.g., a 2D image; e.g., a 3D image). For example, a CNN module may receive as input a 3D image of an individual (e.g., a CT image; e.g., an MRI) and, for each voxel of the image, determine a value representing the likelihood that the voxel is within a region of the 3D image corresponding to a representation of a particular organ or tissue of the individual. A CNN module may be software and / or hardware. For example, a CNN module may be implemented entirely in software or certain functions of the CNN module may be performed via dedicated hardware (e.g., via an application specific integrated circuit (ASIC)).

[0199] As used herein, the term "tissue" refers to bone (osseous tissue) as well as soft tissue.

[0200] In certain embodiments, the methods described herein can be used to segment and identify target tissue regions within a full-body image of an individual. As used herein, the terms "full body" and "wholebody" are used interchangeably in the context of segmentation to refer to methods that evaluate a graphical representation of a substantial portion (e.g., greater than 50%) of an individual's body in a 3D anatomical image to identify target tissue regions of interest. In certain embodiments, whole-body segmentation refers to the identification of target tissue regions within at least the entire torso of an individual. In certain embodiments, this also includes portions of the limbs and the head of the individual.

[0201] A. Detection and assessment of cancer status through nuclear medicine imaging and automatic image segmentation

[0202] This article describes systems and methods that provide artificial intelligence (AI)-based segmentation techniques that provide a foundation for detecting, evaluating, and predicting an individual's cancer status. Specifically, the AI-based segmentation techniques described herein allow for accurate and rapid identification of 3D representations of different tissue regions in medical images. The AI-based segmentation techniques described herein utilize machine learning techniques, such as convolutional neural networks (CNNs), to automatically identify multiple target 3D volumes of interest (VOIs), each corresponding to a specific target tissue region, such as one or more organs, portions of organs, specific bones, skeletal regions, and the like. Each identified 3D VOI can be represented via a segmentation mask. Multiple segmentation masks identifying multiple target tissue regions throughout a patient's body can be stitched together to form a segmentation map. The segmentation map and / or the different segmentation masks it comprises can be used to calculate various quantities from the medical image, such as useful indicators for measuring and / or predicting cancer status, progression, and response to treatment. The segmentation map and masks can also be displayed (e.g., as a graphical representation overlaid on the medical image) to guide physicians and other medical practitioners.

[0203] In certain embodiments, AI-based segmentation is performed using anatomical images that provide detailed structural information about the location and extent of tissue within an individual's body. Examples of anatomical images include, but are not limited to, x-ray computed tomography (CT) images, magnetic resonance images (MRI), and ultrasound. Image contrast in anatomical images is a function of the physical properties of subcutaneous tissue, such as density, water, and fat content. As described in further detail herein, the AI-based segmentation techniques of the present invention analyze contrast variations and patterns in anatomical images to identify target 3D VOIs corresponding to different specific target tissue regions.

[0204] In certain embodiments, structural information and identified target VOIs from anatomical images are combined with and / or used to analyze images obtained via functional imaging modalities. Functional images reflect the physiological characteristics and activity of an individual. They are typically obtained using a probe and have intensity variations that reflect the spatial distribution of the probe within the imaged portion of the individual. For example, nuclear medicine images (e.g., PET scans; SPECT scans; whole-body scans; combined PET-CT images; combined SPECT-CT images) detect radiation emitted from radionuclides in radiopharmaceuticals to form images. The distribution of a particular radiopharmaceutical within a patient can be determined by biological mechanisms (e.g., blood flow or perfusion) and by specific enzymatic or receptor binding interactions. Different radiopharmaceuticals can be designed to exploit different biological mechanisms and / or specific enzymatic or receptor binding interactions and, therefore, when administered to a patient, selectively concentrate in specific tissue types and / or regions within the patient. Greater amounts of radiation are emitted from regions of the patient with higher concentrations of the radiopharmaceutical than other regions, causing these regions to appear brighter in nuclear medicine images. Therefore, intensity variations within nuclear medicine images can be used to map the distribution of the radiopharmaceutical within the patient. This mapped distribution of the radiopharmaceutical within a patient's body can be used, for example, to infer the presence of cancerous tissue in different regions of the patient's body.

[0205] Transferring the segmentation map to a functional image (e.g., a nuclear medicine image, such as a PET image or SPECT image) provides valuable context for evaluating and deriving meaning from intensity fluctuations in the functional image. Specifically, it allows for identifying regions of the functional image as corresponding to specific tissue regions of an individual. In this way, the intensity values ​​and fluctuations of voxels within a specific region can be interpreted as potentially deriving from the accumulation of radiopharmaceuticals in specific tissue regions (e.g., organs or bones) of the individual.

[0206] As described in further detail herein, this anatomical content is applicable to a variety of uses. For example, it allows measurement of the absorption of radioactive drugs in different specific target organs and tissue regions (such as prostate, lymph nodes and bone) to be evaluated, and is used as an indicator of the presence of cancerous tissue therein. Identifying what the intensity of a specific region of a functional image is related to also allows it to be evaluated under appropriate background. For example, if the intensity of a specific value is related to the prostate region, the intensity of the same value found in another organ (such as bladder or kidney) is more likely to indicate cancer. In certain embodiments, a reference region is identified and used to calculate a standardized value and level in a certain proportion, and the intensity in other regions is compared thereto to evaluate cancer status. Background regions excluded from the image or used to correct artifacts can also be identified.

[0207] Thus, the AI-based segmentation methods described herein can serve as a platform on which a variety of tools and techniques for detecting, evaluating, and predicting a patient's cancer status can be built. Furthermore, because the AI-based segmentation tools described herein can identify multiple important tissue regions throughout a patient's body and then transfer those identifications (e.g., segmentation maps) from anatomical images to different functional images, they can be used as building blocks for analysis techniques based on a variety of different imaging methods that obtain contrast via a variety of different radiopharmaceuticals.

[0208] Figures 1 to 3 To illustrate this concept. Specifically, Figure 1 Schematic diagram illustrating how different anatomical and functional imaging modalities can be integrated in the context of platform-neutral AI-based segmentation technology. Figure 1 As described in the present invention, the AI ​​segmentation platform described in this article can be used to analyze a variety of images, such as anatomical CT images themselves, 18NaF images, C-11 images, Ga-68 images, FDG images, and PyL TM The basis of the image. Figure 2 Shows the results of instance segmentation performed on an individual's skeleton, where different specific bones are identified by different colors. Figure 2 The segmentation performed using the CT image shown in FIG is transferred to the functional image in order to identify the 3D volume in the functional image corresponding to the target VOI identified in the CT image. Other VOIs (such as soft tissue) can also be identified and used to identify the corresponding 3D volume in the functional image. For example, Figure 3 is a block flow diagram illustrating how different soft tissue and bone regions can be used based on segmentation of an anatomical image (eg, a CT image) and using different probes (eg, 99m Tc-MIP-1404, 18 F-PyL, 68 Ga-PSMA-11, 18 F-NaF, 11 C-choline, [18F]FDG, [18F]FACBC, etc.) and mapped them to images obtained via different functional imaging modalities to perform image-based diagnosis and assessment of disease progression, status, and the like. Figure 3 As shown in , the whole-body segmentation technology described herein can serve as a platform on which a variety of application-specific image analysis techniques utilizing different functional imaging modalities and probes can be leveraged and built upon.

[0209] Figure 4 Schematic diagram illustrating how artificial intelligence-based image analysis 400 may be performed to classify and localize lesions and quantify positive lesions, such as for use in assessing an individual's cancer status. Figure 4, a machine learning module 402 (e.g., a convolutional neural network (CNN)) receives contextual anatomical information (e.g., a CT image or other anatomical imaging modality) 404 and functional information (e.g., a functional image, such as SPECT, PET, or other functional image obtained using a specific imaging probe) 406. The machine learning module 402 then uses the functional 406 and anatomical 404 information to identify, classify, and / or quantify cancerous lesions 410 in an individual. The machine learning module 402 may also leverage additional information 408, such as individual demographics (e.g., age, weight, etc.) and other diagnostic test results (e.g., tissue biopsy, Gleason score, genetic test results, prostate-specific antigen (PSA) test results), clinical treatments (e.g., external beam radiation, brachytherapy, prostatectomy, anti-androgens, immunotherapy (e.g., Provenge), chemotherapy, etc.) as input and also classify, identify, and / or quantify cancerous lesions based on this additional information.

[0210] FIG5 is a block flow diagram illustrating an example process 500 for performing segmentation to identify multiple target tissue regions. As shown in FIG5 , a machine learning module receives a 3D anatomical image (e.g., a whole-body image) as input 502 and, for each of multiple target tissue regions, identifies a target volume 504 within the 3D anatomical image that includes a graphical representation of the target tissue region. The target volumes are stitched together 506 to form a segmentation map 510 comprising multiple segmentation masks 512, each of which represents the identified target volume. The segmentation map 510 may be stored and / or provided for display and / or other processing 508.

[0211] i. Image Segmentation Using Convolutional Neural Networks (CNN)

[0212] In certain embodiments, the AI-based image segmentation methods described herein utilize machine learning techniques (e.g., CNN) to perform segmentation. As described herein, one or more machine learning modules may be used to perform intermediate analysis and processing of the image and ultimately identify the target VOI.

[0213] In certain embodiments, the AI-based image segmentation described herein utilizes a two-step approach to identify one or more 3D target VOIs (VOIs) within an anatomical image that represent regions of interest of target tissue. In this two-step approach, an initial volume of interest (VOI) is first identified, encompassing one or more target VOIs. The initial VOI can be identified using a first module, referred to as a localization module. The localization module can be a first machine learning module, such as a first convolutional neural network (CNN) module. After identifying the initial VOI, a second module, referred to as a segmentation module (e.g., implementing a second machine learning network (e.g., a second CNN) to perform fine segmentation), is used to identify one or more target VOIs within the initial VOI. This approach is advantageous because it allows the segmentation module to operate on standardized inputs of manageable size, regardless of variations in the size and boundaries of the received original anatomical image. This two-step approach is particularly suitable for performing precise, finely resolved segmentation of tissue regions within larger images, such as whole-body scans. The localization module can perform a coarse analysis, such as by performing a rough segmentation, to identify the initial VOI. Without having to adapt to different image sizes and analyze the entire image, the segmentation module can devote resources to performing fine segmentation to accurately identify the target VOI within the initial VOI.

[0214] Furthermore, as described in further detail herein, for example in Example 1, to identify a set of target VOIs extending across the entire body of an individual, multiple initial VOIs may be identified to segment the whole-body image into a manageable number of initial VOIs, each of which includes a subset of the desired target VOIs to be identified. Segmenting the whole-body image into the multiple initial VOIs may be performed using a single localization module, such as a machine learning module that performs coarse segmentation to identify general anatomical regions (e.g., upper body, lower body, spine, and pelvic regions). In certain embodiments, multiple localization modules may be used (e.g., each tasked with identifying one or more initial VOIs). For each identified initial VOI, one or more segmentation modules may then be used to perform fine segmentation and identify one or more desired target VOIs within the initial VOI.

[0215] The different positioning and segmentation modules may be combined and implemented as a single module and / or a single software application, or may be implemented separately, for example as standalone software applications.

[0216] The localization module can use a number of different methods to identify a specific initial VOI. For example, in one approach, the localization module can implement a CNN that receives a grayscale CT image (e.g., a single input channel) as input and outputs the coordinates of opposing corners of a rectangular VOI (e.g., a bounding box). In another approach, the localization module can implement a CNN that receives two input channels: (i) a grayscale CT image and (ii) a preprocessed version of the CT image. The preprocessed version of the CT image can be a thresholded version of the CT image, where the thresholding provides a coarse identification of the initial VOI to be identified. The CNN is trained to analyze both of these input channels in order to output coordinates representing opposing corners of a rectangular initial VOI (e.g., a bounding box).

[0217] In a third approach, to identify a specific initial VOI that includes one or more specific target VOIs, a localization module performs coarse segmentation to roughly identify the specific target VOIs. In this approach, the localization module is essentially a coarse version of the segmentation module used to identify the specific target VOI within the specific initial VOI. Subsequently, a rectangular initial VOI is generated using the roughly identified specific target VOIs (e.g., by drawing a minimal box that fits within it, or perhaps adding a buffer distance). The difference here is that the output of the localization module in this case is not simply the coordinates of the cube's vertices. For example, in this approach, the likelihood value of each voxel in the image is automatically determined, resulting in a likelihood of how the voxel is classified—e.g., whether the voxel is a specific initial VOI or background.

[0218] In certain embodiments, the voxel classification performed by the machine learning module may be improved through an iterative process to reduce noise in the classification process. Specifically, in certain embodiments, a machine learning module (e.g., a CNN module) receives as input the entire image or a portion thereof and, for each specific voxel, outputs a label that classifies the specific voxel as: (i) one of a predefined set of categories corresponding to an anatomical region or target tissue region, or (ii) background. For example, a localization module trained to identify initial VOIs corresponding to the spine, pelvic region, and left and right upper torso may label each voxel as 1, 2, 3, or 4 for the spine, pelvic region, left and right upper torso, respectively, or 0 for background.

[0219] A segmentation module that identifies one or more target tissue regions within a particular initial VOI may receive the particular initial VOI as input and output an integer label for each voxel, where the integer label identifies the voxel as corresponding to one of the particular target tissue regions or background. For example, a segmentation module that identifies different vertebrae may receive an initial VOI corresponding to the spine as input and label each voxel in the initial VOI with an integer corresponding to a particular vertebra (e.g., 1 for the first lumbar vertebra, 2 for the second lumbar vertebra, 3 for the third lumbar vertebra, etc., with other integers labeling other bones).

[0220] Classification performed by a machine learning module such as the one described above can produce noisy results, where some isolated voxels have a different label than their surroundings. For example, a CNN module can produce an output in which the majority of voxels within a larger volume are assigned a first label, while isolated voxels and / or small groups of voxels are assigned a different label, such as a second or third label, thereby identifying them as corresponding to an anatomical region different from their neighbors. Typically, these isolated voxels and / or islands appear at the edges of a larger, uniformly labeled region.

[0221] These small non-uniformities correspond to noise in the classification process and can be removed via the following iterative procedure. First, for each label representing (i) a specific anatomical region or target tissue region or (ii) background, the associated largest connected component of the labeled voxels is identified. Second, for each specific label, isolated voxels and / or voxel islands with different labels that are immediately adjacent to the voxel of the largest connected component associated with the specific label are identified and reassigned the specific label. In certain embodiments, any remaining isolated voxel islands with different labels can then be removed in a third step. In a fourth step, holes in the segmented region (voxels labeled as background but completely enclosed by non-background labeled voxels) are filled (e.g., by assigning the hole's voxel the label of the non-background region that encloses it). Steps two through four are repeated, with isolated voxels and / or voxel islands relabeled based on their surroundings, until convergence occurs (i.e., no change from one iteration to the next). This method reduces and / or eliminates isolated voxels with different labels from their surroundings, thereby reducing noise in the classification process.

[0222] These different methods for identifying an initial VOI can be used individually or in combination to identify multiple initial VOIs within an anatomical image of an individual. After identifying one or more initial VOIs, one or more target VOIs can be identified within each initial VOI. For example, PCT Publication WO2019 / 136349 (the contents of which are incorporated herein by reference in their entirety) provides further details on how a single initial VOI corresponding to the pelvic region can be identified within a CT image; how fine segmentation can be performed to identify target VOIs corresponding to organs (e.g., prostate, bladder, hip muscles); and that different pelvic bones, such as the sacrum, coccyx, and left and right hip bones, can be identified. The methods described herein, and particularly in Examples 1-4, demonstrate how multiple initial VOIs can be identified and used to identify multiple target VOIs throughout an individual's body, thereby providing accurate whole-body image segmentation that can serve as a basis for detecting and assessing cancer status, progression, and response to treatment at both localized and metastatic stages in different organs and tissue regions.

[0223] ii. Provide anatomical content to 3D functional images

[0224] The segmentation map generated by automated AI-based analysis of the anatomical image can be transferred to the 3D functional image to identify within the 3D functional image a 3D volume corresponding to the target VOI identified in the anatomical image. Specifically, in certain embodiments, individual segmentation masks (of the segmentation map) are mapped from the 3D anatomical image to the 3D functional image. The 3D volume identified within the 3D functional image can be used to analyze the image for a variety of purposes to assess cancer status.

[0225] Certain identified 3D volumes and corresponding target VOIs correspond to tissue regions where cancer is suspected and / or may develop. These regions may include, for example, the prostate, breast tissue, lung, brain, lymph nodes, and bone. Other regions may also be evaluated. Certain regions (e.g., prostate, breast, and lung) are relevant for detecting cancer at early, localized stages, while other regions (e.g., lymph nodes and bone) are relevant for assessing metastatic cancer. Because the intensity of functional images (e.g., PET and SPECT images) maps the spatial distribution of radiopharmaceutical accumulation in a patient's body, by accurately identifying specific 3D volumes in functional images that correspond to specific tissue regions, the voxel intensities within those 3D volumes can be used to determine absorption measurements of radiopharmaceutical probes within their corresponding specific tissue regions. Because radiopharmaceutical probes can be designed to selectively accumulate in cancerous tissue (e.g., by enhancing affinity for biomolecules overexpressed in cancerous tissue, such as prostate-specific membrane antigen (PSMA)), detecting and quantifying the absorption of specific probes in certain target tissue regions and organs of interest is indicative of an individual's cancer status and / or can be used to determine an individual's cancer status. For example, as described in PCT Publication WO2019 / 136349 (the contents of which are incorporated herein by reference in their entirety), assessing 1404 absorption in the prostate volume can be used to determine the prostate cancer status of an individual. Other probes can be used to assess metastatic cancer present in a wide variety of other tissue regions, including bone. For example, Examples 2 to 6 describe methods for identifying PyL TM The accumulation of cancerous lesions in the patient's entire body is segmented.

[0226] Thus, in certain embodiments, segmentation performed on an anatomical image (e.g., a CT image) is transferred (e.g., mapped) to a co-registered functional image (e.g., a PET or SPECT image), thereby allowing specific tissue volumes of interest (VoIs) within the functional image corresponding to specific tissue regions of interest to be identified. Accumulation of the radiopharmaceutical within those specific tissue regions can then be quantified based on the intensity of the functional image voxels located within the specific tissue volumes of interest.

[0227] A variety of methods can be used to analyze the intensity of three-dimensional pixels in functional images. For example, the average intensity, median intensity, total intensity, maximum intensity, etc. within a specific volume can be calculated and used for quantification. This calculated value can then be compared with other values ​​calculated (e.g., standardized) for other tissue regions and used to assign the cancer status to the patient (e.g., based on comparison with one or more predetermined thresholds). In certain embodiments, localized areas of high intensity (called hotspots) are identified within a specific 3D volume. As described in further detail in Section B below, these localized hotspots can be identified as representing cancerous lesions and used to determine the cancer status of an individual. In certain embodiments, a machine learning method can be used. For example, the intensity of functional image three-dimensional pixels within a volume of interest of one or more specific tissues can be used as input to a machine learning module, which calculates a risk indicator that can be used to quantify the risk of cancer, cancer metastasis, or cancer recurrence, etc. and / or is compared with a reference value (e.g., a threshold) to specify a specific cancer status.

[0228] In certain embodiments, in addition to identifying specific tissue regions where cancerous tissue may be present (e.g., to determine the presence and / or status of cancer therein), other additional tissue regions may be identified. These additional tissue regions may correspond to background regions where specific radiopharmaceutical probes normally accumulate without necessarily the presence of a lesion.

[0229] In certain embodiments, the identified background regions are used to normalize voxel intensities in order to normalize intensity values ​​across images. For example, as described in PCT Publication WO2019 / 136349 (the contents of which are incorporated herein by reference in their entirety), gluteal muscles can be identified and their absorption can be used to normalize the intensity of SPECT images obtained after administration of 1404.

[0230] Identification of certain background regions can also be used to explain high probe accumulation levels in these regions. In certain embodiments, certain background regions are identified and excluded from analysis. For example, as described in Example 3, certain probes (e.g., PyL TM ) can normally accumulate in certain background areas where no cancerous lesions or tissues are present. Therefore, these areas can be identified and excluded, for example, by hotspot detection analysis. Examples of these areas include the kidneys, duodenum, small intestine, spleen, liver, pancreas, stomach, adrenal glands, rectum, and testicles.

[0231] In certain embodiments, identifying 3D volumes corresponding to background areas in a functional image can be used to correct for intensity bleed effects, where high accumulation of a radiopharmaceutical in a specific area can produce a high-intensity functional image not only in the 3D volume corresponding to the specific area, but also in its neighborhood. This intensity bleed into other areas of the functional image can obscure useful signals. For example, radiopharmaceuticals typically accumulate in higher amounts in a patient's bladder. When imaged via a functional imaging modality, this high accumulation in the bladder can contribute to the intensity in the 3D volume corresponding to tissue areas outside the bladder (e.g., the prostate) via scattering effects. Therefore, accounting for this intensity bleed or "cross-talk" can improve the accuracy with which intensity in 3D functional images is used to measure potential radiopharmaceutical absorption.

[0232] In certain embodiments, the dilation corresponds to a 3D volume of a particular background region that produces intensity bleed-through.The dilated background region can be excluded from analysis in order to avoid using the intensity of regions directly adjacent to the background region to determine absorption metrics or identification of hot spots.

[0233] In certain embodiments, a suppression method that simulates a profile of functional intensities can also be used to adjust the intensity levels of neighboring regions to correct for intensity bleed-through. In this method, for a specific background region that produces intensity bleed-through, the amount of suppression (i.e., intensity bleed-through) removed from a specific voxel in the functional image is determined by the distance of the voxel from a high-intensity core region within a specific 3D background volume corresponding to the specific background region. The high-intensity core region can be determined as a region including voxels having an intensity above a predetermined value, or within a specific portion of the maximum intensity within the specific region of the functional image.

[0234] In certain embodiments, this suppression is utilized if the maximum functional image intensity within a 3D volume identified as corresponding to a specific background region is greater than a specific multiplier value multiplied by a determined background intensity value. The background intensity value can be determined based on the voxel intensity of the 3D functional image corresponding to a specific reference volume corresponding to a specific tissue region (e.g., buttocks muscle). The suppression method can be applied to a subregion of the 3D functional image near the specific background region that produces intensity bleed. For example, it can be applied to a rectangular subregion encompassing the specific region plus predetermined boundaries.

[0235] In certain embodiments, one or more intensity bleed-through functions are determined to suppress and thereby correct the voxel intensities of the 3D functional image for bleed-through (e.g., crosstalk). For example, the 3D functional image can be cropped into the aforementioned rectangular sub-regions encompassing a specific region plus predetermined boundaries. The determined background intensity value can be subtracted from the voxel intensities within the cropped image region. Sample intensities can then be collected to determine how the intensity of radiopharmaceutical absorption originating from a specific background region decreases as a person moves away from a specific background 3D volume corresponding to the specific background region. Samples can be used that start at the extreme top, right, left, and bottom, and then move upward, right, left, and downward, respectively. Other directions are also possible. The sampled intensities provide curves (e.g., a set of sampled intensity data points) that show the decrease in intensity from a high-intensity core. A template function (e.g., an nth-degree polynomial) can be fit to these sampled curves and used to calculate intensity values ​​near a specific 3D background volume to be used as correction factors.

[0236] For example, PCT Publication WO2019 / 136349 (the contents of which are incorporated herein by reference in their entirety) describes how identification of the bladder region in 1404-SPECT images can be used to adjust prostate voxel intensity to account for intensity bleed-through due to high accumulation in the bladder. In certain embodiments, similar methods can be used for other images acquired with other probes that may accumulate in the bladder and / or other regions (e.g., liver, kidneys, etc.).

[0237] In certain embodiments, the segmentation methods described herein assume that a 3D anatomical image contains certain anatomical regions. For example, embodiments of the systems and methods described herein may assume that their input anatomical images always contain the pelvic region and automatically attempt to segment the pelvic region. For other anatomical regions, the systems and methods described herein may also only segment these regions if they are included in the anatomical image input, e.g., by first performing a determination as to whether they are present.

[0238] Methods for performing whole body segmentation to identify target tissue regions corresponding to bone and / or soft tissue (e.g., organs) are described in further detail below in Examples 1, 2, 3, and 4. Methods of using these whole body segmentations to assess disease status, particularly cancer, in patients are also described.

[0239] B. Hotspot and Lesion Detection

[0240] In some embodiments, instead of or in addition to quantifying overall absorption in a particular volume within a functional image, localized hot spots (localized areas of relatively high intensity) are detected. In some embodiments, hot spots are detected via a thresholding method - by comparing the intensity of stereo pixels within the functional image to one or more thresholds. Groups of stereo pixels with intensities above the threshold can be detected as hot spots. A single global threshold can be used, or in some embodiments, multiple region-specific thresholds can also be used. For example, segmentation of co-registered anatomical images can be used to set different thresholds for different tissue regions for hot spot detection. As described herein, segmentation of co-registered images can also be used to remove the effects of background signal, thereby facilitating hot spot detection (e.g., whether a global threshold and / or multiple region thresholds are used).

[0241] In certain embodiments, hotspots can be detected using additional speckle detection techniques. One approach uses a Difference of Gaussian method, in which the PET image is filtered using a combination of high-pass and low-pass filters approximating a Gaussian kernel. The filters reduce background noise and are constant for variations in background levels in different regions of the image (for example, the thorax may have significantly higher background levels than the pelvis due to significant uptake in the liver and kidneys). This cascaded high- and low-pass filter approach allows hotspot extraction without the use of a fixed threshold, but can instead identify local variations in PET intensity. Another approach uses the Laplacian of Gaussian speckle detection method. The Laplacian of Gaussian method is a method for detecting edges in an image using Laplacian and Gaussian filter kernels. By using kernels of different sizes, edges of structures of different sizes are detected. Choosing the appropriate kernel size allows the method to detect structures with characteristics common to lesions. The described methods can be used individually, with only one of the techniques applied to all regions of interest, or they can be used simultaneously, with different methods applied to different tissue regions, such as by semantic segmentation of co-registered anatomical images combined with a single global threshold or multiple local thresholds.

[0242] Figure 5BAn example process 520 is shown for detecting hotspots representing cancerous lesions using the segmentation methods described herein. In a first step, a 3D anatomical image 522 is received. One or more target VOIs 524 corresponding to specific target tissue regions are identified within the 3D anatomical image, and the identified target VOIs are stitched together 526 to create a segmentation map 534 comprising a plurality of segmentation masks 536. Each segmentation mask of the segmentation map represents a specific target VOI. Next, a 3D functional image (e.g., a PET image) is received 528, and the segmentation map is transferred to the 3D functional image to identify a corresponding 3D volume 530 in the 3D functional image for each target VOI. The identified 3D volumes are then used to detect 532 within the 3D functional image, corresponding to localized areas of high intensity relative to their surroundings. Specifically, one or more specific 3D volumes may correspond to specific tissue regions where cancerous lesions may form and are analyzed to detect hotspots therein. Other 3D volumes may be used to perform background correction or intentionally excluded from the hotspot detection process. As described in further detail herein, yet other volumes may be relevant in that they serve as reference volumes for calculating useful metrics that can be assigned to hotspots in order to score them, eg, to indicate the risk / severity of the underlying lesion they represent.

[0243] Hotspots can also be classified as, for example, cancerous or non-cancerous after their initial detection, and / or assigned a likelihood value representing their likelihood of becoming a cancer metastasis. Hotspot classification can be performed by extracting hotspot features (e.g., metrics describing the characteristics of a particular hotspot) and using the extracted hotspot features as the basis for classification (e.g., via a machine learning module).

[0244] In certain embodiments, hotspot classification can also be performed without the use of machine learning. In these embodiments, anatomical knowledge can be combined with information about the shape and position of the detected hotspot to classify the hotspot as a cancerous lesion or noise. For example, if the detected hotspot is located on the edge of the rib facing the liver and the detected peak is not the largest in the entire region around the hotspot, it is possible to estimate whether the hotspot is a tumor or a non-tumor based on the given anatomical and spatial information. The segmentation of the co-registered anatomical images can also be used to create an intensity model of background tissue areas that are known to contain no cancer cells, but in which particularly high functional image stereo pixel intensities are common. These background areas can cause the intensity outside the boundaries of the background area itself to bleed through and affect the adjacent areas where cancerous lesions may exist. The profile of the intensity level can be estimated and used to subtract the estimated additional intensity levels present in the adjacent tissue areas carrying cancerous lesions to allow for more robust hotspot detection.

[0245] Hot spots representing lesions can be used to determine risk indicators that provide an indication of the presence and / or status of a patient's disease (e.g., cancer status, similar to the Gleason score). For example, metrics (e.g., the number of hot spots, the total summed intensity of identified hot spots, the area fraction of a particular body part or region (e.g., the skeleton) occupied by the hot spots, and the like) themselves can be used as and / or used to calculate these risk indicators. In certain embodiments, regions identified via the segmentation methods described herein can be used to calculate risk indicators, such as for calculating metrics (e.g., area fractions). Examples of methods for calculating risk indicators using identified hot spots are provided in Examples 5 and 7 herein.

[0246] C. Example of CNN-based whole body segmentation and lesion detection method

[0247] The following example demonstrates the use of the AI-based segmentation and hotspot detection methods described herein for whole-body segmentation, detecting cancerous lesions and determining useful metrics for evaluating an individual's cancer status.

[0248] i. Example 1 - CNN-based full body segmentation

[0249] Example 1 describes an example method for whole-body segmentation. The implementation in this example uses five neural networks to segment bones throughout the torso. The first neural network is used to roughly locate different regions of the body. The results are used to divide the body into four regions. In each of these regions, a corresponding neural network is then used to perform segmentation into different bones. The results from all four regions are then combined into a final result (e.g., a final segmentation map).

[0250] This method involves a two-step "bounding box" approach described herein, in which a first machine learning module (e.g., a localization module) is used to roughly localize an initial volume of interest (VOI) corresponding to an anatomical region that includes one or more specific target tissue regions (e.g., a prostate). A second machine learning module (e.g., a segmentation module) then performs fine segmentation within the initial VOI to identify a target volume within the initial VOI that corresponds to the target tissue region. In this case for whole-body segmentation, the first machine learning module (localization module) identifies multiple initial VOIs, each corresponding to a different anatomical region. Subsequently, for each anatomical region, a corresponding second machine learning module (segmentation module) identifies one or more target volumes, each representing a specific target tissue region. The machine learning modules (e.g., neural networks) can be implemented as independent modules, or certain machine learning networks can be combined. For example, each second segmentation network can be implemented as an independent module, or within a single module.

[0251] Thus, in this example, the first module (localization) is used to identify anatomical regions within the CT; that is, to find volumes of interest where the network of the second module can be applied to generate segmentation maps for further analysis. The network in the second module operates on full-resolution CT images, while the localization network operates in low resolution using subsampled versions of the CT images.

[0252] Three example versions of the CNN network were used in software that implemented whole-body segmentation according to the method. Versions 1 and 2 segmented 49 bones, and version 2 segmented 49 bones and 8 soft tissue regions.

[0253] CNN-based segmentation platform instance version 1

[0254] In the first example version of the CNN network for whole body segmentation, the first machine learning module (localization module) is called "coarse-segmentation" in this example and is trained to identify 49 bones in subsampled CT images (subsampling factor 4 along each dimension). The localization module is used to differentiate the regions of the body into the pelvic region, spine, left upper body, and right upper body. The four fine segmentation networks are as follows:

[0255] ● "Fine Segmented Pelvis": trained to identify the left and right ilium, sacrum, and coccyx;

[0256] ● "Fine Segmented Spine": trained to identify 12 thoracic vertebrae, 5 lumbar vertebrae, and the sternum;

[0257] ● "Fine Segmented Left Upper Body": trained to identify the 12 ribs, left shoulder blade, and left clavicle on the left side of the body; and

[0258] ● “Fine Segmented Right Upper Body”: trained to identify the 12 ribs, right shoulder blade and right clavicle on the right side of the body.

[0259] The input image size of each network is as follows:

[0260] Table 1: Input image sizes for five neural networks.

[0261] Network Name Input image size (number of slices, rows, columns) Rough segmentation (81,70,104) Finely segmented pelvis (93,146,253) Fine segmented spine (194,204,94) Fine segmentation of the left upper body (158,187,144) Fine segmentation of the right upper body (158,191,146)

[0262] When the coarse segmentation network of the localization module receives (as input images) a 3D anatomical image representing a large physical volume and encompassing a large portion of an individual's body, the actual number of voxels in its input image is lower than that of the other network due to the subsampling factor of 4. The number and size of the voxels of the input image to the localization module are also reduced by cropping (removing) image regions that do not contain a graphical representation of tissue, but instead represent air. Removing these voxels as a preprocessing step further reduces the size of the image input to the localization module (see, for example, the first two columns of Table 4 below). Reducing the size of the image input to the localization module through subsampling allows the coarse segmentation network to trade the resolution of the image it operates on for additional filtering, allowing it to, for example, account for the rate of change of the image it receives and perform more accurate and robust coarse segmentation to identify different initial volumes of interest.

[0263] The number of filters and parameters used in each neural network are listed in Tables 2 to 4 below:

[0264] Table 2: The number of convolutional filters in five neural networks.

[0265] Network Name Total number of filters The number of filters in the first layer Rough segmentation 4096+49 (49 categories) 32 Finely segmented pelvis 4096+3 32 Fine segmented spine 2048+18 16 Fine segmentation of the left upper body 2048+14 16 Fine segmentation of the right upper body 2048+14 16

[0266] Table 3: Number of parameters in the five neural networks

[0267] Network Name Total parameters Number of trainable parameters Number of non-trainable parameters Rough segmentation 5,881,978 5,878,678 3,300 Finely segmented pelvis 5,880,276 5,877,068 3,208 Fine segmented spine 1,472,815 1,471,177 1,638 Fine segmentation of the left upper body 1,472,731 1,471,101 1,630 Fine segmentation of the right upper body 1,472,731 1,471,101 1,630

[0268] Table 4 below also shows the change rates of the input image sizes for the original data and the five neural networks in this example. As shown in the table, the change rates of the input image sizes for the four second fine segmentation networks are smaller than those for the first localization network because the identification of the initial VOI produces a more standardized input for the second machine learning module.

[0269] Table 4: Original data and input image sizes of five neural networks.

[0270]

[0271] CNN-based segmentation platform instance version 2

[0272] The updated second example version of the CNN whole body segmentation system as described above includes in this example adjustments to the input image size, the number of convolution filters, and the parameters used in the neural network used to perform segmentation. Tables 5, 6, and 7, as seen below, show updated values ​​for different parameters in the five neural networks. Table 5 shows updated values ​​for the input image size shown in Table 1, Table 6 shows updated values ​​for the number of convolution filters for the neural network shown in Table 2, and Table 7 shows updated values ​​for the number of parameters used by the neural network shown in Table 3.

[0273] Table 5: Updated values ​​of input image size for five neural networks.

[0274]

[0275]

[0276] Table 6: Updated values ​​of the number of convolutional filters in five neural networks.

[0277] Network Name Total number of filters The number of filters in the first layer Rough segmentation 4096+49 (49 categories) 32 Finely segmented pelvis 3328+3 26 Fine segmented spine 2048+18 16 Fine segmentation of the left upper body 2048+14 16 Fine segmentation of the right upper body 2048+14 16

[0278] Table 7: Updated values ​​of the number of parameters in five neural networks

[0279] Network Name Total parameters Number of trainable parameters Number of non-trainable parameters Rough segmentation 5,882,423 5,879,132 3,300 Finely segmented pelvis 3,881,200 3,883,812 2,612 Fine segmented spine 1,472,815 1,471,177 1,638 Fine segmentation of the left upper body 1,472,731 1,471,101 1,630 Fine segmentation of the right upper body 1,472,731 1,471,101 1,630

[0280] CNN-based segmentation platform instance version 3

[0281] Another third example version of the CNN whole body segmentation method is used to segment soft tissue regions and bones. As described in this paper, this third version includes two coarse segmentation modules used in parallel, referred to in this paper as "coarse segmentation 02" and "coarse segmentation 03".

[0282] The "Coarse Segmentation 02" module is trained to identify 49 bones in subsampled CT images. The "Coarse Segmentation 03" module is trained to identify 49 bones and the liver in subsampled CT images. The "Coarse Segmentation 02" module outperforms the "Coarse Segmentation 03" module in locating bones, and to leverage the benefits of each module, both are used in parallel to identify the initial volume of interest (e.g., "bounding box") for different fine segmentation networks. Specifically, in Version 3, seven fine segmentation networks are used. Six of the seven fine segmentation networks use "Coarse Segmentation 02" for initial volume of interest identification, and the seventh ("Fine Segmentation Abdomen") uses "Coarse Segmentation 03" for initial volume of interest identification.

[0283] The seven fine segmentation networks of this third instance version of the CNN full body segmentation system are as follows:

[0284] "Fine Segmented Abdomen": trained to identify the liver, left and right kidneys, and gallbladder;

[0285] ● “Fine Segmentation Left Lung”: trained to identify the left lung;

[0286] ● “Fine Segmented Right Lung”: trained to identify the right lung;

[0287] "Finely segmented mixed pelvic region": trained to identify the left and right ilium, prostate, bladder, sacrum, and coccyx;

[0288] ● "Fine Segmented Spinal Bone": trained to identify 12 thoracic vertebrae, 5 lumbar vertebrae and the sternum;

[0289] ● "Fine Segmented Left Upper Body Bones": trained to identify the 12 ribs, left shoulder blade, and left clavicle on the left side of the body; and

[0290] ● “Fine Segmented Right Upper Body Bones”: trained to identify the 12 ribs, right scapula and right clavicle on the right side of the body.

[0291] Tables 8, 9, and 10 below show the values ​​of different parameters used in the seven neural networks in version 3 of the CNN-based segmentation system.

[0292] Table 8: Input image sizes for seven neural networks and two localization networks.

[0293] Network Name Input image size (number of slices, rows, columns) Rough segmentation 02 (81,77,99) Rough segmentation 03 (81,77,99) Finely segmented abdomen (92,176,259) Fine segmentation of the left lung (154,171,140) Fine segmentation of the right lung (154,171,141) Fine segmentation of the left upper body bones (154,171,140) Fine segmentation of the right upper body bones (154,170,140) Finely segmented blended pelvic area (92,144,251) Finely segmented spinal bones (192,183,115)

[0294] Table 9: The number of convolutional filters in seven neural networks and two localization networks.

[0295]

[0296]

[0297] Table 10: Number of parameters in seven neural networks and two localization networks.

[0298] Network Name Total parameters Number of trainable parameters Number of non-trainable parameters Rough segmentation 02 5,882,432 5,879,132 3,300 Rough segmentation 03 5,882,469 5,879,167 3,302 Finely segmented abdomen 1,473,003 1,471,369 1,634 Fine segmentation of the left lung 1,472,752 1,471,120 1,632 Fine segmentation of the right lung 1,472,752 1,471,120 1,632 Fine segmentation of the left upper body 1,472,731 1,471,101 1,630 Fine segmentation of the right upper body 1,472,731 1,471,101 1,630 Finely segmented blended pelvic area 3,883,812 3,881,200 2,612 Fine segmented spinal column 1,472,815 1,471,177 1,638

[0299] Thus, this example demonstrates how the segmentation method described herein can be used to perform effective whole-body segmentation.

[0300] ii. Example 2: Automatic segmentation of skeleton in low-dose CT and [ 18 F]DCFPyLPET for the determination of metastatic prostate cancer quantity

[0301] PSMA-PET / CT hybrid imaging is a promising diagnostic platform for prostate cancer patients. Although manual delineation of organs in 3D CT images is often required for accurate diagnosis and treatment planning, this manual delineation is a time-consuming process. To address this challenge, this example demonstrates the use of deep learning methods to automate the process of accurate bone segmentation in whole-body CT images based on the whole-body segmentation technique described in this article. As described in this example, the anatomical information obtained through this bone segmentation can be used in [ 18 F]DCFPyL(PyL TM -PSMA) PET / CT images to create a fully automatic lesion detection algorithm.

[0302] A deep learning algorithm based on a cascaded deep learning convolutional neural network was developed for semantic segmentation of 12 skeletal regions. Specifically, the 12 skeletal regions are the thoracic and lumbar vertebrae, left / right ribs, sternum, left / right clavicle, left / right scapula, left / right ilium, and sacrum. A training set (N=90) and a validation set (N=22) of low-dose CT images and manually refined segmentation pairs were used to develop the deep learning algorithm. TM The algorithm's performance was evaluated on a test set (N=10) of low-dose CT images obtained from the PSMA study. Five representative body parts were manually segmented from the test set of images: the left ilium, lumbar spine, left ribs, right scapula, and sternum. These manual segmentations served as ground truth for evaluating the automated segmentation algorithm.

[0303] Automatic segmentation can be used for automatic lesion detection. For example, an automatic lesion detection method using a hard threshold based on the Standard Uptake Value (SUV) of the PET image voxel intensity can be implemented.

[0304] The Sorensen-Dice score was used to evaluate the accuracy of the automatic segmentation. The segmentation method achieved a mean and standard deviation of Sorensen-Dice scores of 0.95 and 0.024 on the training set, and a mean and standard deviation of 0.93 and 0.036 on the validation set. For the test set, the mean values ​​(with standard deviation values ​​shown in parentheses) for each of the five regions were as follows: 0.94 (0.016) for the right clavicle, 0.90 (0.023) for the left rib, 0.92 (0.019) for the sternum, 0.94 (0.033) for the lumbar spine, and 0.97 (0.0033) for the left ilium. The overall mean (relative to all body parts) was 0.93, with a standard deviation of 0.030.

[0305] Therefore, this example demonstrates the accuracy of the fully automatic segmentation method for 12 bone regions in whole-body low-dose CT images and its application in PyL TM - Use of automated lesion detection methods for PSMA / CT hybrid imaging.

[0306] iii. Example 3: for PyL TM -Automatic whole-body segmentation for PET image analysis and lesion detection

[0307] This example demonstrates the automatic segmentation of 49 bone and 27 soft tissue regions in a whole-body CT image using a deep learning approach based on the whole-body segmentation technique described in this paper. This example also demonstrates how the anatomical information obtained through this segmentation can be used in [ 18 F]DCFPyL(PyL TM-PSMA) PET / CT images. This example also shows how segmentation can be used to remove background signals from PET images to facilitate the observation and detection of PyL TM Accumulated lesions.

[0308] Figure 6A The PyL described in this example is shown TM -Block flow diagram of an embodiment of the segmentation process described herein in PET / CT image analysis. As with process 500 shown in FIG5 , an anatomical image is received in process 600. Specifically, in process 600, a CT image 610a is received. In this example, process 600 is used to identify the CT image and segment it into target volumes of interest corresponding to 49 specific bones and 8 soft tissue regions (in this example, using Example Version 3 of the CNN-based segmentation method described in Example 1). To identify the target volume, coarse segmentation 620a is performed to locate an initial volume of interest or set of sub-regions, such as described in Example 1. Fine segmentation is then performed within each of the sub-regions to identify specific target volumes of interest 630a corresponding to the target 49 bones and 8 soft tissue regions. Segmentation masks 640a representing the identified target volumes can be created and merged, for example, to create a 3D whole-body segmentation map.

[0309] As mentioned in Example 1 (CNN network version 3), the specific regions segmented are as follows:

[0310] 49 bones:

[0311] Left clavicle

[0312] Right clavicle

[0313] Left hip bone

[0314] Right hip bone

[0315] Left rib_1

[0316] Left rib_10

[0317] Left rib_11

[0318] Left rib_12

[0319] Left rib_2

[0320] Left rib_3

[0321] Left rib_4

[0322] Left rib_5

[0323] Left rib_6

[0324] Left rib_7

[0325] Left rib_8

[0326] Left rib_9

[0327] Right rib_1

[0328] Right rib_10

[0329] Right rib_11

[0330] Right rib_12

[0331] Right rib_2

[0332] Right rib_3

[0333] Right rib_4

[0334] Right rib_5

[0335] Right rib_6

[0336] Right rib_7

[0337] Right rib_8

[0338] Right rib_9

[0339] Sacrum and coccyx

[0340] Left scapula

[0341] Right scapula

[0342] sternum

[0343] Lumbar vertebra_1

[0344] Lumbar vertebra_2

[0345] Lumbar vertebra_3

[0346] Lumbar vertebra_4

[0347] Lumbar vertebra_5

[0348] Thoracic vertebra_1

[0349] Thoracic vertebra_10

[0350] Thoracic vertebra_11

[0351] Thoracic vertebra_12

[0352] Thoracic vertebra_2

[0353] Thoracic vertebra_3

[0354] Thoracic vertebra_4

[0355] Thoracic vertebra_5

[0356] Thoracic vertebra_6

[0357] Thoracic vertebra_7

[0358] Thoracic vertebra_8

[0359] Thoracic vertebra_9

[0360] 8 soft tissue areas:

[0361] gallbladder

[0362] Left kidney

[0363] Right kidney

[0364] liver

[0365] left_lung

[0366] right_lung

[0367] prostate

[0368] bladder

[0369] Figure 6B A series of CT images are shown overlaid with labels illustrating the steps in the whole-body segmentation process 600 described in this example. An example raw CT image received at step 610a is shown in 610b. Image 620b shows the results of a coarse segmentation used to identify an initial volume of interest in the CT image (the identified distinct regions are shown as colored areas). Image 630b shows a "bounding box" identifying the initial volume of interest, within which fine segmentation is performed to identify target volumes of interest corresponding to 49 bone and 8 soft tissue regions. Image 640b illustrates the final merged whole-body segmentation.

[0370] Steering Figure 7 ,To train a machine learning module for performing segmentation, a number of pre-labeled sample images (e.g. Figure 7 The three images (710, 720, 730) shown in Figure 1 were used as a training dataset. The machine learning module was fine-tuned using 1000 manually annotated labels and 100 models trained for 1000 hours on dozens of GPUs before finding the optimal configuration for many components of the segmentation platform. However, once trained, segmentation can be performed quickly, with results (e.g. Figure 8 (810 and 820 are two example images) and Figure 16A and 16B ) are obtained in a few seconds (typically less than 180 seconds).

[0371] The anatomical content obtained by segmenting CT images can be used to detect lesions in PET images that are co-registered with the CT images, for example, in a typical PET / CT imaging mode where CT and PET images of an individual are acquired in rapid succession, with the individual maintained in substantially the same position as the recorded image. Specifically, as demonstrated in this example, segmentation of the CT image can be transferred to the PET image to identify regions of interest (e.g., the prostate or skeleton), wherein the entire functional image, except for regions expected to harbor primary or secondary prostate cancer tumors, can be excluded from the lesion detection algorithm. Without wishing to be bound by any particular theory, exclusion of regions by the lesion detection algorithm can be particularly important for background regions where accumulation can cause high intensities in PET image voxels within and around the background tissue region. If high intensities in background tissue regions are not excluded from the prostate lesion detection process, this can lead to misclassification of background noise as cancerous lesions. In the regions that remain after the background exclusion process, a simple threshold (e.g., an SUV of 3) can be used in conjunction with a lesion classification algorithm to discover hotspots within the tissue region of interest. The classification algorithm can be used as a simple check to determine the location of the hotspot and compare the intensities of the hotspot's neighborhood. If the hotspot is part of a larger hotspot and is located on the edge of a body part (eg, a rib close to the liver), the lesion may be classified as noise and excluded. Figures 9A to 9D The need for and benefits obtained thereby for such a background subtraction and lesion detection algorithm are illustrated, wherein only regions of interest are included in the detection procedure and portions of the image contain high-intensity background voxels. Figure 9A and 9C A PET image is shown superimposed on a CT image in false colors of white to red to blue. Blue indicates low PET image intensity, transitioning from red to white as intensity increases. Figure 9B and 9D 9A and 9C show the same PET images but with all background absorption removed from the PET images. Figure 9B and 9D In the PET image without background subtraction, a few localized hot spots corresponding to cancerous lesions are visible. In the PET image without background subtraction, these hot spots are drowned and obscured by the large background signal.

[0372] Figure 10A An example process 1000 is shown for using the anatomical context provided by whole-body CT segmentation to detect lesions and determine useful risk indicators for use in administering PyL-containing radiopharmaceuticals. TM The PET / CT images obtained after the quantification of the individual's cancer status (e.g., prostate cancer status). Figure 10AAs shown in FIG, a PET / CT image is obtained 1010, the anatomical content is determined via segmentation of bone and soft tissue 1012 and used to obtain measurements (eg, measurements of lesions and / or risk indicators) 1014 from the PET image. Figure 10B An example PET / CT image 1016 is shown, and Figure 10C Example segments 1018a and 1018b of bone and soft tissue are shown. Figure 10D As shown in , segmentation of bone and soft tissue (e.g., shown in image 1020a) can be used to remove background signal (e.g., from background tissue regions, such as those shown in image 1020b), leaving only the desired signal from hotspot 1020c indicative of cancerous lesions.

[0373] Figures 11 to 13 illustrate this process, showing how segmentation of a CT image can be transferred to a PET image and used to provide anatomical content for background removal and lesion detection. Specifically, Figure 11A and Figure 11B Shows different views of a CT image where bone segmentation has been performed. Figure 11A and Figure 11B In the view shown in , different segmented bones are marked with different colors. It is worth noting that with the help of the "bounding box" method described in this article, even this complex and large-scale segmentation can be performed extremely quickly. The image shown in Figure 11A and 11B Segments in .

[0374] like Figure 12 As shown in , the segmentation of bone and soft tissue can be used to account for normal absorption in certain background tissue regions that can occur under normal circumstances and mask useful hot spot signals indicating the presence of cancerous lesions. TM In PET / CT imaging performed, normal uptake can occur in the kidneys, duodenum, small intestine, spleen, liver, pancreas, stomach, adrenal glands, rectum, and testes. Figure 12 The aforementioned areas are shown identified in red. Figure 13A As shown in the figure, after subtracting the normal PyL in these areas TM After the background intensity is accumulated, the lesion becomes easily observable and detectable. One such lesion 1302 is Figures 13A to 13D Lesions (eg, lesion 1302) can be detected, for example, via a threshold method, and classified as PyL, for example, as described above. TM Positive (ie, indicative of cancer). Like segmentation, lesion detection is also rapid. Lesions 1302 are detected in less than 5 seconds.

[0375] Figure 14A and Figure 14BResults of the PET image processing and lesion detection methods described above are presented for PET / CT images obtained from a 65-year-old male with metastatic castration-resistant prostate cancer. The patient's prostate-specific antigen (PSA) score was 6.8 ng / ml. PET images were obtained after administration of PyL-1 as a radiopharmaceutical. TM After obtaining. Figure 14A Displays the initial, raw PET / CT images. Figure 14B The image after processing described in this article is shown. Background intensity is removed, and several small lesions can be easily observed. Segmentation of 49 bones and 8 soft tissue regions (e.g., organs) was performed in less than 180 seconds.

[0376] Figure 15A and Figure 15B The results of PET image processing and lesion detection methods are presented for PET / CT images obtained from a 54-year-old man with metastatic castration-resistant prostate cancer. The patient's PSA score was 33.55 ng / ml. TM After obtaining. Figure 15A Displays the initial, raw PET / CT images. Figure 15B The image after processing described in this article is shown. The background intensity is removed, and several small lesions can be easily observed. Segmentation of 49 bones and 8 soft tissue regions (e.g., organs) was performed again in less than or about 180 seconds.

[0377] Therefore, this example shows how the whole body segmentation method described in this paper can be used to automatically segment 27 soft tissue organs and 49 bones for PyL TM Contextualizing images to detect, quantify, and track PyL TM Affinity lesions. The method will allow clinicians / physicians to ask clinically relevant questions for better management of prostate cancer patients. Advantages (e.g., (statistically) demonstrated increased diagnostic accuracy, precision, speed, and reproducibility in the context of artificial intelligence-assisted 1404-SPECT image analysis (see, e.g., PCT publication WO2019 / 136349, the contents of which are incorporated herein by reference in their entirety)), for PyL TM -PET images are also available.

[0378] iv. Example 4: Example bone and soft tissue segmentation region

[0379] This example provides a list of example bones and soft tissue regions that can be identified through segmentation of CT images using a system developed using the methods and embodiments described herein. Specifically, 67 bones and 22 soft tissue regions that have been manually labeled (e.g., identified by one or more human experts) in a CT image collection are listed below. These manually labeled CT images can be used as training data for the machine learning method described herein. For example, although Examples 1 to 3 describe the current software version of a whole-body segmentation method that segments 49 bones and 8 soft tissue regions, its functionality can be easily updated to segment any number of the 67 bones and 22 soft tissue regions listed in this example. Therefore, this example shows that embodiments of the systems and methods described herein can be developed to identify similar regions, including (but not necessarily limited to) the specific regions described in this example. Certain systems and methods can identify tissue regions that are not necessarily listed in this example. In certain embodiments, both bones and soft tissue regions are identified. In certain embodiments, some systems and methods can only identify bones or only identify soft tissue.

[0380] As indicated in the following list, some left and right bones are identified as independent tissue regions (e.g., left clavicle and right clavicle), and in some cases, the individual members of large groups of bones are identified individually. For example, the following example list shows that individual ribs and vertebrae are identified through the segmentation method of this example (specific ribs and vertebrae are numbered in the list). This example should also be clear that the method described herein can be used for segmenting multiple regions of the whole body, including, but not necessarily limited to, the region listed herein.

[0381] Segmented area:

[0382] Bones (67)

[0383] Left clavicle

[0384] Right clavicle

[0385] Left femur

[0386] Right_FemurLeft_FibulaRight_FibulaLeft_HipboneRight_HipboneLeft_HumerusRight_HumerusMandibularLeft_PatellaRight_PatellaLeft_RadiusRight_RadiusLeft Rib_1Left Rib_2Left Rib_3Left Rib_4Left Rib_5Left Rib_6Left Rib_7Left Rib_8Left Rib_9Left Rib_10Left Rib_11Left Rib_12Right Rib_1Right Rib_2Right Rib_3Right Rib_4Right Rib_5Right Rib_6Right Rib_7Right Rib_8Right Rib_9Right Rib_10Right Rib_11Right Rib_12Sacrum_and_CoccyxLeft_ScapulaRight_ScapulaSkull

[0387] Left sternum_Right tibia_Left tibia_Right ulna_All ulna_Cervical vertebrae_1 Lumbar vertebrae_2 Lumbar vertebrae_3 Lumbar vertebrae_4 Lumbar vertebrae_5 Lumbar vertebrae_6 Thoracic vertebrae_1 Thoracic vertebrae_2 Thoracic vertebrae_3 Thoracic vertebrae_4 Thoracic vertebrae_5 Thoracic vertebrae_6 Thoracic vertebrae_7 Thoracic vertebrae_8 Thoracic vertebrae_9 Thoracic vertebrae_10 Thoracic vertebrae_11

[0388] Thoracic vertebra_12

[0389] Soft tissue (22):

[0390] Left adrenal gland

[0391] Right adrenal gland

[0392] Abdominal aorta_part

[0393] Thoracic aorta_part

[0394] brain

[0395] Left bronchus

[0396] Right bronchus

[0397] gallbladder

[0398] Left_gluteus maximus

[0399] Right_Gluteus maximus

[0400] heart

[0401] Left kidney

[0402] Right kidney

[0403] liver

[0404] left_lung

[0405] right_lung

[0406] pancreas

[0407] prostate

[0408] rectum

[0409] spleen

[0410] bladder

[0411] ventricles

[0412] v. Example 5: Calculation of Hot Spot Indices for Radiopharmaceutical Uptake Quantification and Clinical Endpoint Assessment

[0413] Example 5 is an example method for using the segmentation and hotspot detection methods described herein to calculate a hotspot index value that can be used to infer and / or quantify the absorption of a radiopharmaceutical within a lesion represented by the detected hotspot for a specific detected hotspot. The calculated hotspot index can be related to clinical endpoints, including patient survival and is relevant to determining treatment strategies. When calculated for multiple images collected at different time points, the calculated index value can be compared with each other for a specific patient, and changes in the index are used to evaluate the efficacy of the treatment and predict how the index will change in the near future. In certain embodiments, the calculated index can predict sensitivity to treatment with a targeted imaging ligand. In certain embodiments, the calculated index can also be included in a nomogram for effective patient stratification.

[0414] The method in this example uses a method that is tailored to a particular patient and includes a PSMA-binding agent (e.g., PyL TM However, the method described herein is neutral to the specific radiopharmaceutical used for imaging and can be used with a variety of different radiopharmaceuticals, such as 99m Tc-MIP-1404, 18 F-PyL, 68 Ga-PSMA-11, 18 F-NaF, 11 C-choline, [18F]FDG, [18F]FACBC, etc.

[0415] A machine learning-based approach according to the systems and methods described herein is used to identify different target VOIs within a CT image. As described herein, a target VOI is a volume in a CT image that is automatically identified by a segmentation method to correspond to a specific target tissue region. In this example method, target VOIs corresponding to the liver, a portion of the aorta, and the parotid gland are identified. As described herein, these specific tissue regions (liver, portion of the aorta, and parotid gland) serve as reference regions for calculating hotspot indices. In addition to the reference regions, other target VOIs corresponding to other target tissue regions may also be identified. A segmentation mask representing the identified target VOIs is mapped to a PET image to identify corresponding 3D volumes within the PET image. In this manner, 3D reference volumes corresponding to the liver, portion of the aorta, and parotid gland are identified in the PET image. Once identified, each specific 3D reference volume is used to calculate a corresponding reference intensity value that provides an intensity measurement of the voxels within the specific 3D reference volume. In this example, the mean intensity within each volume is used, although other measurements (e.g., median, maximum, mode, etc.) are possible.

[0416] To calculate an index value for a particular identified hotspot, a hotspot intensity value is calculated for that particular hotspot and compared to a reference intensity value. Similar to the reference value, the hotspot intensity value provides an intensity measurement of the voxels of the hotspot. In this example, a maximum value is calculated, but as with the reference value, other measurements can be used. As this example illustrates, the specific measurement used to calculate the hotspot intensity value need not be the same as the measurement used to calculate the reference intensity value. To calculate the hotspot index value, the reference intensity value can be mapped to the reference index value at a certain ratio, and the hotspot index value can then be calculated based on whether the hotspot intensity value is above, below, or between the reference values.

[0417] For example, the reference intensity value calculated from the reference volume corresponding to the aorta portion (this aortic region is used to provide a measure of absorption in the blood pool and may also be referred to as blood or blood pool reference value) is typically the smallest, followed by the liver and then the parotid gland region. Thus, a hotspot whose hotspot intensity value is equal to the blood reference value is assigned a hotspot index value of 100; a hotspot whose hotspot intensity value is equal to the liver reference value is assigned a hotspot index value of 200; and a hotspot whose hotspot intensity value is equal to the parotid gland reference value is assigned a hotspot index value of 300. Hotspot index values ​​for hotspot intensity values ​​between two reference intensity values ​​can be determined by interpolation (e.g., linear interpolation).

[0418] In this approach, detected hotspots can be assigned index values ​​that quantify the level of radiopharmaceutical uptake in the specific lesion represented by the hotspot in a standardized manner (e.g., so that it can be compared between different images). As described herein, these indices can be correlated with survival rates, which in turn make them useful for treatment management. For example, more or less aggressive treatments can be considered depending on the patient's expected outcome. The cost of treatment can also be factored in. Indicators are particularly useful when measured over time. When comparing indices across patients over time and across multiple imaging examinations, changes in the indices can be used to evaluate treatment efficacy and predict how the indices will change in the near future.

[0419] It is worth noting that the significant advantage of the method described in this example over previous methods is the ability to calculate reference intensity values ​​(and hotspot intensity values) based on the automatically identified 3D volume provided by the artificial intelligence-based segmentation method described herein. Previous methods that attempt to quantitatively represent image areas of lesions rely on manually marking (e.g., via placing circular markers) areas in 2D slices to identify 2D regions of interest located within a reference tissue region. In contrast to these small 2D regions, the 3D volume identified by the method used herein captures intensity throughout the entire organ, thereby providing increased accuracy and repeatability. In addition, by using precise automatic segmentation, a further increase in accuracy and repeatability is provided.

[0420] In addition, the method of this example uses a continuously changing index calculated by interpolation, rather than using one of a small number of values ​​to classify detected lesions. This method provides more detailed information that can be used to manage treatment strategies and track disease progression and / or treatment efficacy over time in a fine-grained and accurate manner.

[0421] vi. Example 6: Automated Hotspot Detection and Uptake Quantification in Bone and Local Lymph

[0422] PyL TM -PSMA PET / CT hybrid imaging (e.g., in patients receiving PyL TM PET / CT images of patients obtained after surgery) are promising tools for detecting metastatic prostate cancer. The image segmentation, hotspot detection and quantification techniques of the present invention can be used to provide abnormal PyL in bones and regional lymph nodes (i.e., lymph nodes localized in the patient's pelvic region and / or in substantial proximity thereto). TM - Basis for Automated Quantitative Assessment of PSMA Uptake. In particular, as demonstrated in this example, image segmentation and hotspot detection techniques according to the systems and methods described herein can be used for automated analysis of PET images to detect hotspots that a physician may identify as malignant lesions.

[0423] In this example, PET / CT scans are evaluated to automatically identify hotspots corresponding to potential bone and regional lymph node lesions within the scanned PET images. For each scan, semantic segmentation of the CT image is performed using the deep learning methods described herein in order to identify a set of specific bone and soft tissue regions (e.g., organs). Once obtained, the CT image segments are transferred (e.g., mapped) to the PET images of the PET / CT scans to identify corresponding 3D volumes in the PET images. In each PET image, the intensity corresponding to background absorption is removed by suppressing the intensity in the identified volumes corresponding to the bladder, liver, and kidneys. Segmentation is also used to define the relevant volume in which the hotspots are detected. Subsequently, a spot detection algorithm is applied to identify abnormal hotspots representing possible bone lesions or possible malignant regional lymph nodes. Reference volumes corresponding to the thoracic portion of the liver and aorta are used to calculate reference SUV values.

[0424] The accuracy of the hotspot detection method in this example was validated by comparing hotspots detected in PET images using the automated method described herein with manual annotations by a panel of physicians identifying bone lesions and lymph node lesions. A set of 157 PET / CT scans annotated as bone lesions (114 images did not have any lesions and 11 images had more than three lesions) was used to evaluate the accuracy of detecting hotspots corresponding to bone lesions and a set of 66 scans annotated as regional lymph node lesions (40 images with no lesions and six with more than three lesions). The bone detection algorithm identified 97% of all annotated bone lesions, with an average of 109 hotspots per image. The regional lymph node detection algorithm found 96% of all annotated malignant regional lymph nodes, with an average of 32 hotspots per scan.

[0425] In addition, the accuracy of deep learning segmentation is evaluated in this example. The segmentation algorithm is trained to segment a set of 52 bones and 7 soft tissue regions (e.g., organs) used to define hot search areas or as reference areas. For each specific area (bone or soft tissue area), manual identification of that area in the CT image is used to perform and evaluate training and validation. For example, 140 manually identified livers are used to train the algorithm for liver segmentation, and 37 manually identified livers are used for validation. Similarly, 61 and 14 manually identified aortas are used to train and validate aortic regional segmentation, respectively. For liver segmentation, segmentation scores of 0.99 and 0.96 were obtained for training and validation sets, respectively. For aortic segmentation, segmentation scores of 0.96 and 0.89 were obtained for training and validation, respectively. Finally, ten additional image sets that were not developed using the algorithm were used to evaluate generalization. For these ten images, the segmentation scores characterizing the segmentation accuracy of the liver and aorta were 0.97±0.01 and 0.91±0.5, respectively.

[0426] Below is a complete list of the specific 52 bones (the sacrum and coccyx are listed on a single line as "Sacrum_and_Coccyx," but correspond to two segmented regions) and 7 soft tissue regions used in this particular example.

[0427] Bone area:

[0428] Left scapula

[0429] Left clavicle

[0430] Left clavicle

[0431] Right clavicle

[0432] Left hip bone

[0433] Right hip bone

[0434] Left rib_1

[0435] Left rib_10

[0436] Left rib_11

[0437] Left rib_12

[0438] Left rib_2

[0439] Left rib_3

[0440] Left rib_4

[0441] Left rib_5 Left rib_6 Left rib_7 Left rib_8 Left rib_9 Right rib_1 Right rib_10 Right rib_11 Right rib_12 Right rib_2 Right rib_3 Right rib_4 Right rib_5 Right rib_6 Right rib_7 Right rib_8 Right rib_9 Sacrum and coccyx Left scapula Right scapula Sternum Lumbar vertebra_1 Lumbar vertebra_2 Lumbar vertebra_3 Lumbar vertebra_4 Lumbar vertebra_5 Thoracic vertebra_1 Thoracic vertebra_10 Thoracic vertebra_11 Thoracic vertebra_12 Thoracic vertebra_2

[0442] Thoracic vertebra_3

[0443] Thoracic vertebra_4

[0444] Thoracic vertebra_5

[0445] Thoracic vertebra_6

[0446] Thoracic vertebra_7

[0447] Thoracic vertebra_8

[0448] Thoracic vertebra_9

[0449] Soft tissue areas:

[0450] Abdominal aorta_part

[0451] Thoracic aorta_part

[0452] Left kidney

[0453] Right kidney

[0454] liver

[0455] prostate

[0456] bladder

[0457] Therefore, this example demonstrates the usefulness of deep learning based semantic segmentation, which is used in [18 F ]DCFPyL(PyL TM -PSMA) PET / CT images and automatically identify hot spots and calculate SUV reference values.

[0458] vii. Example 7: Lesion PSMA Scoring and PSMA-Weighted Lesion Involvement

[0459] This example provides a method for assigning a hotspot index to detected hotspots based on a comparison of individual hotspot intensities with a reference level, and then using the assigned individual hotspot indices to determine an overall index representing a weighted sum of measurements of lesion size. The index used in this example is used to assess the cancer status of patients imaged using a PSMA binder and PET-CT imaging.

[0460] Specifically, individual hotspot indices are determined based on these reference levels via an interpolation method similar to that described in Example 5. In this example, reference VOIs corresponding to the aorta portion and the liver portion are segmented and mapped to corresponding 3D volumes in the PET image. The aorta portion volume is used to determine the blood reference intensity, and the liver volume is used to determine the liver reference intensity. In this example, each reference intensity is determined by its corresponding volume by taking the mean intensity (SUV) in the corresponding volume, but other measurements (such as maximum, peak, or median) can also be used. In a certain proportion, a reference level (called Lesion PSMA Score (LPS)) is assigned to the intensity values ​​based on the blood and liver reference intensities (SUV) as follows: LPS 0 is assigned to 0 SUV level, LPS 1 is assigned to the blood reference intensity, LPS 2 is assigned to the liver reference intensity, and the maximum LPS 3 is assigned to the reference intensity calculated as twice the liver reference intensity.

[0461] Individual hotspots were assigned an LPS score based on their individual intensity. For individual hotspots with intensities ranging from 0 to the maximum reference intensity (twice the liver reference intensity), the LPS score corresponding to the intensity of the individual hotspot was interpolated according to the reference scale. Hotspots with intensities exceeding the maximum reference intensity were assigned a maximum LPS score of 3.

[0462] Two example overall risk indices that can be calculated using detected hotspots and individual hotspot indices (LPS scores) are calculated as a weighted sum of the hotspot sizes in specific volumes corresponding to tissue regions where cancerous lesions may occur. The first example indicator is the PSMA-weighted total bone / lymph / prostate lesion volume or ratio (PTLV or PTLR). This indicator is a weighted sum of lesion volumes, with the weights being the lesion PSMA scores. For 3D volumes corresponding to bone (e.g., skeletal regions), lymph nodes, and prostate, the sum is calculated separately as a weighted sum of the hotspot volumes of the hotspots found in each specific region. Specifically, the weighted sum is calculated as follows:

[0463] ∑(lesion volume × lesion PSMA score)

[0464] In some cases, a ratio may be preferred and can be calculated by dividing the weighted sum by the total 3D volume of a particular region (e.g., in a PET image). Weighting the summed hotspot volume by LPS, as opposed to, for example, SUV or normalized SUV, is advantageous because PSMA, expressed as a (normalized) SUV value, may not be linearly related to the aggressiveness of the disease. That is, for example, one cannot assume that a hotspot with an intensity of 100 represents a lesion five times worse than a lesion represented by a hotspot with an intensity of 20. Calculating the LPS score and weighting the hotspots by the LPS score yields a ratio for comparing different hotspots.

[0465] Another example metric is the PSMA-weighted bone / lymphatic aggregate diameter (PLAD). This metric is also the sum of lesion size measurements (weighted by the LPS score), but this metric uses the mean diameter of each hotspot (e.g., mean x-diameter, y-diameter, and z-diameter) rather than volume. Because volume is a three-dimensional quantity, smaller volume changes of large lesions can dominate over size changes of smaller lesions (e.g., causing large fluctuations in the sum). Using the mean diameter mitigates this effect. This metric is calculated as follows:

[0466] ∑(mean lesion diameter × lesion PSMA score)

[0467] For bones and lymph, a weighted total diameter can be calculated.

[0468] vii. Example 8: Improved AI-assisted image analysis for patients with low-risk or intermediate-risk prostate cancer Improved performance

[0469] 99mTc MIP-1404 (1404) is a PSMA-targeted imaging agent used to detect and stage clinically significant prostate cancer. Manual assessment of tracer uptake in SPECT / CT images introduces inherent limitations in inter- and intra-reader standardization. This example describes a study evaluating the performance of PSMA-AI-assisted readouts, where automated segmentation of prostate volumes and other tissue regions of interest was performed according to embodiments described herein, compared to manual assessment and known clinical predictive values.

[0470] The study analyzed 464 evaluable patients with very low-risk, low-risk, or intermediate-risk prostate cancer whose diagnostic biopsy indicated Gleason grade ≤3+4 and / or were candidates for active surveillance (1404-3301). All individuals received IV injection of 1404 and performed SPECT / CT imaging 3-6 hours after administration. Three independent readers evaluated the images. All individuals underwent voluntary RP (low-risk and intermediate-risk) or prostate biopsy (very low-risk) after administration. Clinically significant disease was declared in individuals with Gleason grade 7 or higher. PSMA-AI was developed and locked before analysis. Three different independent readers used PSMA-AI to obtain quantitative expression of 1404 in the prostate against background (PSMA index). The PSMA index of all readers and individuals were compared with histopathological reference values ​​to obtain 6 receiver operating characteristic (ROC) curves (3 manual readings + 3 PSMA-AI assisted readings). The clinical performance of the 1404PSMA-AI-assisted readout was also evaluated by comparing the area under the receiver operating characteristic (ROC) curve (AUC) of a multivariate model (PSA, clinical stage, and diagnostic Gleason score) with and without the PSMA indicator.

[0471] Manual readings showed AUCs of 0.62, 0.62, and 0.63. Reads with PSMA-AI showed AUCs of 0.65, 0.66, and 0.66. PSMA-AI performance was superior to manual in terms of AUC in all 3×3=9 pairwise comparisons between the two reading groups (with statistically significant improvements observed in five cases (nominal p<0.05)), regardless of multiple comparisons. The predictive power of the baseline multivariate model (without PSMA index) was at AUC 0.74. After adding the PSMA index, the model predictive power increased to AUC 0.77. The logistic regression model indicated that the PSMA index (p=0.004), preoperative PSA (0.018), and % positive cores (p=<0.001) were significantly associated with clinically significant disease. When measuring reproducibility, the logarithm (PSMA index) correlation coefficients of the PSMAAI reading pairs were 0.94, 0.97, and 0.98.

[0472] Thus, the studies described in this example demonstrate that PSMA-AI provides a standardized platform to generate reproducible quantitative assessments of 1404. The PSMA-AI-assisted readout demonstrated additional improvement over known predictive values ​​for identifying men with clinically significant disease.

[0473] ix. Example 9: Automatic calculation of 18F-DCFPyL uptake for quantitative PET / CT images for prostate cancer staging PSMA indicator

[0474] This example demonstrates automated image segmentation, lesion detection, and calculation of standardized index scores based on hotspot indices assigned to detected lesions via embodiments of the methods described herein. TM ) after which the patient's cancer status was evaluated via PET / CT scan imaging.

[0475] In this example, a cascaded deep learning pipeline is used to segment the organs of interest in the CT image and project the segments into the PET image space. Specifically, target regions corresponding to the bone volume corresponding to the individual's skeleton, the lymphatic volume corresponding to the lymphatic region, and the prostate volume corresponding to the prostate are segmented in the CT image and mapped to identify the corresponding 3D volumes in the PET image. Similarly, the aorta and liver volumes corresponding to portions of the aorta and the liver are also segmented and mapped to the PET image for use as reference regions as described herein.

[0476] Hot spots are detected in PET images. Manual segmentation is considered to represent hot spots of lesions, as well as other lesions not detected by the algorithm. Each individual detected hot spot is quantified by calculating an individual hot spot index called the lesion miPSMA index (LPI). As described in Examples 6 and 7 above, the LPI used in this example is a continuous index calculated using automatic segmentation (as opposed to manual identification) volume. Therefore, it provides advantages over previous methods that utilize manual identification of different organs in the image and classify lesions using only a small, limited number of enumerated values. Therefore, this method provides more detailed information that can be used to manage treatment strategies and track disease progression and / or treatment efficacy over time in a fine-grained and accurate manner.

[0477] The LPI for each detected hotspot is calculated based on reference values ​​determined from the blood pool (as measured from the identified aortic volume) and the liver reference region, for example, similar to that described above in Example 7. Specifically, the blood pool reference intensity (SUV value) is measured using the aortic volume within the PET image, and the liver reference intensity is measured from the liver volume identified in the PET image. Both the blood pool and liver reference intensity levels are measured as the mean SUV (SUV) within the corresponding volume in the PET image. mean Reference LPI index values ​​of 1 and 2 are assigned to the blood pool and liver reference intensities, respectively. A maximum reference index level of 3 is assigned to a reference intensity corresponding to twice the liver reference intensity value.

[0478] Each individual detected hotspot is assigned an LPI score based on (i) the measured hotspot intensity value of the hotspot and (ii) a comparison with the blood pool and liver reference intensity values ​​and the aforementioned reference index levels. Specifically, for each hotspot, the individual hotspot intensity value corresponding to the mean lesion uptake is calculated as the average SUV of all voxels of the hotspot. For an individual hotspot, if the mean lesion standard uptake value (SUV) is mean ) equal to the blood pool reference value, an LPI of 1 is assigned; for hotspots with an average lesion uptake value equal to the liver reference uptake value, an LPI of 2 is assigned; and an LPI of 3 is assigned to hotspots with an average lesion uptake value equal to or greater than twice the liver reference uptake. For hotspots whose intensities fall between the reference intensity values, the individual hotspot LPIs are interpolated based on the hotspot intensity value and the reference intensity-index value pair.

[0479] The total lesion volume within the bone region is calculated as the weighted sum of the volumes of the individual hotspots detected within the bone volume in the PET image, where the volume of each hotspot is weighted by its corresponding LPI. In this way, the PSMA-weighted total lesion volume (PTLV) index for the bone region is calculated, denoted as PTLV bone Similarly, the PTLV lymph ) and prostate (PTLV prostate For each of the three regions, PET / CT image datasets of individuals with a variety of different indications were used to automatically determine the PTLV index value.

[0480] The performance of the AI-based automatic technique was evaluated based on comparison with manual organ segmentation and image annotation to identify hotspots, and to compare the PTLV index values ​​calculated for different indications across the entire region.

[0481] Using human expert interpretation as the highest criterion for comparison, the automatic hotspot detection algorithm was determined to have a sensitivity of 92.1% for bone lesions (97.2% in 52 automatically segmented bones) and 96.2% for lymphatic lesions. On average, 17 bone hotspots were detected per scan and 23 lymphatic hotspots were detected per scan.

[0482] PTLV index values ​​were calculated using a sample dataset of 197 PET / CT images of the bone region, 99 PET / CT images of the lymphatic region, and 43 PET / CT images of the prostate region. In the dataset, 94% of the individual hotspots measured had LPIs between 1 and 3, with the minimum LPIs measured in the bone, lymphatic, and prostate regions being 0.82, 1.1, and 1.3, respectively. The median LPIs for the bone, lymphatic, and prostate regions were 1.5, 2.0, and 2.3, respectively.

[0483] For each region, for individuals with multiple different indications, calculate the following PTLV metrics: treatment response (TR), screening (S), new diagnosis (ND), metastatic (M), suspected recurrence (SR), recurrence (R). By calculating the mean of the median of the interquartile range (IQR mean ), compare the PTLV metric values between indications, thus excluding outliers. By the IQR of the PTLV values mean sort the indications to obtain: for the bone region, TR < S < ND < R < SR < M, for the lymph region, S = M < R < ND < SR, and for the prostate region, M < SR < R < S < ND, thus well aligning with the clinical expectations of the disease state.

[0484] Figure 19 Show the PTLV metric values measured for different indications in each region. Figure 20 Show scatter plots of the individual hot spot LPI values of the hot spots detected in each of the three regions.

[0485] Compared with the manual method (e.g., Eiber et al., 2017) for determining the reference value of the grading lesion, the blood and liver automatic reference values are based on a larger image volume compared to the manual reference method and are thus expected to be more robust. Figure 21 Compare the reference values calculated via the automatic segmentation method described herein with the reference values calculated using manual discrimination of the organ boundary for the blood pool (left curve) and liver (right curve) regions. As Figure 21 shown, in a sample of 20 PET / CT scans, the correlation (Pearson's r) between the automatic and manual references is 0.92 for blood and 0.87 for liver.

[0486] Therefore, this example shows the use of automatic segmentation, hot spot detection, and metric value calculation to detect cancer and track cancer progression over time and / or the response to treatment.

[0487] D. Imaging agents

[0488] In some embodiments, the 3D functional image is a nuclear medicine image using an imaging agent including a radiopharmaceutical. The nuclear medicine image is obtained after administering the radiopharmaceutical to the patient and provides information about the distribution of the radiopharmaceutical within the patient. The radiopharmaceutical is a compound including a radionuclide.

[0489] Nuclear medicine images (e.g., PET scans; e.g., SPECT scans; e.g., whole-body scans; e.g., combined PET-CT images; e.g., combined SPECT-CT images) detect radiation emitted from radionuclides in a radiopharmaceutical to form an image. The distribution of a particular radiopharmaceutical within a patient can be determined by biological mechanisms (e.g., blood flow or perfusion) as well as by specific enzymatic or receptor binding interactions. Different radiopharmaceuticals can be designed to exploit different biological mechanisms and / or specific enzymatic or receptor binding interactions and, therefore, when administered to a patient, selectively concentrate in specific types of tissue and / or regions within the patient. Greater amounts of radiation are emitted from regions of the patient's body that have higher concentrations of the radiopharmaceutical than other regions, causing these regions to appear brighter in the nuclear medicine image. Thus, intensity variations within a nuclear medicine image can be used to map the distribution of the radiopharmaceutical within the patient's body. This mapped distribution of the radiopharmaceutical within the patient can be used, for example, to infer the presence of cancerous tissue in different regions of the patient's body.

[0490] For example, upon administration to a patient, technetium-99m methylene diphosphonate ( 99m Tc MDP) selectively accumulates in areas of the patient's skeleton, particularly at sites of abnormal bone formation associated with malignant bone lesions. The selective concentration of the radiopharmaceutical at these sites produces identifiable hot spots - localized areas of high intensity in nuclear medicine images. Therefore, the presence of malignant bone lesions associated with metastatic prostate cancer can be inferred by identifying these hot spots within the patient's whole-body scan. Risk indicators related to the patient's overall survival rate and other prognostic metrics indicating disease status, progression, treatment efficacy, and the like can be based on the presence of high intensity identifiable hot spots - localized areas of high intensity identifiable in nuclear medicine images. 99m In some embodiments, it can also be calculated by automatic analysis of intensity changes in whole body scans obtained after Tc MDP. 99m Other radiopharmaceuticals are used in a similar manner for Tc MDP.

[0491] In certain embodiments, the specific radiopharmaceutical used depends on the specific nuclear medicine imaging modality used. For example, 18 Sodium fluoride (NaF) also accumulates in bone lesions (similar to 99m In some embodiments, PET imaging can also utilize a radioactive form of the vitamin choline, which is readily taken up by prostate cancer cells.

[0492] In certain embodiments, radiopharmaceuticals that selectively bind to specific proteins or receptors of interest, particularly those whose expression is increased in cancer tissue, may be used. These proteins or receptors of interest include, but are not limited to, tumor antigens such as CEA, which is expressed in colorectal cancer; Her2 / neu, which is expressed in various cancers; BRCA1 and BRCA2, which are expressed in breast and ovarian cancers; and TRP-1 and TRP-2, which are expressed in melanoma.

[0493] For example, human prostate-specific membrane antigen (PSMA) is upregulated in prostate cancer, including metastatic disease. Almost all prostate cancers express PSMA, and its expression is further increased in poorly differentiated, metastatic, and hormone-refractory tumors. Therefore, radiopharmaceuticals corresponding to PSMA binders (e.g., compounds with high affinity for PSMA) labeled with one or more radionuclides can be used to obtain nuclear medicine images of patients, from which the presence and / or status of prostate cancer in multiple regions of the patient (e.g., including, but not limited to, skeletal regions) can be assessed. In certain embodiments, when the disease is localized, nuclear medicine images obtained using PSMA binders are used to identify the presence of cancerous tissue within the prostate. In certain embodiments, when the disease is metastatic, nuclear medicine images obtained using radiopharmaceuticals including PSMA binders are used to identify the presence of cancerous tissue in multiple regions, including not only the prostate but also other relevant organs and tissue regions, such as the lungs, lymph nodes, and bones.

[0494] Specifically, after administration to a patient, the radionuclide-labeled PSMA-binding agent selectively accumulates in cancer tissue based on its affinity for PSMA. 99m In a manner similar to that of Tc MDP, selective concentration of a radionuclide-labeled PSMA-binding agent at specific sites within a patient creates detectable hotspots in nuclear medicine images. When the PSMA-binding agent is concentrated in multiple cancerous tissues and regions of the body that express PSMA, localized cancer within the patient's prostate and / or metastatic cancer in different regions of the patient's body can be detected and evaluated. As described below, risk indicators associated with overall patient survival and other prognostic metrics indicative of disease status, progression, treatment efficacy, and the like can be calculated based on automated analysis of intensity changes in nuclear medicine images obtained after administration of a PSMA-binding radiopharmaceutical to a patient.

[0495] A variety of radionuclide-labeled PSMA binding agents can be used as radiopharmaceutical imaging agents for nuclear medicine imaging to detect and evaluate prostate cancer. In certain embodiments, the specific radionuclide-labeled PSMA binding agent used depends on factors such as the specific imaging modality (e.g., PET; e.g., SPECT) and the specific area (e.g., organ) of the patient to be imaged. For example, certain radionuclide-labeled PSMA binding agents are suitable for PET imaging, while others are suitable for SPECT imaging. For example, certain radionuclide-labeled PSMA binding agents help image the patient's prostate and are primarily used when the disease is localized, while others help image organs and regions throughout the patient's body and are suitable for evaluating metastatic prostate cancer.

[0496] Various PSMA-binding agents and radionuclide-labeled versions thereof are described in US Pat. Nos. 8,778,305, 8,211,401, and 8,962,799, each of which is incorporated herein by reference in its entirety.

[0497] i. PET Imaging of Radionuclide-Labeled PSMA Binding Agents

[0498] In certain embodiments, the radionuclide-labeled PSMA-binding agent is a radionuclide-labeled PSMA-binding agent suitable for PET imaging.

[0499] In certain embodiments, radionuclide-labeled PSMA-binding agents include [18F]DCFPyL (also known as PyL TM ; also known as DCFPyL-18F):

[0500]

[0501] or a pharmaceutically acceptable salt thereof.

[0502] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises [18F]DCFBC:

[0503]

[0504] or a pharmaceutically acceptable salt thereof.

[0505] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 68 Ga-PSMA-HBED-CC (also known as 68 Ga-PSMA-11):

[0506]

[0507] or a pharmaceutically acceptable salt thereof.

[0508] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises PSMA-617:

[0509]

[0510] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 68 Ga-PSMA-617 (which is used 68 In certain embodiments, the PSMA-binding agent labeled with a radionuclide comprises 177 Lu-PSMA-617 (which is used 177 Lu-labeled PSMA-617) or a pharmaceutically acceptable salt thereof.

[0511] In certain embodiments, radionuclide-labeled PSMA-binding agents include PSMA-I&T:

[0512]

[0513] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 68 Ga-PSMA-I&T (which is used 68 Ga-labeled PSMA-I&T) or a pharmaceutically acceptable salt thereof.

[0514] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises PSMA-1007:

[0515]

[0516] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 18 F-PSMA-1007 (which is used 18 F-labeled PSMA-1007) or a pharmaceutically acceptable salt thereof.

[0517] ii. SPECT imaging of radionuclide-labeled PSMA binders

[0518] In certain embodiments, the radionuclide-labeled PSMA-binding agent is a radionuclide-labeled PSMA-binding agent suitable for SPECT imaging.

[0519] In certain embodiments, radionuclide-labeled PSMA-binding agents include 1404 (also known as MIP-1404):

[0520]

[0521] or a pharmaceutically acceptable salt thereof.

[0522] In certain embodiments, radionuclide-labeled PSMA-binding agents include 1405 (also known as MIP-1405):

[0523]

[0524] or a pharmaceutically acceptable salt thereof.

[0525] In certain embodiments, radionuclide-labeled PSMA-binding agents include 1427 (also known as MIP-1427):

[0526]

[0527] or a pharmaceutically acceptable salt thereof.

[0528] In certain embodiments, radionuclide-labeled PSMA-binding agents include 1428 (also known as MIP-1428):

[0529]

[0530] or a pharmaceutically acceptable salt thereof.

[0531] In certain embodiments, PSMA binding agents utilize radionuclides by chelating them to radioisotopes of metals [e.g., radioisotopes of technetium (Tc) (e.g., technetium 99m ( 99m Tc)); for example, radioisotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium 186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g., 90 Y); radioactive isotopes such as lutetium (Lu) (e.g., 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g., 111 In); for example, radioactive isotopes of copper (Cu) (e.g., 67 Cu)] to mark.

[0532] In certain embodiments, 1404 is labeled with a radionuclide (e.g., a radioisotope chelated to a metal). In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 99m Tc-MIP-1404, which is used for (e.g., chelated to) 99m Tc-labeled 1404:

[0533]

[0534] or a pharmaceutically acceptable salt thereof. In certain embodiments, 1404 can be chelated to other metal radioisotopes [e.g., radioisotopes of rhenium (Re) (e.g., rhenium 188 ( 188 Re); for example, rhenium 186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g., 90 Y); radioactive isotopes such as lutetium (Lu) (e.g., 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g., 111 In); for example, radioactive isotopes of copper (Cu) (e.g., 67 Cu)] to form a compound having the same properties as described above for 99m Tc-MIP-1404 exhibits a similar structure, in which another metal radioisotope is substituted 99m Tc.

[0535] In certain embodiments, 1405 is labeled with a radionuclide (e.g., a radioisotope chelated to a metal). In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 99m Tc-MIP-1405, which is used for (e.g., chelated to) 99m Tc-labeled 1405:

[0536]

[0537] or a pharmaceutically acceptable salt thereof. In certain embodiments, 1405 can be chelated to other metal radioisotopes [e.g., radioisotopes of rhenium (Re) (e.g., rhenium 188 ( 188 Re); for example, rhenium 186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g., 90 Y); radioactive isotopes such as lutetium (Lu) (e.g., 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g., 111 In); for example, radioactive isotopes of copper (Cu) (e.g., 67 Cu)] to form a compound having the same properties as described above for 99m Tc-MIP-1405 exhibits a similar structure, in which another metal radioisotope is substituted99m Tc.

[0538] In certain embodiments, 1427 is labeled with (eg, chelated to) a radioactive isotope of a metal to form a compound according to the following formula:

[0539]

[0540] or a pharmaceutically acceptable salt thereof, wherein M is a radioisotope of a metal labeled 1427 [e.g., a radioisotope of technetium (Tc) (e.g., technetium 99m ( 99m Tc)); for example, radioisotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium 186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g., 90 Y); radioactive isotopes such as lutetium (Lu) (e.g., 177 Lu); for example, radioactive isotopes of gallium (Ga) (e.g., 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g., 111 In); for example, radioactive isotopes of copper (Cu) (e.g., 67 Cu)].

[0541] In certain embodiments, 1428 is labeled with (eg, chelated to) a radioactive isotope of a metal to form a compound according to the formula:

[0542]

[0543] or a pharmaceutically acceptable salt thereof, wherein M is a radioisotope of a metal labeled 1428 [e.g., a radioisotope of technetium (Tc) (e.g., technetium 99m ( 99m Tc)); for example, radioisotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium 186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g., 90 Y); radioactive isotopes such as lutetium (Lu) (e.g., 177 Lu); for example, radioactive isotopes of gallium (Ga) (e.g., 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g., 111 In); for example, radioactive isotopes of copper (Cu) (e.g., 67 Cu)].

[0544] In certain embodiments, radionuclide-labeled PSMA-binding agents include PSMA I&S:

[0545]

[0546] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 99m Tc-PSMA I&S (which is used 99m Tc-labeled PSMA (I & S) or a pharmaceutically acceptable salt thereof.

[0547] E. Computer system and network architecture

[0548] like Figure 17 , an embodiment of a network environment 1700 for providing the systems, methods, and architectures described herein is shown and described. Figure 17 , a block diagram of an exemplary cloud computing environment 1700 is shown and described. Cloud computing environment 1700 may include one or more resource providers 1702a, 1702b, 1702c (generally, 1702). Each resource provider 1702 may include computing resources. In some embodiments, computing resources may include any hardware and / or software for processing data. For example, computing resources may include hardware and / or software capable of executing algorithms, computer programs, and / or computer applications. In some embodiments, exemplary computing resources may include application servers and / or databases with storage and retrieval capabilities. Each resource provider 1702 may be connected to any other resource provider 1702 in cloud computing environment 1700. In some embodiments, resource providers 1702 may be connected via a computer network 1708. Each resource provider 1702 may be connected to one or more computing devices 1704a, 1704b, 1704c (generally, 1704) via computer network 1708.

[0549] The cloud computing environment 1700 may include a resource manager 1706. The resource manager 1706 may connect to resource providers 1702 and computing devices 1704 via a computer network 1708. In some embodiments, the resource manager 1706 may facilitate the provision of computing resources to one or more computing devices 1704 through one or more resource providers 1702. The resource manager 1706 may receive a request for computing resources from a particular computing device 1704. The resource manager 1706 may identify one or more resource providers 1702 that can provide the computing resources requested by the computing device 1704. The resource manager 1706 may select a resource provider 1702 to provide the computing resources. The resource manager 1706 may facilitate connections between resource providers 1702 and a particular computing device 1704. In some embodiments, the resource manager 1706 may establish a connection between a particular resource provider 1702 and a particular computing device 1704. In some implementations, the resource manager 1706 may redirect a particular computing device 1704 to a particular resource provider 1702 that has the requested computing resources.

[0550] Figure 18 Examples of computing device 1800 and mobile computing device 1850 that can be used to implement the techniques described in this disclosure are shown. Computing device 1800 is intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Mobile computing device 1850 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. The components shown here, their connections and relationships, and their functions are intended only to be examples and are not intended to be limiting.

[0551] Computing device 1800 includes a processor 1802, a memory 1804, a storage device 1806, a high-speed interface 1808 connected to the memory 1804 and a plurality of high-speed expansion ports 1810, and a low-speed interface 1812 connected to a low-speed expansion port 1814 and the storage device 1806. Each of the processor 1802, the memory 1804, the storage device 1806, the high-speed interface 1808, the high-speed expansion port 1810, and the low-speed interface 1812 is interconnected using various buses and can be mounted on a common motherboard or otherwise as desired. The processor 1802 can process instructions for execution within the computing device 1800, including instructions stored in the memory 1804 or on the storage device 1806, to display graphical information for a GUI on an external input / output device, such as a display 1816 coupled to the high-speed interface 1808. In other embodiments, multiple processors and / or multiple buses can be used, along with multiple memories and memory types, as desired. Additionally, multiple computing devices may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system). Thus, when terminology is used herein, where multiple functions are described as being performed by a "processor," this encompasses embodiments where the multiple functions are performed by any number of processor(s) of any number of computing devices(s). Furthermore, when a function is described as being performed by a "processor," this encompasses embodiments where the function is performed by any number of processor(s) of any number of computing devices(s) (e.g., in a distributed computing system).

[0552] Memory 1804 stores information within computing device 1800. In some implementations, memory 1804 is one or more volatile memory units. In some implementations, memory 1804 is one or more non-volatile memory units. Memory 1804 can also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0553] Storage device 1806 is capable of providing mass storage for computing device 1800. In some embodiments, storage device 1806 may be or contain a computer-readable medium, such as a floppy disk drive, a hard disk drive, an optical disk drive, or a magnetic tape drive, a flash memory or other similar solid-state memory device or array of devices, including devices in the form of a storage network, a local area network, or other configurations. Instructions may be stored in an information carrier. When executed by one or more processing devices (e.g., processor 1802), the instructions perform one or more methods, such as those described above. Instructions may also be stored by one or more storage devices (e.g., computer-readable media or machine-readable media, such as memory 1804, storage device 1806, or memory on processor 1802).

[0554] The high-speed interface 1808 manages bandwidth-intensive operations for the computing device 1800, while the low-speed interface 1812 manages less bandwidth-intensive operations. The arrangement of such functions is merely an example. In some implementations, the high-speed interface 1808 is coupled to the memory 1804, the display 1816 (e.g., via a graphics processor or accelerator), and the high-speed expansion ports 1810, which can accommodate various expansion cards (not shown). In an implementation, the low-speed interface 1812 is coupled to the storage device 1806 and the low-speed expansion ports 1814. Various communication ports (e.g., USB, The low-speed expansion port 1814 (network Ethernet, wireless network Ethernet) can be coupled to one or more input / output devices, such as a keyboard, pointing device, scanner, or network connection device (such as a switch or router), for example, through a network adapter.

[0555] Computing device 1800 can be implemented in a variety of different forms, as shown in the figure. For example, it can be implemented as a standard server 1820 or multiple times as a group of such servers. In addition, it can be implemented in the form of a personal computer (such as laptop computer 1822). It can also be implemented as part of framework server system 1824. Alternatively, components from computing device 1800 can be combined with other components (not shown) in a mobile device such as mobile computing device 1850. Each of these devices can contain one or more of computing device 1800 and mobile computing device 1850, and the entire system can be composed of multiple computing devices communicating with each other.

[0556] The mobile computing device 1850 includes a processor 1852, a memory 1864, an input / output device (e.g., a display 1854), a communication interface 1866, and a transceiver 1868, among other components. The mobile computing device 1850 may also be provided with a storage device (e.g., a microdrive or other device) to provide additional storage. Each of the processor 1852, the memory 1864, the display 1854, the communication interface 1866, and the transceiver 1868 is interconnected using various buses, and several of the components may be mounted on a common motherboard or otherwise as desired.

[0557] The processor 1852 can execute instructions within the mobile computing device 1850, including instructions stored in the memory 1864. The processor 1852 can be implemented as a chipset including independent and multiple analog and digital processors. The processor 1852 can provide a cooperative relationship with other components of the mobile computing device 1850, such as user interface controls, applications run by the mobile computing device 1850, and wireless communications using the mobile computing device 1850.

[0558] The processor 1852 can communicate with the user via a control interface 1858 and a display interface 1856 coupled to a display 1854. The display 1854 can be, for example, a TFT (Thin Film Transistor Liquid Crystal Display) display or an Organic Light Emitting Diode (OLED) display or other suitable display technology. The display interface 1856 may include appropriate circuitry for driving the display 1854 to present graphics and other information to the user. The control interface 1858 can receive commands from the user and convert them for presentation to the processor 1852. In addition, an external interface 1862 can provide communication with the processor 1852 to enable nearby area communication between the mobile computing device 1850 and other devices. The external interface 1862 can provide, for example, wired communication in some embodiments, or wireless communication in other embodiments, and multiple interfaces can also be used.

[0559] Memory 1864 stores information within mobile computing device 1850. Memory 1864 may be implemented as one or more of the following: one or more computer-readable media or media, one or more volatile memory units, or one or more non-volatile memory units. Expansion memory 1874 may also be provided and connected to mobile computing device 1850 via expansion interface 1872, which may include, for example, a Single In Line Memory Module (SIMM) card interface. Expansion memory 1874 may provide additional storage space for mobile computing device 1850 or may also store applications or other information for mobile computing device 1850. Specifically, expansion memory 1874 may include instructions to perform or supplement the processes described above and may also include security information. Thus, for example, expansion memory 1874 may be provided as a security module for mobile computing device 1850 and may be programmed with instructions that permit secure use of mobile computing device 1850. Additionally, secure applications and additional information may be provided via the SIMM card, such as placing authentication information on the SIMM card in a non-invasive manner.

[0560] The memory may include, for example, flash memory and / or NVRAM memory (non-volatile random access memory), as discussed below. In some embodiments, the instructions are stored in an information carrier. When the instructions are executed by one or more processing devices (e.g., processor 1852), one or more methods are implemented, such as those described above. The instructions may also be stored by one or more storage devices, such as one or more computer-readable media or machine-readable media (e.g., memory 1864, expansion memory 1874, or memory on processor 1852). In some embodiments, the instructions may be received in the form of a propagated signal, such as via transceiver 1868 or external interface 1862.

[0561] The mobile computing device 1850 can communicate wirelessly via the communication interface 1866, which may include digital signal processing circuitry if necessary. The communication interface 1866 can provide communications in various modes or protocols, such as GSM voice calls (Global System for Mobile Communications), Short Message Service (SMS), Enhanced Messaging Service (EMS) or MMS messaging (Multimedia Messaging Service), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Personal Digital Cellular (PDC), Wideband Code Division Multiple Access (WCDMA), CDMA2000, or General Packet Radio Service (GPRS), among others. These communications can be carried out, for example, using radio frequencies via the transceiver 1868. Additionally, the communication can be carried out, for example, using Wi-Fi TM or other such transceiver (not shown) for short-range communications. Additionally, a Global Positioning System (GPS) receiver module 1870 can provide additional navigation-related and location-related wireless data to the mobile computing device 1850, which can be used as needed by applications running on the mobile computing device 1850.

[0562] The mobile computing device 1850 may also communicate vocally using an audio codec 1860, which may receive voice information from a user and convert it into usable digital information. The audio codec 1860 may likewise produce audible sounds to the user, such as via a speaker, such as in an earpiece of the mobile computing device 1850. Such sounds may include sounds from voice telephone calls, may include recorded sounds (e.g., voice messages, music files, etc.), and may also include sounds produced by applications operating on the mobile computing device 1850.

[0563] The mobile computing device 1850 can be implemented in a variety of different forms, as shown in the figures. For example, it can be implemented as a cellular telephone 1880. It can also be implemented as part of a smart phone 1882, a personal digital assistant, or other similar mobile device.

[0564] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that are executable and / or interpretable on a programmable system that includes at least one programmable processor, which can be special purpose or general purpose, coupled to receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0565] These computer programs (otherwise known as programs, software, software applications, or program code) contain machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions in the form of machine-readable signals. The term machine-readable signal refers to any signal for providing machine instructions and / or data to a programmable processor.

[0566] To provide for user interaction, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide for user interaction; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, voice, or tactile input.

[0567] The systems and techniques described herein can be implemented in a computing system that includes a back-end component (e.g., as a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of these back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0568] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client-server relationship arises by virtue of computer programs running on the respective computers and having a master-slave relationship to each other.

[0569] In some implementations, the various modules described herein may be separated, combined, or incorporated into a single or combined module.The modules depicted in the figures are not intended to limit the systems described herein to the software architecture shown therein.

[0570] The elements of the various embodiments described herein may be combined to form other embodiments not specifically described above. Elements may be removed from the processes, computer programs, databases, etc. described herein without adversely affecting their operation. In addition, the logic flows depicted in the figures do not require the specific order shown or sequential order to achieve the desired results. Various independent elements may be combined into one or more individual elements to perform the functions described herein.

[0571] Throughout the description, where apparatus and systems are described as having, containing, or comprising particular components, or where processes and methods are described as having, containing, or comprising particular steps, it is intended that there are apparatus and systems of the invention that consist essentially of, or consist of, the described components, and that there are processes and methods of the invention that consist essentially of, or consist of, the described processing steps.

[0572] Should be understood that the order of steps or the order in which a certain action is performed is not important, as long as the present invention remains operable.In addition, two or more than two steps or actions can be performed simultaneously.

[0573] Unless technically incompatible, the various described embodiments of the present invention can be used in conjunction with one or more other embodiments.

[0574] While the present invention has been particularly shown and described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined in the following claims.

Claims

1. A method for automatically processing a 3D image to identify a 3D volume corresponding to a specific target tissue region within the 3D image, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of an individual acquired using an anatomical imaging modality, wherein the 3D anatomical image comprises a graphical representation of tissue within the individual; (b) for each of a plurality of target tissue regions, automatically identifying, by the processor, using one or more machine learning modules, a corresponding target volume of interest (VOI) within the 3D anatomical image; (c) determining, by the processor, a 3D segmentation map representing a plurality of 3D segmentation masks, each 3D segmentation mask representing a specifically identified target VOI; (d) storing and / or providing the 3D segmented image for display and / or further processing; (e) receiving, by the processor, a 3D functional image of the individual obtained using a functional imaging modality; (f) identifying, by the processor, using the 3D segmentation map, within the 3D functional image, one or more 3D volumes each corresponding to the identified target VOI; and (g) automatically detecting, by the processor, one or more hot spots within the 3D functional image, the hot spots being determined to represent lesions based on the intensity of voxels within the 3D functional image; wherein the target tissue region comprises one or more reference tissue regions and wherein the method comprises: using the 3D segmentation map to identify, by the processor, within the 3D functional image, one or more 3D reference volumes, each corresponding to a specific reference tissue region; determining, by the processor, one or more reference intensity values, each associated with a particular 3D reference volume of the one or more 3D reference volumes and corresponding to an intensity measurement within the particular 3D reference volume; determining, by the processor, one or more individual hotspot intensity values, each associated with and corresponding to an intensity measurement of a particular hotspot among at least a portion of the detected one or more hotspots; and One or more individual hotspot index values ​​are determined, by the processor, using the one or more individual hotspot intensity values ​​and the one or more reference intensity values. The method according to claim 1 , wherein the 3D anatomical image is a whole body image.

3. The method of claim 1 or 2, wherein step (c) comprises digitally stitching the plurality of 3D segmentation masks together to form the 3D segmentation map.

4. The method according to claim 1 or 2, wherein for at least one specific target tissue area, step (b) comprises: Determine, using a first module, an initial VOI within the 3D anatomical image, the initial VOI corresponding to an anatomical region containing the specific target tissue region; and The target VOI within the initial VOI corresponding to the specific target tissue region is identified using a second module.

5. The method of claim 4, wherein the second module is a CNN module implementing a CNN.

6. The method of claim 4, wherein the first module is a CNN module that implements a CNN to perform coarse segmentation to automatically identify the initial VOI corresponding to the anatomical region containing the specific target tissue region.

7. The method of claim 4, wherein the first module receives as input a subsampled version of the anatomical image and identifies the initial VOI using the subsampled version of the anatomical image.

8. The method of claim 4, wherein the first module is a first CNN module and the second module is a second CNN module, and wherein the first CNN module includes an additional filter to account for a rate of change as image size increases relative to the second CNN module.

9. The method of claim 4, wherein the 3D anatomical image is a whole body image and wherein step (b) comprises: automatically determining, using one or more localization modules implementing machine learning techniques, a plurality of initial VOIs within the 3D anatomical image, each initial VOI corresponding to a particular anatomical region and within which a relevant subset of the target VOIs are located; and For each initial VOI, one or more segmentation modules implementing machine learning techniques are used to automatically identify relevant subsets of the target VOI.

10. The method of claim 9, wherein the plurality of initial VOIs comprises one or more members selected from the group consisting of: an initial VOI of a pelvic region corresponding to a pelvic region of the individual; an initial VOI of a spinal region corresponding to a spinal column of the individual; a left upper body region initial VOI corresponding to the left side of the upper body of the individual; and The initial VOI corresponds to the right upper body region of the individual's upper body.

11. The method according to claim 1, comprising: (h) determining, by the processor, the cancer status of the individual.

12. The method of claim 11, comprising repeatedly performing steps (a) to (h) on a plurality of anatomical and corresponding functional images acquired at different time points to determine the cancer status of the individual at each time point, thereby tracking the cancer status over time.

13. The method according to claim 1, comprising: (i) automatically adjusting, by the processor, the volumetric pixel intensity of the 3D functional image to correct for background absorption in one or more background tissue regions.

14. The method of claim 13, wherein the target tissue region includes the one or more background tissue regions and wherein step (i) comprises: using the 3D segmentation map to identify one or more 3D background tissue volumes within the 3D functional image, each corresponding to a specific background tissue region; and The voxel intensity of the 3D functional image is adjusted based on the voxel intensity within the 3D background tissue volume.

15. The method of claim 13, wherein the one or more background tissue regions include one or more members selected from the group consisting of: bladder, kidney, duodenum, small intestine, spleen, liver, pancreas, stomach, adrenal gland, rectum, and testicle.

16. The method of claim 1, wherein step (g) comprises using one or more thresholds.

17. The method of claim 1, wherein step (g) comprises applying one or more filters to the 3D functional image.

18. The method of claim 1, wherein step (g) comprises using a combination of two or more techniques to identify hot spots in different subsets of the one or more 3D volumes identified within the 3D functional image.

19. The method of claim 1, wherein step (g) comprises detecting an initial set of hotspots, and for at least a portion of the hotspots of the initial set, classifying each hotspot in at least a portion of the detected hotspots as a cancerous lesion or a non-cancerous lesion.

20. The method of claim 1, wherein the target tissue region includes one or more background tissue regions and wherein the method comprises: using the 3D segmentation map to identify one or more 3D background tissue volumes within the 3D functional image, each corresponding to a specific background tissue region; and Voxels within the 3D background tissue are excluded from the voxels used for automatically detecting the one or more hot spots at step (g).

21. The method of claim 1, comprising using the one or more detected hot spots to determine a cancer status of the individual.

22. A method according to claim 1, wherein determining the one or more individual hotspot index values ​​includes mapping each of the one or more reference intensity values ​​to a corresponding reference index value in a certain proportion, and for each individual hotspot intensity value, using the reference intensity value and the corresponding reference index value to interpolate the corresponding individual hotspot index value.

23. The method of claim 1, wherein the reference tissue region comprises one or more members selected from the group consisting of: liver, aorta, and parotid gland.

24. The method of claim 1, wherein: The first reference intensity value (I) is a blood reference intensity value associated with a 3D reference volume corresponding to a portion of the aorta, and (II) is mapped to a first reference index value; The second reference intensity value (I) is a liver reference intensity value associated with a 3D reference volume corresponding to the liver, and (II) is mapped to a second reference index value; and The second reference intensity value is greater than the first reference intensity value and the second reference index value is greater than the first reference index value.

25. The method of claim 1, wherein the reference intensity values ​​comprise a maximum reference intensity value mapped to a maximum reference index value, and wherein hotspots having associated hotspot intensity values ​​greater than the maximum reference intensity value are assigned a hotspot index value equal to the maximum reference index value.

26. The method of claim 1, comprising determining, by the processor, an overall index value indicative of a cancer status of the individual using at least a portion of the one or more hotspot index values.

27. The method of claim 26, wherein the overall index value is determined as a weighted sum of at least a portion of the individual hotspot index values.

28. The method of claim 26, wherein: The overall index value is associated with a specific target tissue region corresponding to a specific target VOI identified within the anatomical image; and The overall index value is determined using hotspot index values ​​of a subset of hotspots located within a particular 3D volume in the 3D functional image corresponding to the particular identified target VOI.

29. The method of claim 28, wherein the specific target tissue region is selected from the group consisting of: a skeletal region including one or more bones of the individual, a lymphatic region, and a prostate region.

30. The method of claim 1, wherein: The 3D anatomical image is an x-ray computed tomography (CT) image, and The 3D functional image is a 3D single photon emission computed tomography (SPECT) image.

31. The method of claim 1, wherein: The 3D anatomical image is an x-ray computed tomography (CT) image, and The 3D functional image is a 3D positron emission tomography (PET) image.

32. The method of claim 31, wherein the 3D positron emission tomography image of the subject is obtained after administering a radiopharmaceutical comprising a prostate specific membrane antigen (PSMA) binding agent to the subject.

33. The method of claim 32, wherein the radiopharmaceutical comprises [ 18 F]DCFPyL.

34. The method of claim 1 or 2, wherein step (b) comprises cropping, by the processor, the 3D anatomical image to remove voxels representing air.

35. The method of claim 1 or 2, wherein the target tissue region comprises one or more members selected from the group consisting of: left hip, right hip, sacrum and coccyx region, left clavicle, right clavicle, left rib, right rib, left scapula, right scapula, sternum, lumbar vertebrae, thoracic vertebrae, skull, cervical vertebrae, left femur, right femur, left humerus, right humerus, prostate, bladder, rectum, left gluteus maximus, right gluteus maximus, aorta, left kidney, right kidney, liver, left lung, right lung, spleen, ventricle, left adrenal gland, right adrenal gland, gall bladder, brain, pancreas, heart, mandible, left bronchus, right bronchus, trachea, left common iliac artery, right common iliac artery and parotid gland.

36. A method according to claim 1 or 2, wherein one or more of the target tissue areas include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left femur, right femur, left fibula, right fibula, left hip, right hip, left humerus, right humerus, mandible, left patella, right patella, left radius, right radius, left tibia, right tibia, left ulna, right ulna, left rib, right rib, sacrum and coccyx, left scapula, right scapula, skull, sternum, spinal regions and individual vertebrae.

37. The method of claim 36, wherein one or more of the target tissue areas comprises one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left hip, right hip, left rib, right rib, sacrum and coccyx, left scapula, right scapula, sternum, vertebral regions, and individual vertebrae.

38. The method of claim 1 or 2, wherein one or more of the target tissue regions comprises one or more soft tissue regions selected from the group consisting of: left adrenal gland, right adrenal gland, aorta, brain, left bronchus, right bronchus, gallbladder, left gluteus maximus, right gluteus maximus, heart, left common iliac artery, right common iliac artery, left kidney, right kidney, liver, left lung, right lung, pancreas, prostate, rectum, spleen, trachea, bladder, ventricle, and parotid gland.

39. The method of claim 38, wherein the one or more target tissue regions comprise one or more soft tissue regions selected from the group consisting of: gallbladder, left kidney, right kidney, liver, left lung, right lung, prostate, and bladder.

40. The method of claim 1 or 2, wherein the target tissue region comprises one or more bone regions each corresponding to a specific bone and one or more soft tissue regions each corresponding to a specific soft tissue region.

41. A system for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a specific target tissue region, the system comprising: a processor of a computing device; and a memory having stored thereon instructions, wherein the instructions, when executed by the processor, cause the processor to: (a) receiving a 3D anatomical image of an individual acquired using an anatomical imaging modality, wherein the 3D anatomical image comprises a graphical representation of tissue within the individual; (b) for each of a plurality of target tissue regions, automatically identifying, using one or more machine learning modules, a corresponding target volume of interest (VOI) within the 3D anatomical image; (c) determining a 3D segmentation map representing a plurality of 3D segmentation masks, each 3D segmentation mask representing a specific identified target VOI; (d) storing and / or providing the 3D segmented image for display and / or further processing; (e) receiving a 3D functional image of the individual obtained using a functional imaging modality; and (f) identifying one or more 3D volumes within the 3D functional image using the 3D segmentation map, each of which corresponds to the identified target VOI; (g) automatically detecting one or more hot spots within the 3D functional image, the hot spots being determined to represent lesions based on the intensity of voxels within the 3D functional image; wherein the target tissue region comprises one or more reference tissue regions and wherein the instructions cause the processor to: using the 3D segmentation map to identify within the 3D functional image one or more 3D reference volumes, each corresponding to a specific reference tissue region; determining one or more reference intensity values, each of which is associated with a particular 3D reference volume of the one or more 3D reference volumes and corresponds to an intensity measurement within the particular 3D reference volume; and determining one or more individual hotspot intensity values, each of which is associated with and corresponds to an intensity measurement value of a particular hotspot among at least a portion of the detected one or more hotspots; and One or more individual hotspot index values ​​are determined using the one or more individual hotspot intensity values ​​and the one or more reference intensity values.

42. The system of claim 41, wherein the 3D anatomical image is a full body image.

43. The system of claim 41 or 42, wherein at step (c), the instructions cause the processor to digitally stitch the plurality of 3D segmentation masks together to form the 3D segmentation map.

44. The system of claim 41 or 42, wherein at step (b), for at least one specific target tissue region, the instructions cause the processor to: Determining, using a first module, an initial VOI within the 3D anatomical image, the initial VOI corresponding to an anatomical region containing the specific target tissue region; and The target VOI within the initial VOI corresponding to the specific target tissue region is identified using a second module.

45. The system of claim 44, wherein the second module is a CNN module implementing a CNN.

46. ​​The system of claim 44, wherein the first module is a CNN module that implements a CNN to perform coarse segmentation to automatically identify the initial VOI corresponding to the anatomical region containing the specific target tissue region.

47. The system of claim 44, wherein the first module receives as input a subsampled version of the anatomical image and identifies the initial VOI using the subsampled version of the anatomical image.

48. The system of claim 44, wherein the first module is a first CNN module and the second module is a second CNN module, and wherein the first CNN module includes additional filters to account for a rate of change as image size increases relative to the second CNN module.

49. The system of claim 44, wherein the 3D anatomical image is a full body image, and wherein at step (b), the instructions cause the processor to: automatically determining, using one or more localization modules implementing machine learning techniques, a plurality of initial VOIs within the 3D anatomical image, each initial VOI corresponding to a particular anatomical region and within which a relevant subset of the target VOIs are located; and For each initial VOI, one or more segmentation modules implementing machine learning techniques are used to automatically identify relevant subsets of the target VOI.

50. The system of claim 49, wherein the plurality of initial VOIs comprises one or more members selected from the group consisting of: an initial VOI of a pelvic region corresponding to a pelvic region of the individual; an initial VOI of a spinal region corresponding to a spinal column of the individual; a left upper body region initial VOI corresponding to the left side of the upper body of the individual; and The initial VOI corresponds to the right upper body region of the individual's upper body.

51. The system of claim 41 , wherein the instructions cause the processor to: (h) determining the cancer status of the individual.

52. A system according to claim 51, wherein the instructions cause the processor to repeatedly perform steps (a) to (h) on multiple anatomical and corresponding functional images acquired at different time points to determine the cancer status of the individual at each time point, thereby tracking the cancer status over time.

53. The system of claim 41, wherein the instructions cause the processor to: (i) automatically adjusting the volumetric pixel intensity of the 3D functional image to correct for background absorption in one or more background tissue regions.

54. The system of claim 53, wherein the target tissue region includes the one or more background tissue regions, and wherein at step (i), the instructions cause the processor to: using the 3D segmentation map to identify within the 3D functional image one or more 3D background tissue volumes, each corresponding to a specific background tissue region; and The voxel intensity of the 3D functional image is adjusted based on the voxel intensity within the 3D background tissue volume.

55. The system of claim 53, wherein the one or more background tissue regions include one or more members selected from the group consisting of: bladder, kidney, duodenum, small intestine, spleen, liver, pancreas, stomach, adrenal gland, rectum, and testicle.

56. The system of claim 41, wherein at step (g), the instructions cause the processor to use one or more thresholds.

57. The system of claim 41, wherein at step (g), the instructions cause the processor to apply one or more filters to the 3D functional image.

58. The system of claim 41, wherein at step (g), the instructions cause the processor to use a combination of two or more techniques to identify hot spots in different subsets of the one or more 3D volumes identified within the 3D functional image.

59. A system according to claim 41, wherein at step (g), the instructions cause the processor to detect an initial set of hotspots, and for at least a portion of the hotspots of the initial set, classify each hotspot in at least a portion of the detected hotspots as a cancerous lesion or a non-cancerous lesion.

60. The system of claim 41, wherein the target tissue region includes one or more background tissue regions and wherein the instructions cause the processor to: using the 3D segmentation map to identify within the 3D functional image one or more 3D background tissue volumes, each corresponding to a specific background tissue region; and Voxels within the 3D background tissue are excluded from the voxels used for automatically detecting the one or more hot spots at step (g).

61. The system of claim 41, wherein the instructions cause the processor to use the one or more detected hot spots to determine a cancer status of the individual.

62. A system according to claim 41, wherein the instructions cause the processor to determine the one or more individual hotspot indicator values ​​by mapping each of the one or more reference intensity values ​​to a corresponding reference index value in a certain proportion, and for each individual hotspot intensity value, using the reference intensity value and the corresponding reference index value to interpolate the corresponding individual hotspot indicator value.

63. The system of claim 41, wherein the reference tissue region comprises one or more members selected from the group consisting of: liver, aorta, and parotid gland.

64. The system of claim 41, wherein: The first reference intensity value (I) is a blood reference intensity value associated with a 3D reference volume corresponding to a portion of the aorta, and (II) is mapped to a first reference index value; The second reference intensity value (I) is a liver reference intensity value associated with a 3D reference volume corresponding to the liver, and (II) is mapped to a second reference index value; and The second reference intensity value is greater than the first reference intensity value and the second reference index value is greater than the first reference index value.

65. The system of claim 41, wherein the reference intensity values ​​comprise a maximum reference intensity value mapped to a maximum reference index value, and wherein hotspots having associated hotspot intensity values ​​greater than the maximum reference intensity value are assigned hotspot index values ​​equal to the maximum reference index value.

66. The system of claim 41, wherein the instructions cause the processor to determine an overall index value indicative of a cancer status of the individual using at least a portion of the one or more hotspot index values.

67. The system of claim 66, wherein the overall index value is determined as a weighted sum of at least a portion of the individual hotspot index values.

68. The system of claim 66, wherein: The overall index value is associated with a specific target tissue region corresponding to a specific target VOI identified within the anatomical image; and The overall index value is determined using hotspot index values ​​of a subset of hotspots located within a particular 3D volume in the 3D functional image corresponding to the particular identified target VOI.

69. The system of claim 68, wherein the specific target tissue region is selected from the group consisting of: a skeletal region including one or more bones of the individual, a lymphatic region, and a prostate region.

70. The system of claim 41, wherein: The 3D anatomical image is an x-ray computed tomography (CT) image, and The 3D functional image is a 3D single photon emission computed tomography (SPECT) image.

71. The system of claim 41, wherein: The 3D anatomical image is an x-ray computed tomography (CT) image, and The 3D functional image is a 3D positron emission tomography (PET) image.

72. The system of claim 71, wherein the 3D positron emission tomography image of the subject is obtained after administration of a radiopharmaceutical comprising a prostate specific membrane antigen (PSMA) binding agent to the subject.

73. The system of claim 72, wherein the radiopharmaceutical comprises [ 18 F]DCFPyL.

74. The system of claim 41, wherein at step (b), the instructions cause the processor to crop the 3D anatomical image to remove voxels representing air.

75. The system of claim 41 or 42, wherein the target tissue region comprises one or more members selected from the group consisting of: left hip, right hip, sacrum and coccyx region, left clavicle, right clavicle, left rib, right rib, left scapula, right scapula, sternum, lumbar vertebrae, thoracic vertebrae, skull, cervical vertebrae, left femur, right femur, left humerus, right humerus, prostate, bladder, rectum, left gluteus maximus, right gluteus maximus, aorta, left kidney, right kidney, liver, left lung, right lung, spleen, ventricle, left adrenal gland, right adrenal gland, gall bladder, brain, pancreas, heart, mandible, left bronchus, right bronchus, trachea, left common iliac artery, right common iliac artery and parotid gland.

76. A system according to claim 41 or 42, wherein one or more of the target tissue areas include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left femur, right femur, left fibula, right fibula, left hip, right hip, left humerus, right humerus, mandible, left patella, right patella, left radius, right radius, left tibia, right tibia, left ulna, right ulna, left rib, right rib, sacrum and coccyx, left scapula, right scapula, skull, sternum, vertebral regions and individual vertebrae.

77. A system according to claim 76, wherein one or more of the target tissue areas include one or more specific bones selected from the group consisting of: left clavicle, right clavicle, left hip bone, right hip bone, left rib, right rib, sacrum and coccyx, left scapula, right scapula, sternum, spinal regions and individual vertebrae.

78. The system of claim 41 or 42, wherein one or more of the target tissue regions comprises one or more soft tissue regions selected from the group consisting of: left adrenal gland, right adrenal gland, aorta, brain, left bronchus, right bronchus, gallbladder, left gluteus maximus, right gluteus maximus, heart, left common iliac artery, right common iliac artery, left kidney, right kidney, liver, left lung, right lung, pancreas, prostate, rectum, spleen, trachea, bladder, ventricle, and parotid gland.

79. The system of claim 78, wherein the one or more target tissue regions comprise one or more soft tissue regions selected from the group consisting of: gallbladder, left kidney, right kidney, liver, left lung, right lung, prostate, and bladder.

80. The system of claim 41 or 42, wherein the target tissue region comprises one or more bone regions each corresponding to a specific bone.

Images