System and method for fast segmentation of images and determination of radioactive drug absorption based on neural network

Abstract

Provided herein are systems and methods for automatically analyzing three-dimensional 3D medical images of an individual to automatically identify specific 3D volumes within the 3D images corresponding to specific organs and / or tissues. In certain embodiments, accurate identification of one or more of the volumes can be used to determine quantitative indices measuring the uptake of radiopharmaceuticals in specific organs and / or tissue regions. These uptake indices can be used to evaluate the disease state of the individual, determine the prognosis of the individual, and / or determine the efficacy of a treatment modality.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 614,935, filed on January 8, 2018, U.S. Patent Application No. 16 / 003,006, filed on June 7, 2018, and U.S. Provisional Application No. 62 / 749,574, filed on October 23, 2018, the contents of each of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present invention generally relates to methods, systems and architectures for automated analysis and / or presentation of medical image data. More specifically, in certain embodiments, the present invention relates to automated identification of one or more specific regions of interest (e.g., corresponding to specific organs or tissues) within an image of an individual, and determination of the amount of radiopharmaceutical uptake within said regions, e.g., for identification and / or staging of a disease (e.g., prostate cancer). Background Art

[0004] Targeted image analysis involves the use of radiolabeled small molecules that bind to specific receptors, enzymes, and proteins that change during the evolution of the disease in the body. After administration to the patient, these molecules circulate in the blood until they find their intended target. The bound radiopharmaceutical remains at the site of the disease, while the rest of the agent is cleared from the body. The radioactive portion of the molecule is used as a beacon so that images depicting the location and concentration of the disease can be obtained using commonly available nuclear medicine cameras (called single photon emission computerized tomography (SPECT) or positron emission tomography (PET) cameras) found in most hospitals around the world. This information can then be used by physicians to determine the presence and extent of the disease in the patient. Physicians can use this information to provide patients with recommended courses of treatment and track the progression of the disease.

[0005] There are a variety of software-based analysis techniques available for analyzing and enhancing PET and SPECT images that can be used by radiologists or physicians. There are also a variety of radiopharmaceuticals available for imaging specific types of cancer. For example, the small molecule diagnostic 1404 targets the extracellular domain of prostate-specific membrane antigen (PSMA), a protein amplified on the surface of >95% of prostate cancer cells and a validated target for detecting primary and metastatic prostate cancer. 1404 is labeled with technetium-99m, a gamma-emitting isotope that is widely available, relatively inexpensive, facilitates efficient preparation, and has spectral characteristics that are attractive for nuclear medicine imaging applications.

[0006] Another example of a radiopharmaceutical is PyL TM (Also known as 18F]DCFPyL), a fluorinated PSMA-targeted PET imaging agent for clinical-stage prostate cancer. A proof-of-concept study published in the April 2015 issue of the Journal of Molecular Imaging and Biology demonstrated that PyL TM PET imaging demonstrated high levels of PyL at sites of putative metastatic disease and primary tumors TM absorbance, which suggests the potential for detecting prostate cancer with high sensitivity and specificity.

[0007] An oncologist may use images from a targeted PET or SPECT study of a patient as input to his or her evaluation of whether the patient has a particular disease (e.g., prostate cancer), what stage of the disease is evident, what course of treatment is recommended (if any), whether surgical intervention is indicated, and the likely prognosis. The oncologist may use a radiologist report in this evaluation. A radiologist report is a technical evaluation of the PET or SPECT images prepared by a radiologist for the physician requesting the imaging study, and includes, for example, the type of study performed, the medical history, comparisons between the images, the techniques used to perform the study, the radiologist's observations and findings, and the overall impression and recommendations that the radiologist may have based on the results of the imaging study. The signed radiologist report is sent to the physician who scheduled the study for the physician's review, followed by a discussion between the physician and the patient regarding the treatment results and recommendations.

[0008] Thus, the process involves having a radiologist perform an imaging study on the patient, analyzing the images obtained, creating a radiologist report, forwarding the report to the requesting physician, having the physician formulate an evaluation and treatment recommendation, and having the physician communicate the results, recommendations, and risks to the patient. The process may also involve repeating the imaging study due to inconclusive results, or ordering further testing based on the initial results.

[0009] If the imaging studies show that the patient has a specific disease or condition (eg, cancer), the physician discusses various treatment options, including surgery, as well as the risks of doing nothing or taking a watchful waiting or active surveillance approach without surgery.

[0010] There are limitations associated with this process from both the physician's perspective and from the patient's perspective. Although the radiologist's report is certainly helpful, the physician ultimately must rely on his or her experience in developing an evaluation and recommendation for the patient. In addition, the patient must place a great deal of trust in his or her physician. The physician can show the patient his or her PET / SPECT images and can tell the patient the numerical risks associated with various treatment options or the likelihood of a particular prognosis, but it may be difficult for the patient to understand this information. In addition, the patient's family may have questions, especially if cancer is diagnosed but the patient chooses not to have surgery. The patient and / or their family members may search online for additional information and may receive unreliable information about the risks of the diagnosed condition. A difficult ordeal may become even more traumatic.

[0011]

[0006] Thus, there remains a need for systems and methods for improved analysis of medical imaging studies and communication of those results, diagnoses, prognoses, treatment recommendations, and associated risks to patients. Summary of the invention

[0012] Provided herein are systems and methods for automatically analyzing a three-dimensional (3D) medical image of an individual to automatically identify a specific 3D volume within the 3D image that corresponds to a specific organ and / or tissue. In certain embodiments, accurate identification of one or more of the volumes is used to automatically determine a quantitative index representing the amount of uptake of a radiopharmaceutical in a specific organ and / or tissue region. These uptake indices can be used to evaluate the disease state of an individual, determine the prognosis of an individual, and / or determine the efficacy of a treatment modality.

[0013] Significantly, by identifying 3D volumes in medical images and determining absorption indices in an automated manner, the systems and methods described herein open the door to streamlined medical image analysis workflows while improving the accuracy, consistency, and reproducibility of results. For example, embodiments of the image analysis techniques described herein may be used to perform an initial fully automated assessment of a patient's cancer status. The fully automated analysis may then be reviewed by a physician, who may (i) accept the automated assessment or (ii) choose to adjust parameters in a semi-automated manner under the guidance of the image analysis system. This approach reduces the effort required by physicians and / or support technicians to prepare and review images to ultimately make a diagnosis compared to conventional methods that do not perform automated 3D segmentation or initial status assessment. In contrast, in these conventional methods, physicians and / or technicians must laboriously scroll through 3D images slice by slice to manually identify the two two-dimensional regions of interest. In addition to being time consuming, the conventional methods are largely based on the subjective judgment of the image reviewer (e.g., physician and / or technician) and are therefore prone to inter-reader or intra-reader variability. In contrast, results based on the image analysis workflows, systems, and methods described herein may be obtained in a fully or semi-automated manner that completely eliminates or significantly reduces subjective variability.

[0014] For example, the systems and methods described herein can be used for automated analysis of medical images to determine an uptake index that provides an estimate of the intraprostatic radiopharmaceutical (e.g., a radionuclide-labeled PSMA-binding agent (e.g., 99m Tc-MIP-1404, for example [ 18 The quantitative measurement of the absorption of F]DCFPyL). The absorption index is relevant to the patient risk of prostate cancer and / or prostate cancer severity / stage in the assessment individual. For example, it has been found that the automated classification for clinically significant prostate cancer and clinically insignificant prostate cancer can achieve high sensitivity and specificity.

[0015] In certain embodiments, the image analysis methods described herein utilize a combination of 3D anatomical and functional images obtained for an individual. Anatomical images (e.g., x-ray computed tomography (CT) images) provide detailed anatomical / structural information. Functional images convey information related to physiological activities (e.g., metabolism, blood flow, regional chemical composition, and / or absorption) within specific organs and tissues. Particularly relevant are nuclear medicine images, such as single photon emission computed tomography (SPECT) and / or positron emission tomography (PET) images, which are obtained by detecting radiation emitted from an individual and can be used to infer the spatial distribution of radiopharmaceuticals administered within the individual.

[0016] For example, SPECT imaging can be used to evaluate radiopharmaceuticals 9m Tc-MIP-1404 (which is 99m In certain embodiments, to assess the prostate gland in an individual, 99m Tc-MIP-1404 absorption, obtain a CT image and a corresponding SPECT image of the individual, so that the anatomical / structural information of the CT image can be associated with the functional information of the corresponding SPECT image. The CT and SPECT images are typically acquired via two separate scans (e.g., a first scan of the CT image and a second scan of the SPECT image) using a single multimodal imaging system, so that the individual is in a substantially fixed position for the duration of the two scans. In this way, a mapping between voxels of the CT image and voxels of the SPECT image is established, and volumes identified within the CT image as corresponding to specific organs and / or tissue regions can be used to identify those voxels in the SPECT image that correspond to those same specific organs and / or tissue regions.

[0017] Thus, in certain embodiments, the image analysis methods described herein utilize a convolutional neural network (CNN) to accurately identify the prostate volume within a CT image that corresponds to the prostate of an individual. The identified prostate volume can be used to identify those voxels in a SPECT image that also correspond to the prostate of the individual. Thus, the intensity of the SPECT image voxels corresponding to the prostate of the individual can be used to calculate the intensity of an imaging agent (e.g., a labeled PSMA binding agent, e.g., 99m Tc-MIP-1404 or [ 18 The absorption index can then be converted into a measure of the amount of absorption of a prostate cancer-derived protein (F]DCFPyL). The absorption index can then be converted into a measure of whether an individual has prostate cancer and / or quantify the risk of an individual having prostate cancer, and / or stage the disease (e.g., tracked over time as part of the disease), for example, which can be used by a medical practitioner to suggest treatment options and / or monitor the efficacy of an administered therapy.

[0018] Various advances are described herein that improve the performance of automated measurement of absorption indices in the prostate (or more generally, a specific organ or tissue region of interest) via such a multi-image approach (i.e., analysis of a combination of 3D anatomical images and 3D functional images). These advances include, for example, automated identification of a bounding box in a 3D anatomical image (e.g., using a first convolutional neural network, CNN) to identify the pelvic region where the prostate is located.

[0019] For example, a collection of 3D anatomical images with identified physiology (e.g., an identified pelvic region) is used to train a first CNN on points representing the boundaries of the pelvic region (e.g., vertices of a cubic bounding box), such that the first CNN can be used to automatically identify the pelvic region in a 3D anatomical image of an individual. This provides a more standard-sized initial volumetric region of the 3D anatomical image to be subsequently processed (e.g., via a second CNN) for detailed segmentation of the region of interest within the initial volumetric region - for example, where the bounding box bounds the region in the image corresponding to the prostate and / or bladder and / or rectum and / or gluteal muscles of the individual. Imaging agent absorption indicators can then be determined from portions of the 3D functional image that are mapped to one or more identified regions of the 3D anatomical image. Note that a 'bounding box' as used herein is not necessarily a cube, but may have other shapes. In some embodiments, the bounding box is a cube.

[0020] Determination of the bounding box via the first CNN may use as input a resolution that is significantly less dense than that used for segmentation of the prostate and / or other organs within the bounding box by the second CNN. For example, a whole body 3D anatomical image having a first number of voxels (e.g., 81x68x96 voxels) may be processed via the first CNN to find the bounding box, and then the second CNN may process an image corresponding to the bounding box area but having a denser resolution (e.g., more voxels (e.g., 94x138x253 voxels)) than the first number of voxels.

[0021] For example, a 'bounding box' approach that identifies one or more portions of a 3D anatomical image that are relevant to the analysis at hand before applying a second CNN (for detailed segmentation) improves computational efficiency by removing large portions of the initial 3D anatomical image prior to more computationally intensive subsequent processing. This approach is more computationally efficient than performing detailed segmentation on the entire initial 3D anatomical image because, for example, identification of the pelvic region (e.g., vertices of a cube bounding box) is simpler than detailed segmentation of the prostate, bladder, and / or other tissues of interest. Not only is this approach more computationally efficient, it can also result in more accurate subsequent processing, such as the more detailed segmentation provided by the second CNN. This is because, for example, 3D anatomical images obtained using different machines at different medical institutions are of variable size (e.g., where the variable size means that the number of voxels in the image is different, and / or the volume of patient tissue represented in the image is different), and training a second CNN for automated detailed segmentation of the prostate using portions of a 3D anatomical training image having a more standardized image volume size and in which the organs and other tissue regions of interest are located produces a more robust and accurate segmentation.

[0022] Another advance described herein that improves the performance of automated measurement of absorption indices in the prostate (or more generally, a particular organ or tissue region of interest) via this multi-image approach (i.e., analysis of a combination of 3D anatomical images and 3D functional images) is to accurately identify one or more tissue regions other than the prostate and take into account the imaging agent uptake in those regions in determining (i) an absorption index in the prostate and / or (ii) identification and / or staging of prostate cancer. Certain imaging agents, including PSMA binders, have high uptake in certain organs, which can affect the identification of diseased tissue (e.g., prostate cancer). For example, uptake of a radionuclide-labeled PSMA binder by the bladder can cause scattering of 3D functional images and can reduce the accuracy of imaging agent intensity measured in the prostate located near the bladder. By training a second CNN for detailed segmentation of the prostate and bladder of an individual, the 'leakage' or 'crosstalk' effect and / or other effects caused by uptake of the imaging agent by the bladder can be accurately and automatically accounted for. Furthermore, by training a second CNN for identifying reference regions (e.g., gluteal muscles) in 3D anatomical images, imaging agent intensity measurements can be more accurately scaled / normalized and the accuracy and diagnostic value of uptake measurements in an individual's prostate can be improved.

[0023] Thus, in certain embodiments, the systems and methods described herein utilize a unique combination of two CNN modules, wherein a first CNN module identifies an initial volume of interest (VOI) within a CT image and a second CNN module receives the VOI as input and identifies a prostate volume therein. As described herein, this approach allows the second CNN module to operate with a smaller input size (e.g., a VOI rather than a full CT image). The savings in computational resources (e.g., memory; e.g., processing time) by reducing the input size in this manner can be allocated to improve the accuracy of the second CNN module and / or to improve the speed of the image processing method.

[0024] In certain embodiments, the systems and methods described herein identify various additional tissue volumes within a CT image together with the prostate. For example, in addition to the prostate, additional tissue volumes corresponding to the pelvic bones, bladder, rectum, and gluteal muscles of an individual can be identified. As described herein, the identification of the additional tissue volume can be used for a variety of functions and imparts advantages over other methods (e.g., binary classification methods), in which voxels of a CT image are identified as corresponding to or not corresponding to the prostate. Specifically, the identification of the additional tissue volume can (for example) (i) improve the accuracy of the CNN module in identifying the prostate volume within the CT image, (ii) provide identification of reference areas that can be used to calculate normalized values ​​for absorption index calculations, and (iii) allow correction of crosstalk of the intensity of the SPECT image voxels corresponding to the prostate, which is generated by (for example) the accumulation of radioactive drugs in the bladder.

[0025] In certain embodiments, the image analysis methods described herein can be used to analyze a variety of anatomical and functional images, and are not limited to CT and SPECT images. For example, positron emission tomography (PET) is another functional imaging modality that provides information about the distribution of radiopharmaceuticals within an individual. Like SPECT images, PET images can be used in combination with CT images to determine absorption indices for various organs and tissue regions of interest. The methods described herein can also be applied to various organs and / or tissue regions of interest, such as bones, lymph nodes, liver, and lungs.

[0026] Thus, by quickly and accurately identifying specific organs and tissue regions in medical images, the systems and methods described herein provide accurate and automated determination of absorption indices that provide quantitative measurements of the absorption of radiopharmaceuticals in various organs and tissue regions within an individual. Absorption indices determined in this automated manner provide a valuable tool for evaluating disease risk, status and progression within a patient, as well as treatment efficacy.

[0027] In one aspect, the present invention relates to a method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a prostate of a subject and determine one or more absorption indices indicative of absorption of a radiopharmaceutical therein (i.e., in the prostate), the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT) (e.g., a whole-body CT image; e.g., a local body CT image); e.g., magnetic resonance imaging (MRI); e.g., 3D ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the subject, at least a portion of which is (a) receiving, by a processor, a 3D functional image of the individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)], wherein the 3D functional image comprises a plurality of voxels, each voxel representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represent a physical volume within the pelvic region of the individual; and (c) determining, by the processor using a first module (e.g., a first machine learning module) an initial volume of interest (VOI) within the 3D anatomical image ( (e.g., a parallelepiped, e.g., a cube), the initial VOI corresponds to tissue within the individual's pelvic region and excludes tissue outside the individual's pelvic region (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; e.g., wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the VOI represent a physical volume within the individual's pelvic region); (d) identifying, by the processor using a second module (e.g., a second machine learning module), a prostate volume within the initial VOI that corresponds to the individual's prostate; and (e) identifying, by the processor using the 3D functional image and the anterior chamber volume within the initial VOI of the 3D anatomical image. The method further comprises determining (e.g., and displaying) one or more absorption indices for a prostate volume [e.g., calculating the amount of radiopharmaceutical in the prostate of the individual based on the intensity values ​​of voxels of a 3D functional image corresponding to the identified prostate volume within an initial VOI of the 3D anatomical image; for example, calculating the sum (weighted sum), average and / or maximum of the intensities of voxels of the 3D functional image representing the physical volume occupied by the prostate of the individual] [e.g., wherein the one or more absorption indices include a tumor to background ratio (TBR) value and / or wherein the method includes determining (e.g., and displaying) a (i) clinically significant or (ii) clinically insignificant prostate cancer classification status based at least in part on the TBR value].

[0028] In some embodiments, the first module receives a 3D anatomical image as input and outputs a plurality of coordinate values ​​representing relative corners of a rectangular volume within the 3D anatomical image (e.g., two sets of coordinate values ​​representing relative corners of the rectangular volume). (hereinafter referred to as "Paragraph A")

[0029] In certain embodiments, step (c) includes determining, using the first module, a 3D pelvic bone mask that identifies volumes within the 3D anatomical image corresponding to the individual's pelvic bones (e.g., one or more (up to all) of the sacrum, coccyx, left hip bone, and right hip bone).

[0030] In some embodiments, the first module is a convolutional neural network (CNN) module (e.g., a neural network module utilizing one or more convolutional layers).

[0031] In certain embodiments, step (d) includes using a second module to identify one or more additional tissue volumes within the 3D anatomical image, each volume corresponding to a specific tissue region within the individual, wherein the one or more additional tissue volumes correspond to one or more specific tissue regions selected from the group consisting of: pelvic bones (e.g., sacrum; e.g., coccyx; e.g., left hip bone; e.g., right hip bone); the individual's bladder; the individual's rectum; and the individual's gluteal muscles (e.g., left gluteal muscles; e.g., right gluteal muscles).

[0032] In some embodiments, step (d) includes using a second module to classify each voxel within the initial VOI as corresponding to a specific tissue region in a set of (predetermined) different tissue regions within the individual {e.g., the group includes the prostate and optionally one or more additional tissue regions [e.g., the individual's pelvic bones (e.g., sacrum; e.g., coccyx; e.g., left hip bone; e.g., right hip bone); the individual's bladder; the individual's rectum; and gluteal muscles (e.g., left gluteal muscles; e.g., right gluteal muscles]}. In some embodiments, classifying each voxel within the initial VOI includes: determining, via the second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, wherein the set of likelihood values ​​includes one or more tissue regions of the set of tissue regions. a corresponding likelihood value for each of the plurality of voxels in the initial VOI, the likelihood value representing the likelihood that the voxel represents a physical volume within the tissue region (e.g., as calculated by the second module); and for each of the plurality of voxels within the initial VOI, based on the set of likelihood values ​​determined for the voxel, classifying the voxel as corresponding to a specific tissue region. In some embodiments, the second module receives the initial VOI (e.g., the entire initial VOI) as input and outputs a plurality of values, the plurality of values ​​comprising, for each voxel within the initial VOI, at least one of the following (i), (ii), and (iii): (i) a value classifying the voxel [e.g., classifying the voxel as corresponding to a specific tissue region, e.g., a region selected from a predetermined group of different tissue regions, e.g., an individual the individual's prostate; for example, the individual's pelvic bones (e.g., sacrum; for example, coccyx; for example, left hip bone; for example, right hip bone); the individual's bladder; the individual's rectum; and the individual's gluteal muscles (e.g., left gluteal muscles; for example, right gluteal muscles)]; (ii) a set of likelihood values ​​for voxels [e.g., the likelihood that a voxel corresponds to a specific tissue region, such as a region selected from a predetermined set of different tissue regions, such as the individual's prostate; for example, the individual's pelvic bones (e.g., sacrum; for example, coccyx; for example, left hip bone; for example, right hip bone); the individual's bladder; the individual's rectum; and the individual's gluteal muscles (e.g., left gluteal muscles; for example, right gluteal muscles)]; and (iii) identifying voxels that do not correspond to any of the predetermined sets of different tissue regions (e.g., or may be The (predetermined) set of different tissue regions includes one or more tissue regions selected from the group consisting of: a prostate of the individual; a pelvic bone of the individual (e.g., a sacrum; e.g., a coccyx; e.g., a left hip bone; e.g., a right hip bone); a bladder of the individual; a rectum of the individual; and a gluteal muscle of the individual (e.g., a left gluteal muscle; e.g., a right gluteal muscle).

[0033] In certain embodiments, step (d) includes using a second module to identify a set of one or more base tissue volumes, the one or more base tissue volumes including the identified prostate volume and one or more additional tissue volumes, and wherein the method further includes: identifying, by the processor, one or more auxiliary tissue volumes within the 3D anatomical image using one or more auxiliary modules (e.g., auxiliary machine learning modules), each auxiliary tissue volume corresponding to {e.g., representing the same specific tissue region [e.g., the subject's prostate; e.g., the subject's pelvic bones (e.g., sacrum; e.g., coccyx; e.g., left hip bone; e.g., right hip bone); the subject's bladder; the subject's rectum; and the subject's gluteal muscles (e.g., left gluteal muscles; e.g., right gluteal muscles)]} the base tissue volume identified by the second module; and merging, by the processor, each auxiliary tissue volume with a corresponding base tissue volume identified by the second module (e.g., by incorporating a portion of the corresponding auxiliary tissue volume that was not included in the initial base tissue volume into the corresponding base tissue volume).

[0034] In certain embodiments, the method includes: identifying, by a processor (e.g., using a second module), a reference volume within the 3D anatomical image (e.g., within an initial VOI), the reference volume corresponding to a reference tissue region (e.g., a gluteal muscle) within the individual; and in step (e), determining at least one of the one or more absorption indicators using the 3D functional image and the identified reference volume within the 3D anatomical image (e.g., calculating a normalized value from intensity values ​​of voxels in the 3D functional image corresponding to the identified reference volume within the 3D anatomical image). In some embodiments, at least one of the one or more absorption indices determined using the 3D functional image and the reference volume comprises a tumor to background ratio (TBR) value, wherein determining the TBR value comprises: determining a target intensity value using the intensity values ​​of one or more voxels in the 3D functional image corresponding to the identified prostate volume within the initial VOI of the 3D anatomical image (e.g., wherein the target intensity value is the maximum value of the intensity of the voxels in the 3D functional image corresponding to the prostate volume); determining a background intensity value using the intensity values ​​of one or more voxels in the 3D functional image corresponding to the identified reference volume within the 3D anatomical image [e.g., wherein the background intensity value is the average intensity of multiple (e.g., all) voxels in the 3D functional image corresponding to the identified reference volume]; and determining the ratio of the target intensity value to the background intensity value as the TBR value. In some embodiments, the method comprises determining the prostate cancer status of the individual based on a comparison of the TBR value with one or more thresholds (e.g., predetermined thresholds). In certain embodiments, one or more thresholds are determined using a plurality of reference TBR values ​​[e.g., each reference TBR value is determined from a corresponding set of reference images (e.g., a reference 3D anatomical image and a reference 3D functional image; e.g., a CT / SPECT image set)], each reference TBR value being associated with a particular classified prostate cancer state (e.g., assigned by a medical practitioner) [e.g., the Gleason grade of the same individual determined by the reference TBR value (e.g., based on histopathology of radical prostatectomy)]. In certain embodiments, one or more thresholds are determined using a receiver operating characteristic (ROC) curve [e.g., using an area under the curve (AUC) analysis; e.g., to provide a specific sensitivity value and / or a specific specificity value]. In certain embodiments, one or more thresholds include a plurality of thresholds, such that comparison of the TBR value with the plurality of thresholds is able to determine the prostate cancer state according to a level on a non-binary scale [e.g., the scale has three or more degrees (e.g., clearly negative, possibly negative, possibly positive, clearly positive)] (e.g., wherein each threshold is associated with the average Gleason score). In certain embodiments, the method comprises determining the prostate cancer status of the individual as follows: (i) clinically significant if the TBR value is above a cutoff threshold; or (ii) clinically insignificant if the TBR value is below the cutoff threshold.

[0035] In certain embodiments, the method includes identifying, by a processor (e.g., using a second module), a bladder volume within the 3D anatomical image (e.g., within the initial VOI) corresponding to the individual's bladder; and in step (e), using the intensity of voxels in the 3D functional image corresponding to the identified bladder volume within the 3D anatomical image to correct for bladder crosstalk (e.g., scatter and / or partial volume effects, through which absorption of radiopharmaceuticals in the bladder affects the intensity values ​​of voxels in the 3D functional image representing physical volumes within the prostate) [e.g., by adjusting the intensity of voxels in the 3D functional image corresponding to the prostate volume based on their proximity to the identified bladder volume and / or bladder absorption (e.g., based on intensity determination of voxels in the 3D functional image corresponding to the bladder volume); for example, by using the intensity of voxels in the 3D functional image to establish a radiation scatter model for the bladder and adjusting the intensity of voxels in the 3D functional image based on the model]. In certain embodiments, correcting for crosstalk from the bladder includes determining one or more bladder intensity dumping functions that model a contribution of intensity of a radiopharmaceutical originating from within the individual's bladder to intensity of one or more voxels of the 3D functional image, the one or more voxels corresponding to one or more regions in the 3D anatomical image that are outside of an identified bladder volume, wherein the one or more bladder intensity dumping functions models the contribution as a function of distance from the identified bladder volume [e.g., wherein each of the one or more bladder intensity dumping functions models intensity dumping along a particular direction]. The template is obtained by fitting a template function (e.g., an nth-degree polynomial) to the intensity of voxels of the 3D functional image corresponding to the identified bladder volume in the 3D anatomical image and oriented along a specific direction]; and for each of the one or more voxels of the 3D functional image corresponding to the identified prostate volume in the 3D anatomical image, adjusting the intensity of the voxel for bladder crosstalk using one or more bladder intensity drainage functions [e.g., by evaluating the one or more bladder intensity drainage functions to determine the bladder intensity drainage value of the voxel and subtracting the bladder intensity drainage value from the intensity of the voxel to obtain a corrected voxel intensity].

[0036] In certain embodiments, the method includes: identifying, by a processor (e.g., using a second module), a bladder volume within a 3D anatomical image (e.g., an initial VOI) that corresponds to a bladder of the individual; determining, by the processor, an expanded bladder volume by applying a morphological expansion operation to the identified bladder volume; and in step (e), determining one or more absorption indices using intensity values ​​of voxels of the 3D functional image that (i) correspond to the identified prostate volume within the VOI of the 3D anatomical image, but (ii) do not correspond to regions of the 3D anatomical image that are within the expanded bladder volume (e.g., thereby omitting from calculation of the one or more absorption indices those voxels in the 3D functional image that correspond to locations in the 3D anatomical image that are within a predetermined distance from the identified bladder volume; e.g., and therefore, are too close to the identified bladder volume).

[0037] In certain embodiments, the 3D functional image is a nuclear medicine image (e.g., a single photon emission computerized tomography (SPECT) scan; e.g., a positron emission tomography (PET) scan) of a subject after administration of a radiopharmaceutical to the subject. In certain embodiments, the radiopharmaceutical comprises a PSMA-binding agent (e.g., 99m Tc-MIP-1404; for example, [ 18 In certain embodiments, the nuclear medicine image is a single photon emission computerized tomography (SPECT) scan of a subject obtained after administration of a radiopharmaceutical to the subject. In certain embodiments, the radiopharmaceutical comprises 99m Tc-MIP-1404.

[0038] In certain embodiments, the method comprises determining one or more diagnostic or prognostic values ​​for an individual based on at least a portion of the one or more uptake indices [e.g., providing values ​​for a measure of disease status, progression, life expectancy (e.g., overall survival), treatment efficacy, and the like for the individual (e.g., Gleason score)]. In certain embodiments, determining at least one of the one or more diagnostic or prognostic values ​​comprises comparing the uptake indices to one or more threshold values. In certain embodiments, at least one of the one or more diagnostic or prognostic values ​​estimates the risk of a subject developing clinically significant prostate cancer.

[0039] In some embodiments, the method includes: (f) causing, by a processor, display of an interactive graphical user interface (GUI) to present a visual representation of a 3D anatomical image and / or a 3D functional image to a user; and (g) generating, by the processor, a graphical representation of the 3D anatomical image and / or the 3D functional image within the GUI as a selectable and overlappable layer so that either can be selected for display (e.g., via a corresponding user-selectable graphical control element (e.g., a toggle element)) and reproduced separately, or both can be selected for display and reproduced together by overlaying the 3D anatomical image with the 3D functional image. In some embodiments, step (g) includes generating a graphical representation of a selectable and overlappable segmentation layer, the segmentation layer comprising one or more identified specific tissue volumes within the 3D anatomical image, wherein upon selection of the segmentation layer for display, a graphic representation of the one or more specific tissue volumes is overlaid on the 3D anatomical image and / or the 3D functional image (e.g., as a contour; e.g., as a semi-transparent color-coded volume). In some embodiments, the one or more specific tissue volumes comprise an identified prostate volume. In some embodiments, the method includes, in step (g), causing a 2D cross-sectional view of the 3D anatomical image and / or the 3D functional image to be reproduced in an interactive 2D viewer so that a user can adjust the position of the 2D cross-sectional view. In some embodiments, the method includes, in step (g), causing an interactive (e.g., rotatable; e.g., sliceable) 3D view of the 3D anatomical image and / or the 3D functional image to be reproduced. In some embodiments, the method includes causing a graphical element (e.g., a crosshair, a target, a color marker, etc.) to be displayed in a GUI, the graphical element indicating the position of a voxel corresponding to the identified prostate volume (e.g., the position also corresponds to a voxel of the 3D functional image having a maximum intensity (e.g., maximum corrected intensity) compared to other voxels of the 3D functional image corresponding to the identified prostate volume), thereby facilitating user review and / or quality control of the method (e.g., allowing a medical practitioner to verify the identified position corresponding to the expected physical position of the prostate of an individual). In certain embodiments, the method includes causing display within the GUI of text and / or a graphic representing the one or more absorption indicators determined in step (e) (e.g., and optionally one or more prognostic values ​​determined therefrom) and a quality control graphic widget for guiding a user through a quality control and reporting workflow to review and / or update the one or more absorption indicators (e.g., wherein the quality control graphic widget includes a selectable graphical control element for receiving user input corresponding to (i) automatic approval by the processor of the one or more absorption indicators as determined, or (ii) disapproval of the automated determination of the one or more absorption indicators).In certain embodiments, the method comprises: receiving user input via a quality control graphic widget, the user input corresponding to approval of automated determination of one or more absorption indices (e.g., and any prognostic values ​​determined therefrom); and in response to receiving the user input corresponding to approval of automated determination of the one or more absorption indices, generating a report for the individual, the report comprising a representation of the one or more automatically determined absorption indices [e.g., the report comprising: an identification of the individual (e.g., an anonymous patient ID number); a representation (e.g., text) of the one or more determined absorption indices (e.g., and any prognostic values ​​determined therefrom); and a representation (e.g., graphic and / or text) of user approval of the automated determination of the one or more absorption indices (e.g., and any prognostic values ​​determined therefrom)].

[0040] In certain embodiments, the method includes: receiving user input, via a quality control graphical widget, corresponding to an unapproved automated determination of one or more absorption indices (e.g., and any prognostic values ​​determined therefrom); in response to receiving the user input corresponding to the unapproved automated determination of the one or more absorption indices, causing a voxel selection graphical element (e.g., a cursor; e.g., an adjustable crosshair) to be displayed for user selection of one or more voxels of the 3D functional image (e.g., directly or indirectly, e.g., via selection of voxels of the 3D anatomical image and subsequent determination of corresponding voxels of the 3D functional image) for determination of updated values ​​of the one or more absorption indices; receiving, via the voxel selection graphical element, a user selection of one or more voxels of the 3D functional image for determination of updated values ​​of the one or more absorption indices [e.g., where the user selection is one or more corrected background intensity measurement locations and / or one or more prognostic values ​​determined therefrom]; or more corrected prostate intensity locations, for example, automated identification in which the user disregards the location of the prostate and / or automated identification in which the user disregards the location of background areas (e.g., gluteal muscles)]; updating, by a processor, values ​​of one or more absorption indices using the user-selected voxels; and generating, by the processor, a report for the individual, the report comprising representations of the one or more updated absorption indices [e.g., the report comprising: an identification of the individual (e.g., an anonymous patient ID number); a representation (e.g., text) of the one or more determined absorption indices (e.g., and any prognostic values ​​determined therefrom); and a representation (e.g., graphic and / or text) of user-approved automated determination of the one or more absorption indices (e.g., and any prognostic values ​​determined therefrom) using the user-selected voxels] (e.g., text indicating that the absorption indices were determined via semi-automated analysis and from user-manually selected voxels)].

[0041] In certain embodiments, the method includes: receiving user input via a quality control graphic widget corresponding to an automated determination of one or more absorption indices that are not approved (e.g., and any prognostic values ​​determined therefrom); receiving user input via the quality control graphic widget corresponding to a rejection of quality control (e.g., due to low image quality); and generating, by a processor, a report for the individual, wherein the report includes an identification of the rejection of the quality control.

[0042] In some embodiments, voxels of the 3D functional image are related to voxels of the 3D anatomical image via a known relationship [e.g., each voxel of the 3D functional image is associated with one or more voxels of the 3D anatomical image; for example, each of multiple groups of one or more voxels of the 3D functional image is associated with a set of one or more voxels of the 3D anatomical image; for example, coordinates associated with voxels of the 3D functional image are related to coordinates associated with voxels of the anatomical 3D image via a known functional relationship (e.g., via a known spatial relationship between the first and second imaging modalities)].

[0043] In some embodiments, the first module is a first CNN (convolutional neural network) module and the second module is a second CNN module, wherein the first CNN module comprises a larger number of convolution filters than the second CNN module (e.g., up to at least 1.5 times, e.g., up to at least twice, e.g., up to about twice, e.g., up to at least three times, e.g., up to about three times).

[0044] In certain embodiments, the method includes performing steps (a) and (c) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each initial VOI being within one of the plurality of 3D anatomical images, wherein a size variability of the initial VOI is less than (e.g., substantially less than) a size variability of the 3D anatomical images (e.g., wherein variability of "size" means either or both of (i) variability of one or more dimensions of the anatomical volume represented in the image, e.g., as measured in mm, and (ii) variability of the number of voxels in the image along each of one or more dimensions of the image) (e.g., wherein the size of the 3D anatomical images (each full image determining each VOI) varies along each of the one or more dimensions by at least 200 mm, and / or at least 300 mm, and / or at least 400 mm, and / or at least 500 mm, and / or up to 400 mm). mm, and / or up to 500 mm, and / or up to 1000 mm, and / or up to 1500 mm); for example, wherein the size of the 3D anatomical image (determining each complete image of each VOI) varies by at least 25 voxels, and / or at least 50 voxels, and / or at least 100 voxels, and / or at least 250 voxels, and / or at least 300 voxels, and / or up to 250 voxels, and / or up to 300 voxels, and / or up to 500 voxels along each of one or more dimensions); for example, wherein the size of the VOI varies by less than or equal to 200 mm (e.g., less than or equal to 100 mm; for example, less than or equal to 50 mm) along each of one or more dimensions; for example, wherein the size of the VOI varies by less than or equal to 250 voxels (e.g., less than or equal to 200 voxels; for example, less than or equal to 150 voxels) along each of one or more dimensions].

[0045] In some embodiments, the first module is a CNN module that receives as input and operates on a downsampled version of a 3D anatomical image having a first resolution (e.g., a low-resolution version of the 3D anatomical image), and wherein the second module is a CNN module that is cropped to the initial VOI and has as input and operates on a high-resolution version of the 3D anatomical image at a second resolution, the second resolution being higher than the first resolution (e.g., at least 2 times higher; e.g., at least 4 times higher; e.g., at least 8 times higher). In some embodiments, the first module receives as input at least a portion of the 3D anatomical image, and wherein the physical volume represented by the input to the first module is (i) at least 2 times (e.g., 4 times; e.g., 8 times) the physical volume represented by the initial VOI and / or (ii) 2 times (e.g., 4 times; e.g., 8 times) the physical volume represented by the initial VOI in at least one dimension.

[0046] In some embodiments, the first module is a CNN module trained to identify a graphical representation of a pelvic region within a 3D anatomical image and wherein the second module is a CNN module trained to identify a graphical representation of prostate tissue within a 3D anatomical image. (hereinafter referred to as "Paragraph B")

[0047] In another aspect, the present invention relates to a method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a target tissue region of a subject and determining one or more absorption indices indicative of absorption of a radiopharmaceutical therein, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the subject obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical image comprises a specific anatomical region of the subject (e.g., a specific group of tissues of interest, such as the pelvic region, the thoracic region, the head and / or neck region, and the like); (a) receiving, by a processor, a 3D functional image of the individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)], wherein the 3D functional image comprises a plurality of voxels, each voxel representing a particular physical volume within the individual and having an intensity value representing detected radiation emitted from the particular physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represent a physical volume within the particular anatomical region of the individual; and (c) ) determining, by the processor using a first module (e.g., a first machine learning module), an initial volume of interest (VOI) (e.g., a rectangular prism) within the 3D anatomical image, the initial VOI corresponding to a specific anatomical region (e.g., a collection of related tissues, such as the pelvic region, the thoracic region, the head and / or the neck region, and the like) that includes the target tissue region (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; e.g., wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the VOI represent physical volumes within the specific anatomical region; (d) determining, by the processor using a second module (e.g., a first machine learning module), an initial volume of interest (VOI) (e.g., a rectangular prism) within the 3D anatomical image, the initial VOI corresponding to a specific anatomical region (e.g., a collection of related tissues, such as the pelvic region, the thoracic region, the head and / or the neck region, and the like) that includes the target tissue region (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; e.g., wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the VOI represent physical volumes within the specific anatomical region; two modules (e.g., a second machine learning module) identify a target volume within the initial VOI corresponding to the target tissue region of the individual; and (e) determining, by a processor, one or more absorption indices using the 3D functional image and the identified target volume within the VOI of the 3D anatomical image [e.g., calculating the amount of radioactive drug in the target tissue region of the individual based on the intensity values ​​of the voxels of the 3D functional image corresponding to the identified target volume within the VOI of the 3D anatomical image; for example, calculating the sum (weighted sum), average and / or maximum value of the intensities of the voxels of the 3D functional image representing the physical volume occupied by the target tissue region of the individual].

[0048] In certain embodiments, the method has one or more of the features expressly set forth in paragraphs "A"-"B".

[0049] In another aspect, the present invention relates to a method for automatically analyzing a 3D functional image [e.g., a nuclear medicine image (e.g., a SPECT image; e.g., a PET image)] to correct for crosstalk in prostate voxel intensity, the crosstalk coming from radiopharmaceuticals absorbed into the bladder, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of a subject obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., 3D ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the subject, at least a portion of which corresponds to the bladder and prostate of the subject; (b) receiving, by the processor, the 3D functional image of the subject, wherein the 3D functional image comprises a plurality of voxels, each voxel representing a particular physical volume within the subject and having an intensity value representing detected radiation emitted from the particular physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within the bladder and / or prostate of the subject; and (c) performing, by the processor, a multi-step process of analyzing the 3D anatomical image of the subject. automatically identifying, by a processor, within the 3D functional image: (i) a prostate volume corresponding to the prostate of the individual and (ii) a bladder volume corresponding to the bladder of the individual; (d) automatically identifying, by a processor, within the 3D functional image: (i) a plurality of prostate voxels corresponding to the identified prostate volume and (ii) a plurality of bladder voxels corresponding to the identified bladder volume; (e) adjusting, by the processor, one or more measured intensities of the prostate voxels (e.g., one or more accumulated measurements and / or peak measurements and / or average measurements and / or median measurements corresponding to the identified prostate volume and / or corresponding to the intensity of each and / or accumulated multiple regions of the identified bladder volume) based on one or more measured intensities of the bladder voxels (e.g., one or more accumulated measurements and / or peak measurements and / or average measurements and / or median measurements corresponding to the identified prostate volume and / or corresponding to the intensity of each and / or accumulated multiple regions of the identified bladder volume); and (f) determining, by the processor, one or more absorption indicators indicative of absorption of a radiopharmaceutical within the prostate of the individual using the adjusted intensities of the prostate voxels.

[0050] In certain embodiments, the method has one or more of the features expressly set forth in paragraphs "A"-"B".

[0051] In another aspect, the present invention relates to a method for detecting a prostate cancer status and / or quantifying a prostate cancer risk in an individual based on automated analysis of a 3D functional image (e.g., a SPECT image) of a portion of the individual, the method comprising: (a) acquiring a 3D functional image (e.g., a SPECT image) after administering a radiopharmaceutical comprising a PSMA binder to the individual; (b) identifying, by a processor of a computing device, a 3D target volume within the 3D functional image, the 3D target volume corresponding to the prostate of the individual; (c) determining, by the processor, a target to background ratio (TBR) value using the intensities of voxels of the 3D target volume; and (d) causing, by the processor, a graphical presentation of text and / or a graphic representing the determined TBR value for display within an interactive graphical user interface (GUI) [e.g., wherein the method comprises determining (e.g., and displaying) a (i) clinically significant (ii) clinically significant prostate cancer classification status based at least in part on the TBR value].

[0052] In certain embodiments, the method has one or more of the features expressly set forth in paragraphs "A"-"B".

[0053] In another aspect, the present invention relates to a system for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a prostate of an individual and determine one or more absorption indices indicative of absorption of a radiopharmaceutical therein (i.e., in the prostate), the system comprising: a processor; and a memory having instructions stored thereon (e.g., external to or embedded in the processor), wherein the instructions, when executed by the processor, cause the processor to: (a) receive a 3D anatomical image of the individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT) (e.g., a whole-body CT image; e.g., a body-region CT image); e.g., magnetic resonance imaging (MRI); e.g., 3D ultrasound], wherein (a) receiving a 3D functional image of the individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)], wherein the 3D functional image comprises a plurality of voxels, each voxel representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represents a physical volume within the pelvic region of the individual; and (c) measuring using a first module (e.g., a first machine learning module) (d) identifying, using a second module (e.g., a second machine learning module), a prostate volume within the initial VOI corresponding to a prostate of the individual; and (e) identifying, using the 3D functional image and the initial VOI within the 3D anatomical image, a volume of interest (VOI) corresponding to tissue within the pelvic region of the individual and excluding tissue outside the pelvic region of the individual (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; e.g., wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the VOI represent physical volumes within the pelvic region of the individual); (f) identifying, using a second module (e.g., a second machine learning module), a prostate volume within the initial VOI corresponding to the prostate of the individual; and (g) identifying, using the 3D functional image and the initial VOI within the 3D anatomical image, a volume of interest (VOI) corresponding to the prostate of the individual. Determine (e.g., and display) one or more absorption indicators for the identified prostate volume [e.g., calculate the amount of radiopharmaceutical in the individual's prostate based on the intensity values ​​of voxels of a 3D functional image corresponding to the identified prostate volume within an initial VOI of the 3D anatomical image; for example, calculate the sum (weighted sum) and / or average and / or maximum of the intensities of voxels of the 3D functional image representing the physical volume occupied by the individual's prostate] [e.g., wherein the one or more absorption indicators include a tumor to background ratio (TBR) value, and wherein the instructions, when executed by a processor, cause the processor to determine (e.g., and display) a (i) clinically significant or (ii) clinically significant prostate cancer classification status].

[0054] In certain embodiments, the system has one or more of the features expressly set forth in paragraphs "A"-"B".

[0055] In another aspect, the present invention relates to a system for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a target tissue region of an individual and determine one or more absorption indices indicative of absorption of a radiopharmaceutical therein (i.e., in the target tissue region), the system comprising: a processor; and a memory having instructions stored thereon (e.g., external to the processor or embedded therein), wherein the instructions, when executed by the processor, cause the processor to: (a) receive a 3D anatomical image of an individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., 3D ultrasound], wherein the 3D anatomical image includes a specific anatomical region of the individual (e.g., tissue of interest); (a) receiving a 3D functional image of an individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)], wherein the 3D functional image comprises a plurality of voxels, each voxel representing a particular physical volume within the individual and having an intensity value representing detected radiation emitted from the particular physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represent a physical volume within the particular anatomical region of the individual; and (c) performing a 3D functional image of an individual using a first module (e.g., a first machine learning module). (d) determining, using a second module (e.g., a second machine learning module), an initial volume of interest (VOI) (e.g., a rectangular prism) within the 3D anatomical image, the initial VOI corresponding to a particular anatomical region (e.g., a collection of related tissues, such as the pelvic region, the thoracic region, the head and / or the neck region, and the like) that includes a target tissue region (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; e.g., wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the VOI represent physical volumes within the particular anatomical region; ...f) identifying, using a second module (e.g., a second machine learning module), a target volume within the initial VOI that corresponds to the target tissue region for the individual; and (g) identifying, using the 3D functional image and the 3D decoder, a target volume within the initial VOI that corresponds to the target tissue region for the individual. The method further comprises: determining one or more absorption indicators for a target volume identified within a VOI of a 3D anatomical image [e.g., calculating the amount of radioactive drug in a target tissue region of an individual based on the intensity values ​​of voxels of a 3D functional image corresponding to the target volume identified within the VOI of the 3D anatomical image; for example, calculating the sum (weighted sum) and / or average and / or maximum of the intensities of voxels of a 3D functional image representing a physical volume occupied by the target tissue region of the individual] [e.g., wherein the one or more absorption indicators include a tumor to background ratio (TBR) value, and wherein the instructions, when executed by a processor, cause the processor to determine (e.g., and display) a (i) clinically significant or (ii) clinically significant cancer classification status based at least in part on the TBR value].

[0056] In certain embodiments, the system has one or more of the features expressly set forth in paragraphs "A"-"B".

[0057] In another aspect, the present invention relates to a system for detecting a prostate cancer status and / or quantifying a prostate cancer risk of an individual based on automated analysis of a 3D functional image (e.g., a SPECT image) of a portion of the individual, the system comprising: a processor; and a memory having instructions stored thereon (e.g., external to or embedded in the processor), wherein the instructions, when executed by the processor, cause the processor to: (a) acquire a 3D functional image (e.g., a SPECT image) of a portion (e.g., any or all) of the individual after administration of a radiopharmaceutical comprising a PSMA binder to the individual; (b) identify a 3D target volume within the 3D functional image, the 3D target volume corresponding to the prostate of the individual; (c) determine a target to background ratio (TBR) value using the intensities of the voxels of the 3D target volume; and (d) generate a graphical representation of text and / or graphics representing the determined TBR value for display within an interactive graphical user interface (GUI) [e.g., wherein the instructions, when executed by the processor, cause the processor to determine (e.g., and display) a prostate cancer classification status as (i) clinically significant or (ii) clinically insignificant based at least in part on the TBR value].

[0058] In certain embodiments, the system has one or more of the features expressly set forth in paragraphs "A"-"B".

[0059] In certain embodiments, the invention relates to a computer aided detection (CADe) device comprising any of the systems described herein. In certain embodiments, the instructions cause the processor to identify a classification of an individual as having clinically significant prostate cancer or clinically insignificant prostate cancer.

[0060] In certain embodiments, the invention relates to a computer-aided diagnosis (CADx) device comprising any of the systems described herein. In certain embodiments, the instructions cause the processor to identify a classification of an individual as having clinically significant prostate cancer or clinically insignificant prostate cancer.

[0061] In certain embodiments, the invention relates to a combination product comprising: (a) a radiolabeled PSMA-binding agent (e.g. 99m Tc-MIP-1404; for example, [ 18 F] DCFPyL, for example, other known PSMA binding agents); and (b) a computer-assisted detection (CADe) device comprising any of the systems described herein. In certain embodiments, the combination product comprises a label indicating that the radiolabeled PSMA binding agent is used in conjunction with the computer-assisted detection device.

[0062] In another aspect, the present invention relates to a method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a target tissue region (e.g., a prostate; e.g., a lung; e.g., one or more bones; e.g., a lymph node; e.g., a brain), the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of an individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) of the individual; (b) determining, by the processor using a first module (e.g., a first machine learning module), an initial volume of interest (VOI) within the 3D anatomical image (e.g., a rectangular volume of interest); prism), the initial VOI corresponds to a specific anatomical region (e.g., a collection of related tissues, such as the pelvic region, the thoracic region, the head and / or the neck region, and the like) that includes the target region (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; for example, wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; for example, wherein the majority of voxels within the VOI represent physical volumes within the specific anatomical region; (c) identifying, by the processor using a second module (e.g., a second machine learning module), a target volume within the initial VOI that corresponds to the target tissue region of the individual; and (d) storing and / or providing, by the processor, a 3D segmentation mask corresponding to the identified target volume for display and / or further processing.

[0063] In some embodiments, the first module receives a 3D anatomical image as input and outputs a plurality of coordinate values ​​representing relative corners of a rectangular volume within the 3D anatomical image (e.g., two sets of coordinate values ​​representing relative corners of the rectangular volume). (hereinafter referred to as "Paragraph C")

[0064] In some embodiments, the first module is a convolutional neural network (CNN) module (e.g., a neural network module utilizing one or more convolutional layers).

[0065] In certain embodiments, step (c) comprises identifying, using the second module, one or more additional tissue volumes within the 3D anatomical image, each additional tissue volume corresponding to a specific tissue region within the subject.

[0066] In some embodiments, step (c) includes using a second module to classify each voxel within the initial VOI as corresponding to a specific tissue region from a set of (predetermined) different tissue regions of the individual {e.g., the group includes the target tissue region and optionally one or more additional tissue regions [e.g., the individual's pelvic bones (e.g., sacrum; e.g., coccyx; e.g., left hip bone; e.g., right hip bone); the individual's bladder; the individual's rectum; and gluteal muscles (e.g., left gluteal muscles; e.g., right gluteal muscles]}.

[0067] In some embodiments, classifying each voxel within the initial VOI includes: determining a set of likelihood values ​​for each of a plurality of voxels within the initial VOI via a second module, wherein the set of likelihood values ​​includes a corresponding likelihood value for each of one or more tissue regions of a tissue region group, the likelihood value representing the likelihood that the voxel embodies a physical volume within the tissue region (e.g., as calculated by the second module); and for each of the plurality of voxels within the initial VOI, classifying the voxel as corresponding to a specific tissue region based on the set of likelihood values ​​determined for the voxel.

[0068] In some embodiments, the second module receives an initial VOI (e.g., the entire initial VOI) as input and outputs a plurality of values, the plurality of values ​​comprising, for each voxel within the initial VOI, at least one of the following (i), (ii), and (iii): (i) a value classifying the voxel [e.g., classifying the voxel as corresponding to a specific tissue region, e.g., a region selected from a predetermined set of different tissue regions, e.g., the predetermined set comprising a target tissue region, e.g., a prostate of the individual; e.g., a pelvic bone of the individual (e.g., a sacrum; e.g., a coccyx; e.g., a left hip bone; e.g., a right hip bone); a bladder of the individual; a rectum of the individual; and a gluteal muscle of the individual (e.g., a left gluteal muscle; e.g., a right gluteal muscle)]; (ii) a set of likelihood values ​​for the voxel [e.g., that the voxel corresponds to a specific tissue region likelihood of, for example, a region selected from a predetermined set of different tissue regions, for example, a prostate of the individual; for example, a pelvic bone of the individual (e.g., a sacrum; for example, a coccyx; for example, a left hip bone; for example, a right hip bone); a bladder of the individual; a rectum of the individual; and a gluteal muscle of the individual (e.g., a left gluteal muscle; for example, a right gluteal muscle)]; and (iii) identifying voxels that do not correspond (e.g., or may not correspond, or identifying the likelihood of voxels that do not correspond) to any predetermined set of different tissue regions (e.g., identifying voxels that correspond or may correspond to background regions, or the likelihood of voxels that correspond to background regions, for example, the background regions are not of diagnostic significance) values ​​(e.g., so that the second module classifies and / or calculates the likelihood value of the entire VOI in one round rather than operating on each voxel one at a time).

[0069] In certain embodiments, step (c) includes using a second module to identify a set of one or more base tissue volumes, the one or more base tissue volumes including the identified target volume and one or more additional tissue volumes, and wherein the method further includes: identifying, by the processor, using one or more auxiliary modules (e.g., auxiliary machine learning modules) one or more auxiliary tissue volumes within the 3D anatomical image, each auxiliary tissue volume corresponding to {e.g., representing the same specific tissue region [e.g., the subject's prostate; e.g., the subject's pelvic bones (e.g., sacrum; e.g., coccyx; e.g., left hip bone; e.g., right hip bone); the subject's bladder; the subject's rectum; and the subject's gluteal muscles (e.g., left gluteal muscles; e.g., right gluteal muscles)]} the base tissue volume identified by the second module; and merging, by the processor, each auxiliary tissue volume with a corresponding base tissue volume identified by the second module (e.g., by adding to the corresponding base tissue volume a portion of the corresponding auxiliary tissue volume that was not included in the initial base tissue volume).

[0070] In certain embodiments, the method includes: (e) causing, by a processor, display of an interactive graphical user interface (GUI) to present a visual representation of a 3D anatomical image to a user; and (f) generating, by the processor within the GUI, a graphical representation of the 3D anatomical image and selectable and overlappable segmentation layers of one or more identified specific tissue volumes within the 3D anatomical image, wherein upon selection of the segmentation layer display, graphics representing the one or more specific tissue volumes are superimposed on the 3D anatomical image (e.g., as outlines; e.g., as semi-transparent color-coded volumes).

[0071] In certain embodiments, one or more specific tissue volumes comprise the identified target volume.

[0072] In certain embodiments, the method includes, at step (f), causing the 2D cross-sectional view of the 3D anatomical image to be rendered within an interactive 2D viewer so that a user can adjust a position of the 2D cross-sectional view.

[0073] In certain embodiments, the method includes, in step (f), causing an interactive (eg, rotatable; eg, sliceable) 3D view rendering of the 3D anatomical image.

[0074] In some embodiments, the first module is a first CNN (convolutional neural network) module and the second module is a second CNN module, wherein the first CNN module comprises a larger number of convolution filters than the second CNN module (e.g., up to at least 1.5 times, e.g., up to at least twice, e.g., up to about twice, e.g., up to at least three times, e.g., up to about three times).

[0075] In certain embodiments, the method includes performing steps (a) and (b) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each initial VOI being within one of the plurality of 3D anatomical images, wherein a size variability of the initial VOI is less than (e.g., substantially less than) a size variability of the 3D anatomical images (e.g., wherein variability of "size" means either or both of (i) variability of one or more dimensions of the anatomical volume represented in the image, e.g., as measured in mm, and (ii) variability of the number of voxels in the image along each of one or more dimensions of the image) [e.g., wherein the size of the 3D anatomical images (each full image determining each VOI) varies along each of the one or more dimensions by at least 200 mm, and / or at least 300 mm, and / or at least 400 mm, and / or at least 500 mm, and / or up to 400 mm]. mm, and / or up to 500 mm, and / or up to 1000 mm, and / or up to 1500 mm); for example, wherein the size of the 3D anatomical image (determining each complete image of each VOI) varies by at least 25 voxels, and / or at least 50 voxels, and / or at least 100 voxels, and / or at least 250 voxels, and / or at least 300 voxels, and / or up to 250 voxels, and / or up to 300 voxels, and / or up to 500 voxels along each of one or more dimensions); for example, wherein the size of the VOI varies by less than or equal to 200 mm (e.g., less than or equal to 100 mm; for example, less than or equal to 50 mm) along each of one or more dimensions; for example, wherein the size of the VOI varies by less than or equal to 250 voxels (e.g., less than or equal to 200 voxels; for example, less than or equal to 150 voxels) along each of one or more dimensions].

[0076] In some embodiments, the first module is a CNN module that receives as input and operates on a downsampled version of a 3D anatomical image having a first resolution (e.g., a low resolution version of the 3D anatomical image), and wherein the second module is a CNN module that is cropped to an initial VOI and has as input and operates on a high resolution version of the 3D anatomical image having a second resolution that is higher than the first resolution (e.g., at least 2 times higher; e.g., at least 4 times higher; e.g., at least 8 times higher).

[0077] In some embodiments, the first module receives as input at least a portion of a 3D anatomical image, and wherein the physical volume represented by the input to the first module is (i) at least 2 times (e.g., 4 times; e.g., 8 times) the physical volume represented by the initial VOI and / or (ii) 2 times (e.g., 4 times; e.g., 8 times) the physical volume represented by the initial VOI in at least one dimension.

[0078] In certain embodiments, the first module is a CNN module trained to identify graphical representations of specific anatomical regions within 3D anatomical images and wherein the second module is a CNN module trained to identify graphical representations of target tissue regions within 3D anatomical images.

[0079] In certain embodiments, the method further includes: receiving, by a processor, a 3D functional image of an individual obtained using a functional imaging modality [e.g., single photon emission computed tomography (SPECT); e.g., positron emission tomography (PET)], wherein the 3D functional image comprises a plurality of voxels, each voxel representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represent a physical volume within a specific anatomical region of the individual; and identifying a 3D volume within the 3D functional image corresponding to the identified target volume using a 3D segmentation mask (e.g., by mapping the 3D segmentation mask to the 3D functional image).

[0080] In some embodiments, voxels of the 3D functional image are related to voxels of the 3D anatomical image via a known relationship [e.g., each voxel of the 3D functional image is associated with one or more voxels of the 3D anatomical image; e.g., each of a plurality of groups of one or more voxels of the 3D functional image is associated with a set of one or more voxels of the 3D anatomical image; e.g., coordinates associated with voxels of the 3D functional image are related to coordinates associated with voxels of the anatomical 3D image via a known functional relationship (e.g., via a known spatial relationship between the first and second imaging modalities)]. (hereinafter referred to as "Paragraph D")

[0081] In another aspect, the present invention relates to a system for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a target tissue region (e.g., prostate; e.g., lung; e.g., one or more bones; e.g., lymph nodes; e.g., brain), the system comprising: a processor of a computing device; and a memory having instructions stored thereon, wherein the instructions, when executed by the processor, cause the processor to: (a) receive a 3D anatomical image of an individual obtained using an anatomical imaging modality [e.g., x-ray computed tomography (CT); e.g., magnetic resonance imaging (MRI); e.g., 3D ultrasound], wherein the 3D anatomical image comprises a graphical representation of tissue (e.g., soft tissue and / or bone) within the individual; (b) determine, using a first module (e.g., a first machine learning module), the 3D solution; and (c) determine, using a first module (e.g., a first machine learning module), a 3D volume within the 3D image corresponding to a target tissue region (e.g., prostate; e.g., lung; e.g., one or more bones; e.g., lymph nodes; e.g., brain). (c) identifying, using a second module (e.g., a second machine learning module), a target volume within the initial VOI corresponding to a specific anatomical region (e.g., a set of related tissues, such as the pelvic region, the thoracic region, the head and / or the neck region, and the like) that includes a target region (e.g., wherein the VOI excludes more voxels of the 3D anatomical image than it includes; e.g., wherein the VOI includes less than 25% of the voxels of the 3D anatomical image; e.g., wherein a majority of the voxels within the VOI represent physical volumes within the specific anatomical region; (c) identifying, using a second module (e.g., a second machine learning module), a target volume within the initial VOI that corresponds to the target tissue region of the individual; and (d) storing and / or providing a 3D segmentation mask corresponding to the identified target volume for display and / or further processing.

[0082] In certain embodiments, the system has one or more of the features expressly set forth in paragraphs "C"-"D".

[0083] Features of embodiments described in relation to one aspect of the invention may be applicable to another aspect of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] This patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0085] Together with the attached Figure 1 The foregoing and other objects, aspects, features and advantages of the present invention will become more apparent and better understood with reference to the following description, in which:

[0086] Figure 1 is a block diagram showing a method for automatically identifying a 3D volume corresponding to a prostate of an individual within a 3D image and determining an uptake index indicative of radiopharmaceutical uptake therein, according to an illustrative embodiment.

[0087] Figure 2A is an image showing a 3D view of a CT image including a graphical representation of soft tissue according to an illustrative embodiment.

[0088] Figure 2B is an image showing a 3D view of a CT image including a graphical representation of a bone according to an illustrative embodiment.

[0089] Figure 2C is an image showing a 3D view of a CT image including a graphical representation of bone overlaid with a SPECT image and a graphic representation of an identified tissue volume corresponding to a prostate organ within a pelvic bone of an individual, according to an illustrative embodiment.

[0090] Figure 2D is an image showing a 3D view of a CT image including a graphical representation of bone overlaid with a SPECT image and a graphic representation of an identified tissue volume corresponding to a prostate organ within a pelvic bone of an individual in accordance with an illustrative embodiment.

[0091] Figure 2E is an image showing a 3D view of a CT image including a graphical representation of a bone, the CT image being overlaid with a SPECT image, according to an illustrative embodiment.

[0092] Figure 3A is a collection of images, each image showing a 2D cross-sectional view of a 3D CT image overlaid with a graphic representing an identified initial volume of interest (VOI) within the 3D CT image, according to an illustrative embodiment.

[0093] Figure 3B is a collection of images, each image showing a 2D cross-sectional view of a 3D CT image overlaid with a graphic representing an initial volume of interest (VOI) identified within the 3D CT image, according to an illustrative embodiment.

[0094] Figure 4A is a collection of images showing a CT image of an individual and a 2D cross-sectional view of a cubic region identified as an initial volume of interest within the CT image, according to an illustrative embodiment.

[0095] Figure 4B is a collection of images showing a CT image of a subject and 2D cross-sectional views of identified tissue volumes corresponding to the subject's pelvic bone and prostate.

[0096] Figure 5A is a collection of images showing a CT image of an individual and a 2D cross-sectional view of a cubic region identified as an initial volume of interest within the CT image, according to an illustrative embodiment.

[0097] Figure 5B is a collection of images showing a CT image of a subject and 2D cross-sectional views of identified tissue volumes corresponding to the subject's pelvic bone and prostate.

[0098] Fig. 6A is a collection of images showing a CT image of an individual and 2D cross-sectional views of a cubic region identified as an initial volume of interest within the CT image, according to an illustrative embodiment.

[0099] Figure 6B is a collection of images, each image showing a 2D cross-sectional view of a 3D CT image overlaid with a graphic representing an initial volume of interest (VOI) identified within the 3D CT image, according to an illustrative embodiment.

[0100] Figure 6C is a collection of images, each image showing a 2D cross-sectional view of a 3D CT image overlaid with a graphic representing an initial volume of interest (VOI) identified within the 3D CT image, according to an illustrative embodiment.

[0101] Figures 7A-7E A block diagram of a CNN module architecture (localization network) is provided for identifying a volume of interest (e.g., VOI) corresponding to a pelvic region within a CT image of an individual according to an illustrative embodiment (where the VOI is subsequently processed by a second CNN module for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region).

[0102] Figure 7F-7J A block diagram of a CNN module architecture (segmentation network) for processing previously identified VOIs for accurate segmentation of the prostate and / or other tissues in the pelvic region according to an illustrative embodiment is provided.

[0103] Fig. 8A is a collection of images, each showing a 2D cross-sectional view of a 3D CT image with graphical overlays representing identified tissue volumes corresponding to three different pelvic bones (left and right hip bones and sacrum) according to an illustrative embodiment.

[0104] Figure 8B is an image showing a 3D view of a 3D CT image with graphical overlays representing identified tissue volumes corresponding to three different pelvic bones (left and right hip bones and sacrum) according to an illustrative embodiment.

[0105] Fig.9A is an image showing a 2D cross-sectional view of a 3D CT image with a graphic overlay representing identified tissue volumes corresponding to the pelvic bones (left and right hip bones) and prostate of an individual, according to an illustrative embodiment.

[0106] Fig. 9Bis a collection of images showing different 2D cross-sectional views of a 3D CT image with graphical overlays representing identified tissue volumes corresponding to a subject's pelvic bone, gluteal muscles, rectum, prostate, and bladder, according to an illustrative embodiment.

[0107] Fig.10 is an image showing a 2D cross-sectional view of a low quality 3D CT image overlaid with a graphic representing identified tissue volumes corresponding to the pelvic bone and prostate of an individual according to an illustrative embodiment.

[0108] Fig.11 is a schematic diagram showing an exemplary architecture according to an illustrative embodiment, wherein image segmentation for identifying a prostate volume as described herein is performed by a dedicated module.

[0109] Fig.12 is a block flow diagram showing an exemplary architecture of a module for performing CNN-based image segmentation according to an illustrative embodiment.

[0110] Fig.13 is a block flow diagram showing the structure of a CNN according to an illustrative embodiment.

[0111] Fig.14 is a block flow diagram showing the structure of a CNN for auxiliary prediction according to an illustrative embodiment.

[0112] Fig.15 is a schematic diagram illustrating crosstalk between a subject's bladder and prostate, according to an illustrative embodiment.

[0113] Fig.16A is a screenshot of a graphical user interface (GUI) for reviewing patient image data showing a window for selecting an individual for analysis and / or review of data in accordance with an illustrative embodiment.

[0114] Fig. 16B is a screenshot of a graphical user interface (GUI) for reviewing patient image data showing a window for selecting an individual for analysis and / or review of data and graphical control elements for initiating processing of patient image data and reviewing patient image data according to an illustrative embodiment.

[0115] Fig. 16C is a screenshot of a graphical user interface (GUI) for reviewing patient image data showing a window for selecting an individual for analysis and / or review of data and graphical control elements for initiating processing of patient image data and reviewing patient image data according to an illustrative embodiment.

[0116] Fig.17Ais a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment, showing a collection of windows including images each showing a different 2D cross-sectional view of a subject's CT image overlaid with the subject's SPECT image and a graphic representing identified tissue volumes.

[0117] Fig. 17B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment, showing a collection of windows including images each showing a different 2D cross-sectional view of a subject's CT image overlaid with the subject's SPECT image and a graphic representing identified tissue volumes.

[0118] Fig. 17C is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment, showing a collection of windows including images each showing a different 2D cross-sectional view of a subject's CT image overlaid with the subject's SPECT image and a graphic representing identified tissue volumes.

[0119] Fig.17D is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment, showing a collection of windows including images each showing a different 2D cross-sectional view of a subject's CT image overlaid with the subject's SPECT image and a graphic representing identified tissue volumes.

[0120] Fig.17E is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment, showing a collection of windows including images each showing a different 2D cross-sectional view of a subject's CT image overlaid with the subject's SPECT image and a graphic representing identified tissue volumes.

[0121] Fig.18A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including graphical controls for toggling display of selectable layers and illustrating a user turning off a SPECT image layer.

[0122] Fig.18B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window comprising a collection of images each showing a different 2D cross-sectional view of an individual's CT image, the CT images being overlaid with a graphic representing an identified tissue volume.

[0123] Fig. 18C is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window comprising a collection of images each showing a different 2D cross-sectional view of an individual's CT image, the CT images being overlaid with a graphic representing an identified tissue volume.

[0124] Fig.18D is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window comprising a collection of images each showing a different 2D cross-sectional view of an individual's CT image, the CT images being overlaid with a graphic representing an identified tissue volume.

[0125] Fig.18E is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window comprising a collection of images each showing a different 2D cross-sectional view of an individual's CT image, the CT images being overlaid with a graphic representing an identified tissue volume.

[0126] Fig.18F is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window comprising a collection of images each showing a different 2D cross-sectional view of an individual's CT image, the CT images being overlaid with a graphic representing an identified tissue volume.

[0127] Fig.19A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including graphical control elements for toggling display of selectable layers and illustrating a user turning off a SPECT image layer and a segmentation layer, respectively.

[0128] Fig.19B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a collection of windows containing images each showing a different 2D cross-sectional view of a subject's CT image superimposed with a subject's SPECT image.

[0129] Fig.19C is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a collection of windows containing images each showing a different 2D cross-sectional view of a subject's CT image superimposed with a subject's SPECT image.

[0130] Fig.19D is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a collection of windows containing images each showing a different 2D cross-sectional view of a subject's CT image superimposed with a subject's SPECT image.

[0131] Fig. 20A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of soft tissue and superimposed with a SPECT image of the subject.

[0132] Fig. 20Bis a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of soft tissue and superimposed with a SPECT image of the subject.

[0133] Fig. 20C is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of soft tissue and superimposed with a SPECT image of the subject.

[0134] Fig.21A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0135] Fig.21B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0136] Fig.22A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0137] Fig. 22B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0138] Fig. 22C is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a CT image of a subject including a graphical representation of bone and a 3D view overlaid with a SPECT image of the subject and a graphic representation of identified tissue volumes.

[0139] Fig.22D is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a CT image of a subject including a graphical representation of bone and a 3D view overlaid with a SPECT image of the subject and a graphic representation of identified tissue volumes.

[0140] Fig.22Eis a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a CT image of a subject including a graphical representation of bone and a 3D view overlaid with a SPECT image of the subject and a graphic representation of identified tissue volumes.

[0141] Fig.22F is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a CT image of a subject including a graphical representation of bone and a 3D view overlaid with a SPECT image of the subject and a graphic representation of identified tissue volumes.

[0142] Fig.23A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0143] Fig. 23B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0144] Fig.24 is a screenshot of a GUI for reviewing patient image data showing a window containing an individual generated report according to an illustrative embodiment.

[0145] Fig.25 is a block flow diagram showing the workflow of user interaction with a GUI, wherein the user reviews images, segmentation results, and absorption indicators and generates a report, according to an illustrative embodiment.

[0146] Fig.26A is a screen shot of a view of a GUI window that allows a user to upload an image in accordance with an illustrative embodiment.

[0147] Fig.26B is a screen shot of a view of a GUI window showing a list of images uploaded by a user in accordance with an illustrative embodiment.

[0148] Fig.27A is a screenshot of a view of a GUI window showing a patient's checklist in accordance with an illustrative embodiment.

[0149] Fig.27B is a screenshot of a view of a GUI window showing a patient's checklist in accordance with an illustrative embodiment.

[0150] Fig.27Cis a screen shot of a view of a GUI window showing a selected patient and a menu providing the user with options for reviewing study data and generating a report in accordance with an illustrative embodiment.

[0151] Fig.28A is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a CT image of a subject including a graphical representation of bone and a 3D view overlaid with a SPECT image of the subject and a graphic representation of identified tissue volumes.

[0152] Fig.28B is a screenshot of a GUI for reviewing patient image data according to an illustrative embodiment showing a window including an image showing a 3D view of a subject's CT image including a graphical representation of bones and superimposed with a SPECT image of the subject.

[0153] Fig.29A is a screenshot of a GUI for reviewing patient image data showing a window for reviewing determined absorption indices according to an illustrative embodiment.

[0154] Fig.29B is a screenshot of a GUI for reviewing patient image data showing a quality control graphical widget according to an illustrative embodiment.

[0155] Fig.29C is a screen shot of a view of a quality control graphical widget that allows a user to sign off on a generated report in accordance with an illustrative embodiment.

[0156] Fig.29D is a screenshot showing a generated report (eg, an automatically generated report) according to an illustrative embodiment.

[0157] Fig.29E is a screen shot of a view of a quality control graphical widget displayed in response to user input of an unapproved automated determination of an absorption index according to an illustrative embodiment.

[0158] Fig.29F is a screenshot of a portion of a generated report showing a graphical indication of rejected quality control in accordance with an illustrative embodiment.

[0159] Figure 29G is a screenshot of a quality control graphical widget that allows a user to manually update one or more values ​​for determining an absorbance index according to an illustrative embodiment.

[0160] Fig.29H is a screenshot of a quality control graphical widget that allows a user to manually update one or more values ​​for determining an absorbance index according to an illustrative embodiment.

[0161] Fig.30is a block diagram showing a microservices network architecture for performing image segmentation to identify prostate volumes within an image, determining an uptake index, and providing the results to a client, according to an illustrative embodiment.

[0162] Fig.31 is a block flow diagram showing the data flow between microservices of a cloud-based application to process a study by performing image segmentation and determining absorption indicators, according to an illustrative embodiment.

[0163] Fig.32 is a block flow diagram showing communication between microservices and clients of a cloud-based application according to an illustrative embodiment.

[0164] Fig.33 is a block diagram of an exemplary architecture for performing a cloud-based platform including a cloud-based application for performing image segmentation and calculating absorption indicators according to the systems and methods described herein according to an illustrative embodiment.

[0165] Fig.34 is a block diagram showing a process for automatically identifying a 3D target volume within a 3D image and determining an absorption index indicative of radiopharmaceutical absorption therein, according to an illustrative embodiment.

[0166] Fig.35A is a cluster map of clinically insignificant and clinically significant images.

[0167] Fig.35B is a ROC curve determined based on varying the TBR threshold calculated according to the systems and methods described herein.

[0168] Fig.36 is a block diagram of an exemplary cloud computing environment used in certain embodiments.

[0169] Fig.37 is a block diagram of an example computing device and an example mobile computing device used in some embodiments.

[0170] Fig.38A is a screen shot of a GUI used in a conventional 2D slice-based image analysis software package.

[0171] Fig.38B is a screen shot of a portion of a GUI used in a conventional two-dimensional slice-based image analysis software package, showing CT image slices.

[0172] Fig.38C is a screen shot of a portion of a GUI used in a conventional two-dimensional slice-based image analysis software package, showing CT image slices.

[0173] Fig.39 is a block flow diagram showing an AI-assisted image analysis workflow according to an illustrative embodiment.

[0174] Fig.40A is a screenshot of a GUI window indicating completion of automated analysis after uploading a SPECT / CT image according to an illustrative embodiment.

[0175] Fig.40B is a screenshot of a GUI window indicating potential errors in automated analysis after a SPECT / CT image has been uploaded and physician review is required, according to an illustrative embodiment.

[0176] Fig.40C is a screenshot of a GUI window indicating that an automated analysis cannot be completed due to an incomplete data set according to an illustrative embodiment.

[0177] Fig.41 is a screenshot of an image analysis GUI window showing a patient's checklist according to an illustrative embodiment.

[0178] Fig.42A is a screenshot of an image analysis GUI for reviewing patient data showing a 2D slice view of a subject's CT image including graphical representations of bone and tissue overlaid with SPECT image data and a segmentation overlay of the subject's prostate in accordance with an illustrative embodiment.

[0179] Fig.42B is a screenshot of a GUI for reviewing patient data showing a 3D view of a CT image including graphical representations of tissue and bone overlaid with SPECT image data and a segmentation overlay of an individual's prostate gland according to an illustrative embodiment.

[0180] Fig.42C is a screenshot of a portion of a GUI for reviewing patient data illustrating a likelihood severity scale displayed within the GUI in certain embodiments.

[0181] Fig.42D is a screenshot of a GUI for reviewing patient data and allowing the Reader to update values ​​used in TBR calculations in a semi-automated, guided manner, according to an illustrative embodiment.

[0182] Fig.42E is a screenshot of a GUI for reviewing patient data showing updated evaluation results after reader input to update the values ​​used in TBR calculations according to an illustrative embodiment.

[0183] Fig.43A is a screenshot showing a generated report containing results of an assay in a fully automated manner, according to an illustrative embodiment.

[0184] Fig.43B is a screenshot showing a generated report containing results determined in a semi-automated manner.

[0185] Fig.44is a screenshot showing a generated report for an un-evaluable situation according to an illustrative embodiment.

[0186] Fig.45A are cluster maps of clinically insignificant and clinically significant images superimposed with three TBR thresholds.

[0187] Fig.45B Three different TBR thresholds were identified based on ROC curves determined by varying the TBR threshold calculated according to the systems and methods described herein.

[0188] Fig.45C is a graph showing total Gleason scores and calculated TBR values ​​for several patients.

[0189] Fig.45D is a table showing the Gleason scores of cases classified according to TBR values ​​compared with several pre-determined thresholds.

[0190] Fig.46 is a set of three histograms showing the time taken by three independent AI-assisted readers to evaluate patient data from a clinical trial.

[0191] Fig.47 is a collection of graphs demonstrating the consistency and reproducibility of results obtained using embodiments of the image analysis methods described herein to analyze clinical experimental data.

[0192] Fig.48A is a graph of ROC curves comparing results obtained using a single independent AI-assisted reader with three independent human readers.

[0193] Fig.48B is a collection of histograms showing the AUC values ​​for three independent manual readers and three independent AI-assisted readers.

[0194] Fig.49A Data comparing the results of the first manual reader with those of the AI-assisted reader are presented.

[0195] Fig.49B Data comparing the results of a second human reader with those of an AI-assisted reader are presented.

[0196] Fig.49C Data comparing the results of a third manual reader with those of an AI-assisted reader are presented.

[0197] The features and advantages of the present invention will become more apparent from the detailed description set forth below when taken in conjunction with the drawings, wherein like reference characters identify corresponding elements throughout. In the drawings, like numerical symbols generally indicate identical, functionally similar, and / or structurally similar numbers. DETAILED DESCRIPTION

[0198] It is contemplated that the systems, architectures, devices, methods, and processes of the present invention encompass changes and modifications developed using information from the embodiments described herein. Changes and / or modifications of the systems, architectures, devices, methods, and processes described herein may be made as encompassed by the present description.

[0199] Throughout this description, where articles, devices, systems and architectures are described as having, including or comprising specific components, or where processes and methods are described as having, including or comprising specific steps, it is further encompassed that the articles, devices, systems and architectures of the present invention consist essentially of or consist of the components, and that the processes and methods according to the present invention consist essentially of or consist of the processing steps.

[0200] It should be understood that the order of steps or the order in which specific actions are performed is not important, as long as the present invention remains operable. In addition, two or more steps or actions may be performed simultaneously.

[0201] Reference to any publication herein (e.g., reference to any publication in the Background section) is not an admission that the publication serves as prior art with respect to any claim provided herein. The Background section is provided for clarity purposes and is not intended to be an indication of prior art with respect to any claim.

[0202] The documents are incorporated herein by reference as described. If there is any difference in the meaning of a particular term, the meaning provided in the Definitions section above shall prevail.

[0203] Headings are provided for the convenience of the reader - the presence and / or placement of headings is not intended to limit the scope of the subject matter described herein.

[0204] As used herein, "radionuclide" refers to a moiety that contains a radioactive isotope of at least one element. Exemplary suitable radionuclides include, but are not limited to, those described herein. In some embodiments, the radionuclide is one used in positron emission tomography (PET). In some embodiments, the radionuclide is one used in single photon emission computed tomography (SPECT). In some embodiments, a non-limiting list of radionuclides includes 99m Tc, 111 In, 64 Cu, 67 Ga, 68 Ga, 186 Re, 188 Re, 153 Sm, 177 Lu, 67 Cu, 123 I. 124 I.125 I. 126 I. 131 I. 11 C. 13 N. 15 O. 18 F. 153 Sm, 166 Ho, 177 Lu, 149 Pm, 90 Y. 213 Bi, 103 Pd, 109 Pd, 159 Gd, 140 La, 198 Au, 199 Au, 169 Yb, 175 Yb, 165 Dy, 166 Dy, 105 Rh, 111 Ag, 89 Zr, 225 Ac, 82 Rb, 75 Br, 76 Br, 77 Br, 80 Br, 80m Br, 82 Br, 83 Br, 211 At and 192 Ir.

[0205] As used herein, the term "radiopharmaceutical" refers to a compound containing a radionuclide. In certain embodiments, the radiopharmaceutical is used for diagnostic and / or therapeutic purposes. In certain embodiments, the radiopharmaceutical includes a small molecule labeled with one or more radionuclides, an antibody labeled with one or more radionuclides, and an antigen-binding portion of an antibody labeled with one or more radionuclides.

[0206] As used herein, "3D" or "three-dimensional" when referring to an "image" is meant to convey information about three dimensions. A 3D image may be rendered as a data set in three dimensions and / or may be displayed as a collection of two-dimensional representations or three-dimensional representations.

[0207] As used herein, an "image" (e.g., a 3D image of an individual) includes any visual representation, such as a photograph, a video frame, a streaming video, and any electronic, digital, or mathematical analog of a photograph, a video frame, or a streaming video. In certain embodiments, any of the devices described herein include a display for displaying an image or any other result produced by a processor. In certain embodiments, any of the methods described herein include a step of displaying an image or any other result produced via the method.

[0208] As used herein, "subject" means a human or other mammal (eg, rodents (mice, rats, hamsters), pigs, cats, dogs, horses, primates, rabbits, and the like).

[0209] As used herein, "administering" an agent means introducing a substance (e.g., an imaging agent) into a subject. In general, any route of administration can be utilized, including, for example, parenteral (e.g., intravenous), oral, topical, subcutaneous, peritoneal, intraarterial, inhalation, vaginal, rectal, nasal, introduction into cerebrospinal fluid, or instillation into a body compartment.

[0210] As used herein, the terms "filter" and "filtering" (as in "filter function" or "filter") refer to a function that operates on a local portion of an input array (e.g., a multidimensional array) of data (e.g., image data, such as values ​​calculated by a CNN layer), referred to herein as a "subpatch", to calculate a reaction value for a given subpatch. In general, the filter is applied in a sliding window manner across the array to calculate multiple reaction values ​​for the array. Specifically, for a given multidimensional array, a subpatch of the array may be a rectangular region of the array of a specific size (e.g., having the same dimensions as the array). For example, for a 6x 3x 3 array, a given 3x 3x 3 subpatch refers to a given 3x 3x 3 group of adjacent values ​​(e.g., a neighborhood) of the array, such that there are five different 3x3x 3 subpatches in the 6x 3x 3 array (each patch is shifted by one position along the first dimension).

[0211] For example, the filtering function may calculate a reaction value for a given sub-patch of an array using the value of the sub-patch. The filtering function may be applied in a sliding window manner across the array, thereby calculating a reaction value for each of a plurality of sub-patches of the array. The calculated reaction values ​​may be stored in an output array such that a positional correspondence between the reaction values ​​and the sub-patches of the input array is maintained.

[0212] For example, in the first step, starting with a sub-patch at a corner of the input array, the filter can compute a first response value and store the first response value in the corresponding corner of the output array. In some embodiments, in the second step, the filter then computes a second response value for a second sub-patch that is shifted one position along a particular dimension of the input array. The second response value can be stored in the corresponding position in the output array - that is, shifted one position along the same dimension of the output array. The following steps can be repeated for the entire input array along each dimension of the input array: shifting the position of the sub-patch, computing the response value, and storing the response value in the corresponding position in the output array. In some embodiments (e.g., the strided filtering method), the sub-patch for which the filter computes the response value is shifted more than one position at a time along a given dimension, such that the response values for every possible sub-patch of the input array are not computed.

[0213] As used herein, the term "convolutional neural network (CNN)" refers to an artificial neural network in which at least one layer performs one or more filtering functions. As used herein, the term "convolutional layer" refers to a layer of a CNN that receives an input array as input and computes an output array, where the values of the output array are computed by applying one or more filters to the input array. Specifically, in some embodiments, the convolutional layer receives an input array having n + 1 dimensions as input and produces an output array also having n + 1 dimensions. The first n dimensions of the input and output arrays that are operated on by the filtering layers of the CNN are referred to herein as "spatial dimensions". The (n + 1)th dimension of the input is referred to herein as the "input channel" dimension. The size of the input channel dimension is referred to herein as the "number of input channels". The (n + 1)th dimension of the output is referred to herein as the "output channel" dimension. The size of the output channel dimension is referred to herein as the "number of output channels".

[0214] In some embodiments, the convolutional layer computes the response values by applying filters that operate on sub-patches that are smaller than the input array along the spatial dimensions but extend across the entire output channel dimension. For example, an input array of size NxMxLxK0 has three spatial dimensions and K0 output channels. The filters of the convolutional layer can operate on sub-patches having a size of N f xM f xL f xK0, where N f ≤N, M f ≤M and L f ≤L. Generally, the filters of the convolutional layer operate on sub-patches where N f <N, M f <M and / or L f <L. For example, in some embodiments, N f <<N, M f<<M and / or L f <<L.

[0215] Thus, for each of the one or more filters applied by a convolutional layer, the response values computed by a given filter are stored in corresponding output channels. Thus, a convolutional layer receiving an input array having n + 1 dimensions computes an output array also having n + 1 dimensions, where the (n + 1)-th dimension represents the output channels corresponding to the one or more filters applied by the convolutional layer. In this way, the output array computed by a given convolutional layer can be received as input by a subsequent convolutional layer.

[0216] As used herein, the term "size" when referring to a filter of a convolutional layer refers to the size of the spatial dimensions of the sub-patch over which the filter operates (e.g., considering the sub-patch size along the output channel dimension as the total number of output channels). As used herein, the term "size" as in "size of a convolutional layer" when referring to a convolutional layer refers to the size of the filters of the convolutional layer (e.g., each filter of a convolutional layer having the same size). In some embodiments, the filters of a convolutional layer have a number of variable parameters determined via a machine learning training process. In some embodiments, the number of parameters of a given filter is equal to the number of values in the sub-patch over which the given filter operates. For example, a filter of size N f xM f xL f operating on an input array having K0 output channels has N f xM f xL f x K0 parameters. In some embodiments, the filters are implemented in array form, and the response value computed by the filter for a given sub-patch is calculated as the dot product between the filter and the given sub-patch.

[0217] As used herein, the term "fully convolutional neural network (FCNN)" refers to a CNN in which each layer of the CNN is a convolutional layer.

[0218] As used herein, the term "volume" as used in reference to the input or output of a CNN layer refers to the received input array or the output array computed by the CNN layer.

[0219] As used herein, the term "CNN module" refers to a computer-implemented process that implements a particular CNN in order to determine one or more output values ​​for a given input, such as an image (e.g., a 2D image; e.g., a 3D image). For example, a CNN module may receive as input a 3D image of an individual (e.g., a CT image; e.g., an MRI), and for each voxel of the image, determine a value representing the likelihood that the voxel is located within a region of the 3D image corresponding to a representation of a particular organ or tissue of the individual. A CNN module may be software and / or hardware. For example, a CNN module may be implemented entirely as software, or certain functions of the CNN module may be performed via dedicated hardware, such as via an application-specific integrated circuit (ASIC).

[0220] As used herein, the term "tissue" refers to bone (bone tissue) as well as soft tissue.

[0221] The systems and methods described herein provide automated analysis of medical images of an individual to automatically identify regions of interest corresponding to specific organs and / or tissues represented in the image. In certain embodiments, a convolutional neural network (CNN) module is employed to accurately segment the image. In certain embodiments, accurate automated identification of specific organs and / or tissues in an image of an individual allows for the determination of a quantitative index measuring the uptake of a radiopharmaceutical (e.g., a radiolabeled small molecule; e.g., a radiolabeled antibody; e.g., an antigen-binding portion of a radiolabeled antibody) in a specific organ.

[0222] In certain embodiments, the systems and methods described herein facilitate automated identification of regions of interest corresponding to specific organs or tissues where tumors and / or tumor metastases may be present. Radionuclide-labeled molecules that selectively bind to specific tumor cell surface proteins can be used in combination with the methods described herein for imaging tumors. For example, molecule 1404 specifically binds to prostate-specific membrane antigen (PSMA), which is overexpressed on many cancer cells. Molecule 1404 can be labeled with a radionuclide (e.g., 99m Tc) labeling is used for single photon emission computed tomography (SPECT) imaging.

[0223] In certain embodiments, the radiopharmaceutical uptake index is relevant to assessing the disease state and prognosis of the individual.

[0224] A. Identification of prostate volume and determination of absorption indexes

[0225] Figure 1 A process 100 is shown for automatically processing a 3D image to identify a 3D volume corresponding to a prostate gland of an individual and to determine one or more absorption indices indicative of radiopharmaceutical absorption therein.

[0226] refer to Figure 1 In steps 102 and 104 of process 100: The panels of FIG. 2 illustrate CT images showing bone and soft tissue ( Figure 2A and Figure 2B ), and highlight the pelvic bones ( Figure 2C and 2D Green 202 and blue 206) and prostate ( Figure 2C and 2D The highlighted region is automatically generated by the systems and methods described herein and identifies the pelvic bones and prostate. Figure 2E A SPECT image is shown superimposed on a CT image (bright pink and purple areas 252, 254, and 256). It can be seen that the CT image provides detailed anatomical information, while the anatomical / structural information in the SPECT image is more limited. Thus, for example, the CT image can be segmented and the mapping between the voxels of the CT image and the voxels of the SPECT image can be used to identify specific voxels in the SPECT image that correspond to tissue volumes of interest (e.g., prostate, bladder, rectum, and gluteal muscles).

[0227] refer to Figure 1 Steps 106 and 108 of process 100: The method of identifying the prostate volume in the CT image first uses a first machine learning module (e.g., a CNN module) to identify an initial volume of interest within the CT image. This generates a standardized input to a second machine learning module (e.g., a second CNN module), which is responsible for identifying the prostate volume. The second CNN module can also identify other tissue volumes, such as pelvic bones, gluteal muscles, rectum, and bladder. The first and second machine learning modules can be combined and implemented as a single module and / or a single software application. The first and second machine learning modules can also be implemented separately as (e.g.,) separate software applications.

[0228] In certain embodiments, as described herein, the architecture (e.g., the type and arrangement of layers) of the CNNs implemented by the first and second machine learning modules is the same, but the parameters (e.g., the number of filters in each layer and the input size) between the two CNNs may be different. In addition, the two CNNs are trained to perform different functions. Specifically, the first CNN is trained to recognize graphical representations of anatomical regions (e.g., pelvic regions) in various 3D anatomical images. The second CNN is trained to recognize specific target tissues (e.g., prostate) in a more standardized input (initial volume of interest). Training the two CNNs differently in this manner produces different weights and biases for the filters that make up the CNNs and allows them to perform their different functions.

[0229] i. Bounding box generation (positioning machine)

[0230] 3-5 provide examples of processing of various CT images, which first show the identification of the initial volume of interest (VOI) (panel A) and secondly show the segmentation of the prostate and pelvic bones (the prostate volume is represented by the middle blue-green volume). As shown in these examples, the initial CT image size may depend generally on the imaging system used, the imaging protocol followed by the radiologist, and other factors, but the initial VOI size is fairly standard.

[0231] For example, the size of the initial 3D anatomical image can range from as large as 700mmx700mmx1870mm (depth x width x height) (i.e., a full body scan) or even larger to as small as 180mmx240mmx290mm (covering the pelvic bones) or even smaller. In the examples described herein, the minimum test matrix size of the initial 3D anatomical image is 256x256x76 pixels. The bounding box of the examples described herein (e.g., identified by the first CNN) has an approximate size range of 180-220mmx240-320mmx290-380mm. The matrix size range of the bounding box of the examples described herein is 80-220x150-390x50-300 pixels.

[0232] In the example, the 3D anatomical image used to train the first CNN to detect the pelvic region bounding box has the following image dimensions:

[0233] Training images (3D anatomical images):

[0234] Number of rows: 247-512 (sp1.37), size (mm): 340-700

[0235] Number of columns: 319-512 (sp1.37), size (mm): 430-700

[0236] Number of slices: 274-624 (sp3.0), size (mm): 820-1870

[0237] In the example, the 3D anatomical image used to validate the first CNN for detecting bounding boxes has the following image dimensions, with the resulting range of dimensions of the bounding boxes identified by the first CNN:

[0238] Verification images (3D anatomical images):

[0239] Number of rows: 256-512, Size (mm): 500-600

[0240] Number of columns: 256-512, size (mm): 500-600

[0241] Number of slices: 76-427, size (mm): 380-1070

[0242] Bounding Box:

[0243] Number of rows: 82-222, Size (mm): 180-220

[0244] Number of columns: 148-386, size (mm): 240-320

[0245] Number of slices: 50-295, size (mm): 290-380

[0246] The following are three example methods for automatically generating a bounding box (cube) of the pelvic region from an initial 3D anatomical image for subsequent processing to identify the prostate in detail.

[0247] In the first approach, the first CNN receives as input a grayscale CT image (single input channel) and outputs the coordinates of the relative corners of the bounding box.

[0248] In the second method, the grayscale CT image is processed via thresholding to produce a thresholded image that roughly identifies the pelvic region. In this second method, the first CNN receives two input channels - one is the grayscale CT image and the other is the thresholded image. The second CNN outputs the coordinates of the relative corners of the bounding box.

[0249] In a third approach, the first CNN is essentially a rough version of the second CNN - that is, the first CNN identifies the prostate, pelvic bones, and sacrum (and background). A bounding box is generated using the identified pelvic bones (e.g., by drawing a minimal box that fits the pelvic bones, or perhaps adding some buffer distance). The difference here is that the output of the first CNN is not just the coordinates of the vertices of the cube. In this approach, a likelihood value is automatically determined for each voxel of the image, which gives a likelihood about how the voxel is classified - for example, whether the voxel is (for example) prostate, left / right pelvic bone, sacrum, or background.

[0250] In an illustrative embodiment of this third method, the localization and segmentation networks are very similar. The localization network segments the left and right pelvic bones, sacrum, and background (a total of 4 categories) in a very downsampled image. Based on this rough segmentation of the pelvic bones, a bounding box is created. The segmentation network then segments the pelvic bones, prostate, and background at a higher resolution within this bounding box. The localization CNN and the segmentation CNN have the same architecture. The difference is what input shape it requires, where the localization network resizes the image to size (81, 68, 96), and the segmentation CNN resizes the pelvic image (as the output from the localization network) to size (94, 138, 253). The number of convolutional filters is also different. The segmentation network has 20 convolutional filters in the first layer, while the localization network has only 8. The size and number of filters in subsequent layers are scalable, as shown in the flowchart of this article. In order to create a bounding box after the localization network, the method finds the corners of the first segmentation and draws a square around it with a margin. The input image to the localization network is smaller because it has been downsampled to a lower resolution, while the segmentation processes a cropped version of the CT image in the original resolution.

[0251] Fig. 6A An example showing method 1. Figure 6B and Figure 6C Shows the comparison between Method 1 and Method 3.

[0252] Fig. 7A -E shows an example architecture 700 of the first CNN network described herein. The CNN module architecture (localization network) is used to identify a volume of interest (e.g., VOI) corresponding to the pelvic region within a CT image of an individual (where the VOI is subsequently processed by a second machine learning module (e.g., a second CNN module) for more detailed segmentation / identification of the prostate and / or other tissues within the pelvic region).

[0253] ii. Prostate volume and additional tissue volume determination (single segmentation machine)

[0254] Figure 7F -J shows an example architecture 750 of the second CNN network described herein. As described above, the second CNN operates on the VOI to identify the prostate volume and various additional tissue volumes. The additional tissue volumes may include left / right pelvic bones, sacrum, bladder, gluteal muscles, and rectum. In contrast to performing a binary classification (e.g., where voxels are simply identified as prostate or background), identifying multiple tissue volumes improves the accuracy of the classification method.

[0255] Fig. 8A showing the left and right pelvic bones (blue, 806, and yellow, 802) and sacrum (red, 804) in 2D, and Figure 8B Shows the left and right pelvic bones in 3D.

[0256] Fig.9A The left and right pelvic bones (red, 902, and yellow, 906) and the prostate (blue, 904) are shown.

[0257] Fig. 9B Shown are segmentations of the left and right pelvic bones (two lighter green regions, 952 and 956), prostate (dark green, 958), bladder (greenish beige, 960), rectum (brown, 962), and gluteal muscles (blue, 964, and red, 966).

[0258] Fig.10 Showing low-quality CT images where segmentation still works.

[0259] iii. Split module architecture

[0260] Data input and output

[0261] Fig.11 An example architecture 1100 is shown in which image segmentation as described herein is performed by a segmentation service module 1104 that receives requests from a client-facing module Pioneer Web 1102 to process an image (eg, perform segmentation as described herein).

[0262] Fig.11 The specific exemplary architecture 1100 shown in is used in the embodiments of the image segmentation and analysis system and method described herein, specifically a cloud-based software application, called Pioneer. Pioneer is a software device that uses MIP-1404 SPECT / CT image data to assist in the evaluation and characterization of prostate cancer in the prostate gland. The software uses artificial intelligence to automatically segment the image data into different anatomical regions and then analyze the volume region (ROI) of interest. Pioneer extracts quantitative data from the ROI to assist in determining the presence or absence of clinically significant prostate cancer.

[0263] Fig.11The architecture and module organization shown in and other architectures and module organizations described herein with respect to Pioneer may be applicable to other imaging modalities and / or other radiopharmaceuticals. For example, various radiopharmaceuticals suitable for SPECT imaging are described in Section M below, “Imaging Agents” of this document. Various 3D functional imaging modalities may also be used to image the absorption of radiopharmaceuticals in individuals, and combined with 3D anatomical imaging modalities (such as, but not limited to, CT imaging), and analyzed via an adapted version of the method described herein with respect to Pioneer. For example, various nuclear medicine imaging modalities (such as PET imaging) may be used to image the absorption of radiopharmaceuticals. As with SPECT imaging, PET imaging may be combined with CT imaging to obtain an image set comprising PET images and CT images—a CT / PET image. Therefore, the methods described herein with respect to Pioneer and CT / SPECT images may also be applicable to CT / PET images. Various radiopharmaceuticals suitable for PET imaging are also described in Section M below, “Imaging Agents”.

[0264] See again Fig.11 ,like Fig.11 As shown in , Pioneer is implemented as a cloud-based service using Amazon Web Services, where requests are processed using Simple Notification Service (SNS) and Simple Queue Service (SQS) communications. The segmentation service 1104 listens for requests on a dedicated SQS queue. Events occurring in the Pioneer network 1102 are published to a dedicated SNS topic "Pioneer Events" 1106. To allow parallel asynchronous processing, a fan-out pattern can be used so that SNS messages are sent to a topic and then replicated and pushed to multiple Amazon SQS queues 1108a, 1108b, where the segmentation service is the consumer.

[0265] The segmentation service 1104 can download input data (e.g., image data) from a source Uniform Resource Locator (URL) provided by a requester (e.g., Pioneer Network 1102) and linked to a data set in a local file system. In some embodiments, the input data includes pre-processed 3D anatomical images, such as CT images and 3D functional images, such as SPECT scans. Together with the 3D anatomical and functional images, image metadata may also be included.

[0266] In some embodiments, the CT and SPECT images have a resolution that meets specific requirements established for the system. For example, CT images with a resolution range of 1.0 to 2.2 mm in the x and y directions and between 1.0 and 5.0 mm in the z (slice) direction may be used. In some embodiments, SPECT images with a resolution range of 2.9 to 4.8 mm in all directions are used.

[0267] Once the segmentation service 1104 has performed image segmentation as described herein to identify the prostate volume and any additional tissue volumes in a particular CT image, the segmentation service 1104 provides segmentation output data identifying the prostate volume within the particular CT image for storage and / or further processing (e.g., display in a GUI; e.g., calculation of an absorption index). In some embodiments, the request submitted to the segmentation service 1104 includes a target URL specifying a storage location for the segmentation output data. Thus, the segmentation service 1104 can upload the segmentation output data to the target URL included in the request.

[0268] The segmentation output data may include data such as a segmentation mask module that identifies each voxel of a 3D anatomical image (e.g., a CT image) as corresponding to a specific tissue region or background, as determined by a second machine learning module described herein. The segmentation mask module may include or be used to determine a corresponding segmentation mask for each specific tissue region, which identifies voxels of the CT image classified as belonging to the specific tissue region. For example, the segmentation mask module may be stored in a .tiled.png format, and the voxels are labeled with different digital labels that identify different specific tissue regions or backgrounds. An example set of labels for various tissue regions is shown below, where voxels identified as background are labeled with 0:

[0269] {'prostate':1,

[0270] 'Left gluteus maximus': 2,

[0271] 'Right gluteus maximus': 3,

[0272] 'Rectum':4,

[0273] 'Bladder':5,

[0274] 'Sacrum and coccyx':6,

[0275] 'Left hip':7,

[0276] 'Right hip':8

[0277] }

[0278] In certain embodiments, the segmentation mask module identifies the prostate volume and a reference volume (eg, left gluteus muscle) and labels all other voxels as background.

[0279] In some embodiments, segmentation mask metadata is also included in the segmentation output data. The segmentation mask metadata may be used to store information about the particular image being processed, such as the number of voxels in each direction (eg, number of rows, columns, and slices) and their spacing.

[0280] In some embodiments, the segmentation service 1104 also uses the results of the segmentation (e.g., the intensity of the voxels of the identified prostate and reference volumes and 3D functional images (e.g., SPECT images)) for quantification. Such quantification is discussed in more detail herein (e.g., in the subsection entitled "Absorption Indicators" below). For example, to calculate the target to background ratio (TBR) value of the individual as described herein, the segmentation service 1104 can identify the prostate maximum intensity voxel together with the background value. The prostate maximum intensity voxel is the voxel identified as having the maximum intensity corresponding to the identified prostate volume within the set of voxels of the 3D functional image. In some embodiments, as described herein, the voxels corresponding to the identified prostate volume in the 3D functional image are corrected for crosstalk (also called drainage) from the individual's bladder, and after correcting for bladder crosstalk, the prostate maximum intensity voxel is identified. The background value used to determine the TBR value is the average of the intensities of multiple voxels of the 3D functional image corresponding to the reference volume within the 3D anatomical image (e.g., the volume corresponding to the individual's left hip muscle). As described herein, all voxels corresponding to an identified reference volume are typically used to calculate a background value.

[0281] Data corresponding to the quantization performed by the segmentation service 1104 may be included in the segmentation output data. For example, the quantization results may be included in a quantization results dictionary as shown in the .json format example below:

[0282] {

[0283] "prostate_max": 988.0,

[0284] "prostate_max_pos":{

[0285] "z": 92,

[0286] "y": 74,

[0287] "x": 67

[0288] },

[0289] "background": 9.056859205776174,

[0290] }

[0291] The keywords of the quantization result dictionary in the above example are as follows: "prostate_max" stores the intensity of the prostate maximum intensity voxel; "prostate_max_pos" stores the position of the prostate maximum intensity voxel in the SPECT image; and "background" stores the background value.

[0292] Example segmentation service architecture including CNN module organization and interaction

[0293] Fig.12 An example architecture 1200 of a module for performing CNN-based image segmentation as described herein is shown. The example architecture 1200 includes a first machine learning module (in Fig.12 ) 1204, and a second machine learning module for identifying the prostate volume, additional tissue volume, and reference volume described herein (referred to as the "localization machine" in Fig.12 The input and output of the first and second machine learning modules are also shown, as well as several additional modules, including an auxiliary machine learning module (in Fig.12 The single-segmentation machine (aux) is referred to as the auxiliary single-segmentation machine in the text, and is referred to as the auxiliary single-segmentation machine in short) 1212 and a module 1214, which is used to merge the outputs of the second machine learning module (the basic single-segmentation machine) and any auxiliary machine learning module (auxiliary single-segmentation machine) and perform post-processing. Fig.12 The outputs generated by the module tissue shown in FIG. 1 include a segmentation mask module 1216 that identifies various tissue volumes in a 3D anatomical image (eg, a CT image) and segmentation metadata 1218 .

[0294] As described herein, the first machine learning module 1204 receives the CT image 1202 as input and generates a bounding box that identifies an initial VOI corresponding to the pelvic region of the individual. The second machine learning module 1208 receives the initial VOI and the CT image 1202 as input. The second machine learning module 1208 can add a cropping margin to the initial VOI (e.g., adding a margin around the initial bounding box to expand the bounding box) and provide a cropping end point to assist the machine learning module. As described herein, the second machine learning module 1208 identifies the prostate volume within the initial VOI. The second machine learning module 1208 can also identify additional tissue volumes corresponding to additional tissue regions of the individual (e.g., left gluteal muscle, bladder, and left and right hip bones). As described herein, the tissue volume corresponding to the left gluteal muscle of the individual can be used as a reference volume to (e.g.) calculate the background value in the TBR value of the individual. The tissue volume corresponding to the bladder of the individual can be used to correct the bladder crosstalk of the intensity value of the voxel of the SPECT image, for example, as described in Part B "Correcting Bladder Intensity Penetration" herein. The identified tissue volume corresponding to the left and right hip bones of the individual can be used for post-processing.

[0295] In certain embodiments, as described herein, performance is improved by using one or more auxiliary machine learning modules 1212 in addition to the second machine learning module 1208, wherein the auxiliary machine learning module 1212 performs image segmentation in a similar manner as the second machine learning module, identifies prostate volume and additional tissue volume (e.g., Fig.12 These auxiliary tissue volumes identified by the auxiliary machine learning module may be merged with the base tissue volumes (prostate volume and any additional tissue volumes) identified using the second machine learning module 1208 via module 1214.

[0296] In some embodiments, the first and second machine learning modules and any auxiliary machine learning modules are implemented as trained CNN modules. Each trained CNN module can be represented (e.g., via computer code) as a directory containing an associated trained neural network with model structure and weights. Neural network libraries (e.g., Keras) can be used to represent trained CNN modules in this manner. The dictionary representing a particular CNN module may also include metadata describing any preprocessing to be performed on the image before feeding it as input to a particular CNN module. Metadata describing how the neural network model is built and trained may also be included.

[0297] Thus, a particular CNN module may perform steps such as (1) loading and preprocessing an image, given an image file name and image metadata; (2) feeding the preprocessed image through an associated trained neural network (e.g., as included in a directory representing the particular CNN module) to obtain a raw prediction output (e.g., for each of one or more voxels of the CT image, a map including a likelihood (e.g., probability) that the voxel belongs to a particular class as determined by the trained neural network); and (3) post-processing the raw prediction output. As described herein, a first machine learning module 1204 may post-process the raw prediction output generated by its associated trained neural network to determine a cropping endpoint corresponding to a bounding box of an initial VOI. A second machine learning module 1208 may post-process the raw prediction output generated by its associated trained neural network to determine a segmentation mask module that marks voxels of the CT image with values ​​(e.g., numerical values) that indicate a particular tissue volume to which they correspond or that identify the voxel as background, as determined by the second machine learning module.

[0298] First Machine Learning Module ( Fig.12 An exemplary model structure of the CNN module implemented by the positioning machine 1204 is shown in Fig.13 The second machine learning module ( Fig.12 An exemplary model structure of a CNN module implemented by a single segmentation machine) 1208 is shown in Fig.14 middle.

[0299] Example first machine learning module (positioning machine) implementation

[0300] As described herein, a first machine learning module identifies an initial VOI—e.g., a "bounding box" that identifies a region in a CT image that corresponds to the individual's pelvic region (which includes the individual's pelvic bones) and tissue regions (e.g., prostate, bladder, rectum, and left and right gluteal muscles). Identification of the bounding box (e.g., cropping endpoints) is provided to a second machine learning module, as well as any auxiliary machine learning modules, allowing them to limit their processing to a small, specific target region of the CT image, rather than having to operate and process the entire CT image. This reduces the amount of data processed by these machine learning modules that perform computationally intensive fine segmentation, thereby improving their performance and efficiency.

[0301] In an exemplary implementation of a first machine learning module according to method 3 as described herein, Fig.12 The localization machine 1204 in extracts a bounding box (to identify an initial VOI) from a coarse segmentation of the CT image performed by its associated trained neural network (CNN) (referred to as the localization CNN).

[0302] In some embodiments, the CT image is preprocessed to prepare for segmentation by the localization CNN. A configuration file may be used to specify preprocessing steps and parameters. Preprocessing may include steps such as cropping the CT image to remove one or more regions corresponding to ambient air, normalizing the intensities of voxels in the CT image, and resizing the CT image (e.g., to produce a resized CT image that conforms to a fixed input size expected by the localization CNN). The intensity normalization preprocessing step adjusts the voxel intensities of the CT image to produce a specific mean and standard deviation of the intensities of all voxels of the CT image. For example, the voxel intensities of the CT image can be normalized by subtracting a first fixed value and then dividing by a second fixed value to produce a CT image in which the average intensity of all voxels is a specific mean value (e.g., 0) and the standard deviation of the intensities of all voxels is a specific standard deviation value (e.g., 1). The resizing step may be performed by sampling the CT image.

[0303] The localization CNN receives the preprocessed CT image as input and passes it through a series of layers such as Fig.13As shown in . The output of the localization CNN is a coarse segmentation represented by a first segmentation mask module that classifies voxels of the preprocessed CT image as belonging to one of four categories, for example, three of which represent specific tissue regions: (i) sacrum and coccyx, (ii) left hip bone, (iii) right hip bone, and a fourth category representing (iv) background (e.g., everything else). This coarse segmentation is obtained from the original prediction map generated by the free localization CNN. The original prediction map has the same shape - the same set of voxels as the preprocessed CT image received as input, but has four channels. Each channel corresponds to a specific classification category - (i) sacrum and coccyx, (ii) left hip bone, (iii) right hip bone, and (iv) background. Each channel represents a probability map of its corresponding classification category. That is, for each voxel of the preprocessed CT image, each specific channel corresponding to a specific classification category includes a likelihood value representing the probability that the voxel belongs to the specific category (e.g., representing the physical volume within the tissue region represented by the category, or representing the background). Therefore, for each voxel, the sum of the likelihood values ​​of the voxels in each channel is 1.

[0304] In order to determine a first segmentation mask module that classifies each voxel as belonging to a particular category, each voxel of the preprocessed CT image is assigned to the category (e.g., tissue region or background) having the maximum likelihood value (e.g., probability) of the voxel. The determined first segmentation mask module has the same shape as the preprocessed CT image, and each voxel is labeled with a value identifying the category to which it is classified. For example, a particular voxel may be labeled with a numerical value, such as 1, 2, or 3 corresponding to categories (i), (ii), and (iii) as described above, or a value of 0 if it is classified as a background voxel.

[0305] A bounding box identifying the initial VOI may be determined from the first segmentation mask module as a minimum box (e.g., a rectangular volume) that includes all voxels labeled as belonging to categories (i)-(iii). Coordinates of the identification bounding box (e.g., coordinates corresponding to the diagonal corners of the rectangular volume of the bounding box) are determined and output as the cropping end point. In certain embodiments (where the pre-processed CT image input to the localization CNN is a resized version of the initial CT image), the coordinates of the identification bounding box are converted to the coordinate system of the initial CT image and output as the cropping end point.

[0306] Example Second Machine Learning Module (Segmentation Machine) Implementation

[0307] exist Fig.12 In the exemplary architecture of , the second machine learning module (also referred to as the basic single segmentation machine 1208 ( Fig.12The single segmentation machine (base) in CT performs high-resolution segmentation of CT images to identify a prostate volume corresponding to an individual's prostate, as well as additional tissue volumes corresponding to specific tissue regions (e.g., left and right gluteal muscles, rectum, bladder, sacrum and coccyx, and left and right hip bones).

[0308] In certain embodiments, the CT image is preprocessed to prepare for segmentation by a second machine learning module. As with the positioning machine 1204, a configuration file may be used to specify preprocessing steps and parameters. Similar to the method described above with respect to the positioning machine 1204, the preprocessing step may include normalization of the intensity of the CT image. The CT image may also be cropped using a cropping endpoint that identifies an initial VOI (bounding box) and output by the positioning machine 1204 to produce a reduced fixed-size preprocessed CT image input for providing as input to a trained neural network (trained CNN) associated with and implemented in a single segmentation machine 1208. Preprocessing may also include a resizing step.

[0309] Similar to the localization CNN of the localization machine, the single segmentation machine implements a trained CNN, called the segmentation CNN, which receives the preprocessed CT image as input. The segmentation CNN passes the preprocessed CT image through a series of layers and outputs a second raw prediction map. Similar to the raw prediction map produced by the localization CNN, the second raw prediction map includes multiple channels, each of which corresponds to a different specific category to which each voxel of the input CT image will be classified. Each channel represents a probability map of the classification category to which it corresponds, and for each voxel of the preprocessed CT image, includes a likelihood value corresponding to the probability (determined by the segmentation CNN) that the voxel belongs to the category.

[0310] As described above, the second original prediction map output by the segmentation CNN contains a channel corresponding to the prostate of the individual. The second original prediction map may include other channels corresponding to various other tissue regions (e.g., left gluteal muscle, right gluteal muscle, rectum, bladder, sacrum and coccyx, left hip bone and right hip bone). The second original prediction map may also include a background channel.

[0311] Some variants of high-resolution CNNs give predictions from multiple levels of the network, resulting in multiple probability maps for each class. These multiple probability maps are called auxiliary predictions. Fig.13 The CNN model structure shown in corresponds to a model structure that does not produce auxiliary predictions, and Fig.14 The CNN model structure shown in is for a model that generates auxiliary predictions. In some embodiments, although multiple filters in the top layer (e.g. Fig.13 The 20 filters in Fig.14The 28 filters in the CNN may vary, but the multiple filters in the subsequent lower layers double each layer down. In some embodiments, the segmented CNN produces a single probability map and does not produce any auxiliary predictions. In some embodiments, the segmented CNN produces auxiliary predictions.

[0312] In some embodiments, when the segmentation CNN generates auxiliary prediction maps, the likelihood values ​​for each class included in the second original prediction map and each auxiliary probability map are averaged together so that for a particular voxel, a single likelihood value for each class is determined.

[0313] In some embodiments, in order to determine a second fine segmentation mask module that classifies each voxel as belonging to a specific category, each voxel of the preprocessed CT image is assigned to a highest probability value. In some embodiments, a number of labels in the fine segmentation mask module are reduced so that only certain tissue volumes (prostate volume and reference volume (e.g., left hip muscle volume)) are included in the fine segmentation mask module.

[0314] Auxiliary single cutting machine

[0315] In some embodiments, one or more auxiliary machine learning modules are used to generate auxiliary fine segmentation mask modules similar to the fine segmentation mask modules generated by the second machine learning module. These auxiliary fine segmentation mask modules identify the same set of tissue volumes as the fine segmentation mask modules generated by the second machine learning module. In this way, the second machine learning module generates a base fine segmentation mask module, and one or more auxiliary machine learning modules each generate an auxiliary fine segmentation mask module, thereby providing classification of parallel groups of voxels for the CT image.

[0316] In some embodiments, for each category representing a specific tissue region, a corresponding basic fine segmentation mask of or determined using a basic fine segmentation mask module (e.g., a volume in a CT image determined as corresponding to a specific tissue region via a basic single segmentation machine) and one or more corresponding auxiliary fine segmentation masks (e.g., each of which identifies a volume in the CT image determined as corresponding to the same specific tissue region via an auxiliary single segmentation machine) are merged to produce a merged fine segmentation mask. For example, for a specific category, certain voxels in one or more auxiliary fine segmentation masks may be identified (e.g., marked) as belonging to the specific category, but not belonging to the set of voxels in the basic fine segmentation mask identified as belonging to the specific category. These voxels may be added (e.g., by marking them as such) to the set of voxels identified as belonging to the specific category in the basic fine segmentation mask to produce a final merged fine segmentation mask. For example, in Fig.12In the architecture shown in , a base single segmentation machine 1208 generates a base fine segmentation mask, and one or more auxiliary single segmentation machines 1212 each generate an auxiliary segmentation mask. The base fine segmentation mask generated by the base single segmentation machine 1208 and the one or more auxiliary fine segmentation masks generated by the auxiliary single segmentation machine 1212 are combined by module 1214 to generate a final fine segmentation mask module 1216 containing a final fine segmentation mask for each class.

[0317] In some embodiments, one or more final fine segmentation masks are filtered so that only the largest connected parts are retained. In some embodiments, when filtering to retain only the largest connected components of the prostate segmentation mask (e.g., the final fine segmentation mask of the volume of the CT image of the prostate corresponding to the individual is identified), a subgroup of the connected components of the prostate segmentation mask is considered. The subgroup only includes the following components: (i) having a mass center between the mass centers of the hip bones in the left-right direction of the axial plane (x direction), and (ii) being located in a defined bounding box so that only left and right hip bones are contained.

[0318] iv. Absorption index

[0319] refer to Figure 1 Step 110 of process 100 , determining one or more absorption indices from which a diagnosis or staging of a condition (e.g., prostate cancer) can be informed or automatically presented. One or more absorption indices are determined using a prostate volume identified within a VOI of a 3D functional image and a 3D anatomical image. For example, the amount of radiopharmaceutical in the prostate of an individual can be calculated based on the intensity values ​​of voxels of the 3D functional image corresponding to the prostate volume identified within the VOI of the 3D anatomical image. This may involve calculating a sum (e.g., a weighted sum), an average, and / or a maximum of the intensities of the voxels of the 3D functional image representing the physical volume occupied by the prostate of the individual. In certain embodiments, this involves calculating a normalized value based on the intensity values ​​of the voxels of the 3D functional image corresponding to a reference volume identified within the 3D anatomical image. For example, the normalized value of the intensity identified in the prostate can be normalized using the intensity values ​​of one or more voxels corresponding to the gluteal muscles or another reference volume within the VOI (or, in certain embodiments, elsewhere within the 3D anatomical image). The absorption index can then be converted into identifying whether an individual has prostate cancer and / or quantifying an individual's risk of having prostate cancer, and / or staging the disease (e.g., tracking as part of the disease over time), which can be used by a medical practitioner, for example, to recommend treatment options and / or monitor the efficacy of an administered therapy.

[0320] Target to Background Ratio

[0321] In certain embodiments, the one or more absorption indices determined include a target to background ratio (TBR) value of the individual. Determining the TBR value includes (i) determining a target intensity value using intensity values ​​of one or more voxels of a 3D functional image (e.g., a PET or SPECT image) corresponding to the identified prostate volume within an initial VOI of a 3D anatomical image (e.g., a CT image), and (ii) determining a background intensity value using intensity values ​​of one or more voxels of a 3D functional image corresponding to an identified reference volume. The TBR value is calculated as a ratio of the target intensity value to the background intensity value.

[0322] Specifically, in certain embodiments, the target intensity value is the maximum value of the intensity of the voxel corresponding to the 3D functional image (such as PET or SPECT image) of the identified prostate volume. As described above, the prostate maximum intensity voxel corresponds to the voxel identified as having the maximum intensity within the set of voxels corresponding to the 3D functional image of the identified prostate volume. The target value can then be regarded as the intensity of the prostate maximum intensity voxel. The background intensity value can be calculated as the average value of the intensity of multiple voxels corresponding to the 3D functional image of the reference volume identified. As described herein, the gluteal muscle (such as the left gluteal muscle) or a part thereof can be used as the reference volume identified.

[0323] For example, a TBR value can be calculated from a SPECT / CT image of a subject recorded after administration of a radiopharmaceutical (e.g., 1404) to the subject. The SPECT image corresponds to a 3D functional image and the CT image corresponds to an anatomical image. The image segmentation method described herein can be used to identify a prostate volume within the CT image and a reference volume corresponding to the left hip muscle of the subject. The prostate segmentation mask and the left hip muscle segmentation mask can be used to identify the prostate volume and the left hip reference volume, respectively.

[0324] The prostate volume segmentation mask identifies the voxels of the CT image, which are classified as belonging to the prostate of an individual based on the machine learning segmentation method described herein. In certain embodiments, in order to identify the voxels of the SPECT image corresponding to the prostate volume identified in the CT image, the voxels of the prostate segmentation mask are mapped to the corresponding voxels of the SPECT image. Since the prostate segmentation mask identifies the voxels in the CT image, it can have a resolution different from the SPECT image (for example, because SPECT and CT images can have different resolutions). Interpolation (for example, bilinear interpolation) and / or sampling can be used to match the resolution of the prostate segmentation mask with the resolution of the SPECT image, so that each voxel of the prostate segmentation mask is mapped to the specific corresponding voxel of the SPECT image. In this way, a SPECT prostate mask is obtained, which identifies the voxels in the SPECT image corresponding to the prostate volume identified in the CT image. A SPECT left hip muscle mask can be obtained in a similar manner, which identifies those voxels in the SPECT image corresponding to the left hip muscle reference volume identified in the CT image.

[0325] The SPECT left buttock muscle mask can be used to determine background intensity values. Specifically, the intensities of the voxels identified by the SPECT left buttock muscle mask in the SPECT image are extracted and divided into quartiles. The background value is calculated as the average of the extracted left buttock muscle voxel intensities that fall within the first and third quartiles. Other methods for calculating background values, such as calculating the overall mean or median of all extracted left buttock muscle voxel intensities, can also be used. It was found that the above-mentioned method of dividing the extracted left buttock muscle voxel intensities into quartiles is more stable than calculating the overall mean for outliers and more accurate than calculating the median (for example, because the intensities are discretized and many of them have the same value). The measured background intensity value can be output and stored in the result dictionary, for example under the keyword 'background'.

[0326] The target intensity value can be calculated using the SPECT prostate mask. The prostate maximum intensity voxel can be identified as a voxel of the SPECT prostate mask and has the maximum SPECT image intensity. The target intensity value is determined as the intensity of the prostate maximum intensity voxel. The target value intensity value and the position of the prostate maximum intensity voxel can be stored. For example, the result dictionary can store the target intensity value and the prostate maximum intensity voxel position under keywords such as "prostate_maximum value" and "prostate_maximum value_position" respectively. In certain embodiments, before determining the target intensity value, the bladder "crosstalk" or "drainage" of the intensity of the voxel corresponding to the identified prostate volume in the SPECT image is corrected as described in Part B below, so that, for example, the maximum intensity stored is the maximum correction intensity and the prostate maximum intensity voxel is the voxel of the SPECT prostate mask with the maximum corrected intensity.

[0327] In certain embodiments, the prostate cancer status of an individual can be determined by comparing the measured TBR value of the individual with one or more threshold values. Specifically, the measured TBR value can be compared with a specific threshold value (e.g., a cutoff threshold value) to distinguish patients with clinically significant prostate cancer (e.g., assigned to a clinically significant state) from those patients who do not have clinically significant prostate cancer (e.g., assigned to a clinically insignificant state). Example 4 below shows an exemplary method for determining a TBR threshold based on a TBR value calculated from a reference image to obtain the desired sensitivity and specificity.

[0328] B. Correction of bladder strength and back

[0329] Fig.15 15 is a schematic diagram illustrating crosstalk of imaging agent intensity from the bladder to the prostate of an individual. In the figure, reference 1502 is the bladder, 1506 is the crosstalk / intensity of the discharge from the bladder, and 1504 is the prostate. Certain imaging agents including PSMA binders have high absorption in the bladder, which may affect the identification of diseased tissue (e.g., prostate cancer). For example, absorption of radionuclide-labeled PSMA binders by the bladder can cause 3D functional image scattering and can reduce the accuracy of imaging agent intensity measured in the prostate located near the bladder. By training a second CNN for detailed segmentation of the prostate and bladder of an individual, the 'drainage' or 'crosstalk' effect and / or other effects caused by the absorption of the imaging agent by the bladder can be accurately and automatically accounted for. In addition, by training a second CNN for identifying reference areas (e.g., gluteal muscles) in 3D anatomical images, the imaging agent intensity measurement can be more accurately weighed / normalized and the accuracy and diagnostic value of absorption measurements in the prostate of an individual can be improved.

[0330] i. Bladder expansion

[0331] In some embodiments, bladder crosstalk correction includes a step in which the identified bladder volume is expanded (e.g., via morphological expansion) in two iterations. This expansion can be used to inhibit high intensities very close to the segmented bladder for determining target intensity values ​​when calculating TBR values ​​(e.g., selected as maximum intensity), and also for stabilizing the bladder suppression method described below.

[0332] ii. Bladder inhibition calculation

[0333] In some embodiments, a bladder suppression method is used to remove intensity leakage from the bladder to other areas of the functional image. The amount of suppression, that is, the intensity leakage removed from a particular voxel of the functional image, depends on the distance of the voxel to the core bladder region corresponding to the area of ​​the bladder with high intensity voxels.

[0334] In certain embodiments, bladder suppression occurs if the maximum functional image intensity within a volume of the functional image identified as corresponding to a bladder (e.g., corresponding to an identified bladder volume within a 3D anatomical image; e.g., as identified via a bladder mask) is greater than a specific multiplier value multiplied by a determined background intensity value. As described herein, the background intensity value may be determined based on the intensity of voxels of the 3D functional image corresponding to an identified reference volume within the 3D anatomical image (e.g., a gluteal muscle volume). For example, bladder suppression may occur if the maximum functional image intensity within the region corresponding to the identified bladder volume is 15 times the determined background value (e.g., the bladder to background ratio should be at least 15).

[0335] In certain embodiments, bladder suppression is calculated from and applied to a portion of the 3D functional image that is within a bladder suppression bounding box of a specific size around the identified bladder volume. For example, a bladder suppression bounding box may be determined that contains the bladder with a margin of a predetermined size in a particular direction (e.g., 40 mm in the vertical direction) and has the same number of voxels in other directions.

[0336] For example, after masking the region of the 3D functional image corresponding to the prostate, the core bladder region may be determined as a region of the 3D functional image within a bladder suppression bounding box that contains voxels having intensities within a certain fraction of the maximum intensity within the bounding box (e.g., excluding voxels of the masked prostate region). For example, the core bladder region may be determined as a region containing voxels having intensities greater than or equal to 50% of the maximum intensity within the bladder suppression bounding box. The core bladder region may include high intensity regions that were not included in the original bladder mask.

[0337] In certain embodiments, one or more bladder intensity dumping functions are determined to perform bladder suppression, and thereby correct bladder crosstalk of the voxel intensities of the 3D functional image. For example, the 3D functional image can be cropped using a bladder suppression bounding box, and the determined background intensity value is subtracted from the intensity of the voxels in the cropped image region. Sample intensities are then collected to determine how the bladder intensity (e.g., the intensity resulting from the absorption of a radiopharmaceutical in the bladder of an individual) decreases as one moves away from the bladder. Samples are collected starting at the very top, very front, very right, and very left of the core bladder region, and then moving directly upward, forward, right, or left, respectively, one voxel at a time. If the edge of the cropped image region is encountered, then an extrapolated intensity can be used as a sample, such as a linear extrapolation from the first two or more samples.

[0338] The intensity samples provide four curves (e.g., a collection of sampled intensity data points) of intensity reduction from the bladder core, to each of which a template function can be fitted to establish four bladder intensity discharge functions that model the variation of bladder intensity with distance from the core region. Prior to further analysis (e.g., fitting), outlier removal can be performed, for example, to remove the curve farthest from the other curves. A template function such as an nth-order polynomial (e.g., a 5th-order polynomial) is fitted to the data points in the remaining three curves, thereby generating a function that models the variation of bladder intensity discharge with distance from the core bladder. This bladder intensity discharge function describes how much should be subtracted from the original intensity to obtain a corrected intensity, depending on the distance to the core bladder region.

[0339] In certain embodiments, to reduce the risk of the bladder intensity discharge function underestimating the discharge in the bladder direction (the discharge may vary to some extent between different directions), the distance is multiplied by a dilation factor (e.g., in the range of 1 to 2; e.g., 1.2) before fitting to the sampled intensity data points, so that the fitted bladder intensity discharge function is stretched. The risk of underestimating the bladder intensity discharge may also be reduced by multiplying the sampled intensity data points by the scaling factor. The scaling factor may be a variable scaling factor having a value that depends on the bladder to background intensity ratio. For example, a scaling factor may be used that has a specific value when the bladder to background ratio is sufficiently high and increases (e.g., linearly) as the bladder to background ratio decreases. For example, when the bladder to background ratio is sufficiently high, the sample intensity may be multiplied by a factor of 1.2, and for lower bladder to background ratios, linearly increased from 0 for a bladder to background ratio of 15. This method of multiplying the sampled intensity data points by the scaling factor also improves the robustness of the bladder suppression method.

[0340] In some embodiments, the range of bladder suppression, i.e., the farthest distance from the bladder to which it is applied, is where the decay rate of the base function is sufficiently small. If the decay rate never becomes sufficiently small, then the distance is chosen as the length of the intensity sample vector. In some embodiments, bladder suppression is ensured to be non-negative at all distances.

[0341] Once determined, the bladder intensity discharge function can be evaluated at various voxel locations of the 3D functional image, such as at voxel locations corresponding to locations within the prostate of an individual, to determine a bladder intensity discharge value for each specific voxel location. Thus, the intensity of a particular voxel can be corrected for bladder crosstalk by subtracting the determined bladder intensity discharge value for the specific voxel location from the intensity of the specific voxel. In this manner, the voxel intensity can be corrected for bladder crosstalk at various locations within the 3D functional image, such as at locations corresponding to an identified prostate volume (e.g., identified in a 3D anatomical image).

[0342] iii. Corrected maximum intensity and position of the prostate

[0343] In certain embodiments, the bladder suppression methods described herein are used to correct the intensity values ​​of voxels of a 3D functional image corresponding to an identified prostate volume within a 3D anatomical image, and determine an absorption index such as a TBR value using the corrected intensity values.

[0344] C. Visualizing Image Data and Calculating Absorption Indicators

[0345] Fig.16A A window showing a GUI interface with two individuals (patients) listed. The user can select the desired individual to process and view its image data.

[0346] Fig. 16B An exemplary GUI with graphical control elements for processing and manipulating individual data is shown, which appears after a user selects the individual "John Doe." The user clicks on a selectable button element to complete processing of the selected individual's image data.

[0347] exist Fig. 16C In , after processing of the image data of the selected individual is complete, the user clicks another selectable button to view the processed image data and the calculated absorption index.

[0348] exist Figures 17A-17E , the 2D viewer is displayed. Figures 17A-17E A collection of 2D cross-sectional views of the CT image data are displayed overlaid with the SPECT image and a graphic representation of identified tissue volumes (specifically, the prostate and pelvic bone). The user can scan the cross-sectional slices as shown in the figure.

[0349] The CT images and SPECT images are reproduced as selectable layers that can be switched on and off. Graphics representing the identified tissue volumes are also reproduced as selectable segmentation layers. Fig.18A In , the user switches off the display of the SPECT image layer so that only the CT image and segmentation layer are displayed. Figures 18B-18F , the user scans a cross section.

[0350] exist Fig.19A In the example, the user switches the display of the SPECT image layer on, and the segmentation layer off. Figures 19B-19D The user is shown scanning the cross-sectional view again, this time showing only the CT image and SPECT image layers.

[0351] Figures 20A-20C A rotatable and sliceable 3D viewer for viewing CT images and overlaying with SPECT image data is presented, wherein the CT images include graphical representations of soft tissue.

[0352] Figures 21A-21B A rotatable and sliceable 3D viewer is also shown, this time showing only a graphical representation of the bones in the CT image.

[0353] Figures 22A-22F Demonstrates how the user can use the 3D viewer to examine image data and automatically identify the prostate and other additional tissue areas. The user slices down to focus on Fig. 22B The pelvic region in the CT image is highlighted and the segmentation layer is turned on to display a graphic representing the identified prostate volume and pelvic bone volume superimposed on the CT image as colored areas. The user can slice and rotate the image to view bright spots of SPECT image intensity in the prostate region (purple volume).

[0354] Fig.23A and 23B The viewer is shown but with a black background to improve contrast and simulate a radiologist's scan.

[0355] Fig.24 Present reports (eg, automatically generated reports) with various ingestion metrics.

[0356] D. User Interface, Quality Control, and Reporting

[0357] In certain embodiments, the systems and methods described herein are implemented as part of a GUI-based workflow that allows a user (e.g., a medical practitioner (e.g., a radiologist, a technician)) to upload an image of a patient (individual), initiate an automated analysis according to the methods described herein, wherein a prostate volume is identified in a 3D anatomical image and used to determine an absorption index using corresponding voxels of a 3D functional image. The user may then view the results of the automated analysis, including the determined absorption index and any prognostic value determined therefrom. The user may be guided through a quality control workflow, wherein the user selects to approve or disapprove the results of the automated analysis, and if the results are approved, a report is generated for the patient. The quality control workflow may also allow the user to manually adjust and update the results of the automated analysis, such as via interaction with a GUI, and generate a report based on the manually updated results.

[0358] Fig.25 An example workflow 2500 for analyzing SPECT / CT images in certain embodiments is shown. Fig.25 As shown in , users can upload content that conforms to a specific accepted standard format 2502, specifically Fig.25 The example of the DICOM standard 2504 SPECT / CT image. Fig.26A An exemplary GUI window 2600a of a web-based portal is shown that allows a user to upload 2504 an image. Fig.26B exhibit Fig.26A2600b, wherein several images have been selected for uploading and checked for compliance with the DICOM standard. One image is identified as not compliant with the DICOM standard and is indicated as not being able to be uploaded 2602b.

[0359] See also Fig.25 In another step 2506, the user may view a list of patients who have uploaded images. Fig.27A View 2700a of a GUI window is shown listing patients by anonymous numeric identifier. Fig.27B exhibit Fig.27A Another view 2700b of a GUI window is shown in which a row of the patient list corresponding to a particular patient is highlighted for selection. Fig.27C Another view 2700c of the GUI is shown in which, upon selection of the row corresponding to a particular patient, a menu listing studies performed for the particular patient is displayed, including selectable buttons allowing the user to review image data for the study and generate a report.

[0360] In some embodiments, user review of the image data and the results of any automated analysis performed using the image data are prerequisites for generating a report. User review of the image data and automated analysis may be required to verify the accuracy of the image segmentation. For example, Fig.25 As shown in , after viewing the patient list 2506 and selecting a patient, in the next step 2508, the user reviews the patient's image data and the results of the automated processing as described herein.

[0361] Specifically, in Fig.25 In the review step 2508 of the workflow 2500 shown in FIG. 25, the user examines the SPECT / CT image data of the selected patient using a GUI-based viewer. The user can choose to view and examine the SPECT / CT image data as a collection of 2D slices or a 3D rendering. Fig.28A A view of a GUI viewer 2800 is shown in which SPECT / CT image data is displayed as a collection of 2D slices 2802a. Fig.28B A view of a GUI-based viewer 2800 is shown in which SPECT / CT image data is displayed as a 3D rendering 2802b.

[0362] like Fig.28A and Fig.28BAs shown in , the GUI viewer 2800 displays the SPECT image and the CT image as selectable layers overlapping each other. The user can select one layer at a time to view a separate SPECT and / or CT image, or can select both the CT layer and the SPECT layer to view the SPECT image superimposed on the CT image. The user can adjust the opacity of the SPECT image, for example, to emphasize or de-emphasize the SPECT image features superimposed on the CT image. The user can also view a segmentation layer that shows the location of various tissue regions identified within the CT image via the second machine learning module, as described herein. In this way, the user can verify the image segmentation performed by the second machine learning module, for example, via visual inspection of the CT image layer and the segmentation layer.

[0363] The user can also view and verify one or more absorption indicators determined by the automated image analysis methods described herein. For example, Fig.29A GUI view 2900 is shown in which a panel 2902 of the GUI displays the patient's automatically determined TBR value 2904. The panel also displays a determined TBR-based classification 2906 indicating that the TBR value is associated with clinically significant prostate cancer.

[0364] In certain embodiments, to aid user verification of the measured absorption index, a graphical element is displayed within the GUI to indicate the location of the voxel of the identified prostate volume. For example, as described herein, when the TBR value is calculated as the ratio of the target intensity value to the background intensity value, the maximum intensity of the voxel in the SPECT image corresponding to the identified prostate volume is identified. Thus, the GUI may display a graphical element indicating the location of the voxel of the SPECT image that corresponds to the identified prostate volume and has the maximum intensity compared to other voxels in the SPECT image that correspond to the identified prostate volume. In this way, the location of the maximum SPECT intensity voxel corresponding to the location within the prostate volume is displayed to the user. The user can then visually verify that this maximum SPECT intensity voxel is indeed located within the individual's prostate, for example, by examining the relationship of the graphical element compared to the CT image. For example, as Fig.29A As shown in , a set of cross-shaped graphic elements 2950, ​​2952 and 2954 identifying the locations of the maximum SPECT intensity voxels are superimposed on the 2D slice shown in the image viewer.

[0365] See also Fig.25 In another step 2510, the user may choose to generate a report summarizing the analysis performed on the patient using the uploaded SPECT / CT images and be guided through a quality control workflow. Fig.29A As shown in , the user may select (e.g., click) a generate report button 2908 of the GUI. When the user selects the generate report button 2908, as shown in Fig.29B 2910. The quality control graphic widget 2910 can guide the user to check whether various acceptance criteria are met. For example, the quality control graphic widget 2910 guides the user to check whether the image requirements are met, and the target value and background value used to determine the TBR value are correct. The user can approve the automated analysis results by selecting button 2912, or can disapprove the automated analysis results by selecting button 2914.

[0366] like Fig.25 As shown in , after the user approves the quality control check 2512, the automated evaluation results 2514 are used to generate a report 2532 for the patient. Fig.29C As shown in , before a report is generated, a widget 2916 may be presented to the user requesting user confirmation of quality control approval. Fig.29D An example report 2900d is shown. The report may include identifying quality control criteria 2918, and identifying a user 2920 who approved the quality control and signed the report.

[0367] like Fig.25 As shown in FIG. 2 , if the user disapproves 2516 the automated determination of an absorption index (e.g., TBR), the quality control widget may begin a guided evaluation 2518 workflow. In the guided evaluation workflow, the user may manually update the values ​​used to determine one or more absorption indices via manual interaction with the GUI. The user may also provide input indicating that the patient's images cannot be used to accurately determine the absorption index, which input may also be received via the quality control widget. For example, following user input corresponding to disapproved quality control, the user may be presented with a prompt such as Fig.29E The GUI elements shown in . Fig.29E The GUI element 2900e shown in allows the user to select, via selection button 2922, whether the patient's TBR value cannot be determined, or whether he or she wants to manually update the target and background intensity values ​​used for TBR value calculation via selection button 2924.

[0368] For example, if the user determines (e.g., via visual inspection of the image) that the quality of the image is too poor to be used to accurately determine a TBR value, the user may select button 2922 in GUI element 2900e to identify the situation as not evaluable. Fig.25 As shown in , after the user receiving the situation identifies it as not evaluable 2520, the situation is marked as not evaluable 2530, and a report 2532 is generated that identifies the quality control as rejected. Fig.29F An example of a graphic and text 2900f is shown that may be included in the report to identify a quality control as rejected.

[0369] In some embodiments, the user may choose to update the target and / or background intensity values ​​used to determine the TBR value via manual interaction with the GUI. Once user input 2522 indicating that the user wishes to update the target and / or background intensity values ​​is received, the user is guided to manually set the target value 2524 and / or background value 2526. Figure 29G View 2900g shows a manual input graphical widget displayed to the user to allow the user to set target and / or background values. The user can click button 2926 to manually set the target intensity value via interaction with the GUI. Upon selecting button 2926, the user is presented with a voxel selection graphical element that allows the user to select a voxel of the SPECT image to be used as the maximum intensity voxel for calculating the TBR value. For example, the user may be provided with a movable crosshair that allows the user to locate a location in the SPECT and CT images, such as Fig.29A The user can move the crosshairs and / or click a location within the viewer to select a specific voxel to use as the maximum intensity voxel. Fig.29H As shown in , the updated view 2900h of the manual input graphic widget may show the intensity value 2930 of the selected maximum intensity voxel, which will be used as the updated target intensity value for calculating the updated TBR value.

[0370] The user may also select button 2928 to set a background value for calculating the TBR value. Upon selecting button 2928, the user may be presented with a voxel selection element and directed to select a plurality of voxels identified (e.g., based on visual inspection of CT image layers and / or segmentation layers displayed in a GUI viewer) as belonging to the left gluteus muscle. Since the background intensity value used to calculate the TBR is an average of the intensities of a plurality of voxels of the SPECT image corresponding to physical locations within the left gluteus muscle, the user may be directed to select a sufficient number of voxels to ensure that an accurate background intensity value is determined. For example, if Fig.29H As shown in view 2900h of the manual input graphic widget shown in , as the user selects voxels for determining updated background intensity values, multiple samples and updated values ​​of background intensity 2932 may be displayed. In some embodiments, the user is required to select at least a predetermined number of samples (e.g., 100), and the displayed background intensity value and number of samples 2932 include a visual indication (e.g., a color change) to let the user know once a sufficient number of samples have been selected.

[0371] See also Fig.25 Once the user completes manual entry of the updated target value and / or the updated background intensity value, an updated TBR value is calculated using the updated values ​​and stored as a semi-automatic evaluation 2528. A report 2532 may then be generated using the semi-automatic evaluation results.

[0372] E. Example cloud-based architecture and service organization

[0373] In certain embodiments, the systems and methods described herein are implemented as cloud-based applications. Cloud-based applications can use multiple modules (e.g., client-oriented modules that provide an interface for receiving input from a user and presenting image data and results to the user) to implement various functions. The client-oriented modules can in turn interface with other modules (e.g., a segmentation service module that performs automated image segmentation to identify prostate volume and calculate absorption indices as described herein).

[0374] Fig.30 An example microservices network architecture 3000 is shown for implementing the systems and methods described herein for cloud-based applications. As described herein, a particular example microservices network is used in an example cloud-based application called Pioneer. Fig.30 In the exemplary microservice network architecture 3000 of FIG. 3000 , the microservice pioneer network 3016 is a client-oriented module that serves the client 3026 to provide it with an interface (e.g., through service code instructions, such as javascript and HTML) for interacting with the stored images and analysis results. The pioneer network 3016 also communicates with other microservices, such as Fig.30 . Authentication service 3014 records events and saves event logs in log file storage database 3012. Cognito 3020 stores and authenticates users. Authentication service 3022 tracks user groups and consumer product registrations. Registration service 3024 is a front-end service that allows users to register to use, for example, a cloud-based system. Slice box 3004 stores image files in standardized DICOM format, including 3D anatomical images such as CT images and 3D functional images such as SPECT and PET images. Images are stored in image storage database 3002. Image service 3006 reads images (e.g., DICOM files) and stores images and image metadata in a specific (e.g., convenient; e.g., standardized) format. Segmentation service 3010 performs automated image segmentation and absorption index determination as described herein. Segmentation service 3010 obtains image data prepared by image service 3006 from data storage database 3008, performs image segmentation and absorption index determination, and stores the results in data storage database 3008.

[0375] Fig.30The architecture shown in can be used to implement the applications and platforms described herein on various data centers, including publicly available data centers. The data center provides infrastructure in the form of servers and networks, and provides services such as network connectivity, communication, authentication, logging, and storage. The architecture 3000 for the application uses a series of functional units with limited scope, called microservices. Each microservice handles a collection of independent tasks, such as image storage, calculation of risk index, identification of medical image types, and other tasks. Services (e.g., microservices) can communicate with each other using standard protocols such as the Hypertext Transfer Protocol (HTTP). Fig.30 As shown in the architecture 3000, organizing applications into a network of microservices allows each part of the platform to be scaled individually to meet high demand and ensure minimal downtime. In some embodiments, the architecture allows components to be modified or replaced without affecting other parts of the application, or the platform and other parts including the application.

[0376] Fig.31 is to demonstrate the use of Fig.30 Block flow diagram of an example data flow 3100 for automated segmentation and analysis of CT / SPECT images using a microservice network as shown in . Images uploaded by a user are stored in a slice box 3110. A user can initiate processing of a study (e.g., an image of a patient) to initiate automated image segmentation and analysis, for example, via client 3102a via interaction with Pioneer Network 3104a. Processing of a study can also be started automatically (e.g., after uploading an image; e.g., at regular time intervals). Fig.31As shown in , once the study process is initiated, the pioneer network 3104a interacts with the image service 3106 to initiate the preparation of the image data (e.g., preprocessing; e.g., formatting) for automated segmentation and analysis. The image service 3106 saves the preprocessed image in the database S3 3112. The pioneer network 3104a initiates image segmentation and analysis through the segmentation service (e.g., by sending an SQS message). The segmentation service 3108 obtains image data including 3D anatomical images and 3D functional images (e.g., CT / SPECT images) from S3 3112. The segmentation service 3108 performs image segmentation to identify the tissue volume (including the prostate volume) within the 3D anatomical image, and uses the identified tissue volume and 3D functional image to calculate one or more absorption indicators as described herein. The results of the automated image segmentation and absorption index determination are stored in S3 3112. The segmentation service sends a callback back to the pioneer network 3104a, indicating whether the result was successfully obtained or whether an error / abnormality occurred. Upon receiving the callback, the pioneer network 3104a publishes the updated status of the study on the channel Pubsub 3118. Multiple pioneer network services 3104a and 3104b can receive the status updates of the study and notify the clients 3102a and 3102b interacting with them of the status updates.

[0377] Fig.32 The data flow between microservices 3200 is shown, which shows how the client requests the research status and references it in a similar manner as described above. Fig.31 Notify of research status updates. Fig.32 As shown in , a client 3202 may send a study id to a PioneerNet service instance 3204, which receives status updates via a Pub / Sub channel 3212. Callbacks for completed computations (e.g., completed image segmentation and absorption index analysis as described herein) may enter PioneerNet from any segmentation service 3208. The status of completed computations for a study may be saved in a database 3210, and updated status of the study is sent to the Pub / Sub channel 3212. All PioneerNet instances (e.g., 3204 and 3206) subscribe to status updates and may provide status updates (e.g., notifications of completed computations for a study) to the clients 3202 with which they interact.

[0378] In certain embodiments, for example, as described above with respect to Fig.30 and Fig.31 As mentioned, implementations of the systems and methods described herein may include an image service module that performs image data preprocessing and formatting, such as to provide appropriately formatted image data to a segmentation service that performs automated image segmentation and absorption index determination.

[0379] The image service module can preprocess different images from various methods, including 3D functional and 3D anatomical images, to standardize and format the images. For example, preprocessing of 3D functional images (e.g., nuclear medicine images, such as SPECT images) may include performing basic compliance checks and interpreting voxel intensity values ​​as specified in a standard format (e.g., DICOM PS3NM image IOD). The frames of the 3D functional images can be arranged in a specific order (e.g., head to toe order) corresponding to the direction along the individual. The position can be adjusted to represent the outer corner of the first voxel. Attributes from the 3D functional image that may be needed in further processing can also be extracted.

[0380] Preprocessing of 3D anatomical images (e.g., CT images) may also include performing basic compliance checks and interpreting voxel intensity values ​​as specified in a standard format (e.g., DICOM PS3 NM image 10D). Slices of the 3D anatomical image may be arranged corresponding to a specific order along the direction of the individual (e.g., head to toe order). Clipping boundaries may also be determined to remove one or more regions of the 3D anatomical image that correspond to the air surrounding the imaged patient. The position may be adjusted to represent the outer corners of the first voxel. Attributes from the 3D anatomical image that may be needed in further processing may also be extracted.

[0381] Fig.33 Architecture 3300 is shown that illustrates how a cloud-based application implementing the image segmentation and analysis methods described herein can be combined with other systems for performing other types of image analysis to provide a cloud-based platform that clients / users can use for multiple image analysis applications. Fig.33 The architecture 3300 shown in includes a microservice group 3320 shared between two or more applications. The left figure 3310 and the right figure 3330 show the microservices in the two applications. The microservice network 3330 implements a version of the pioneer cloud-based application as described herein, and provides automated analysis of CT / SPECT images, calculation of absorption indicators (e.g., TBR), and generation of reports. The microservice network 3310 shown in the left figure implements a web-based application called aBSI, which analyzes whole-body scans obtained with a gamma camera and calculates an automated bone scan index (BSI). Additional details about aBSI and automated BSI determination are provided in U.S. patent application No. 15 / 794,220 filed on October 10, 2017, the entire contents of which are incorporated herein by reference.

[0382] F. Identify other issues of concern Target Volume and calculated absorption index

[0383] about Fig.34, the methods described herein can be used to identify other target volumes, such as lungs, lymph nodes, bones, liver. Other image types can be used. Various radiopharmaceuticals can be used to generate 3D functional images. For example, in some embodiments, 1404 is an imaging agent for SPECT images, and in some embodiments, PyL is an imaging agent for PET images. In one embodiment, tissues (e.g., lymph nodes) throughout the body of an individual are imaged to detect disease (e.g., tumors). The total number and / or size of tumors and / or another indicator related to the spread and / or amount of diseased tissue (e.g., cancer) are automatically determined using the methods described herein. For example, in method 3400, at steps 3402 and 3404, a 3D anatomical image and a corresponding 3D functional image are received, respectively. A first CNN can be used to effectively identify 3406 one or more sub-regions of the initial 3D anatomical image so that a second CNN performs a more detailed segmentation 3408, and the corresponding 3D functional image is analyzed within the identified one or more sub-regions to quantify overall imaging agent absorption and / or provide a calculated indicator 3410 indicating the level and / or extent of disease in the individual.

[0384] G. Example 1: Automated detection and quantification of prostate PSMA uptake using SPECT / CT imaging

[0385] In this example, 99mTc MIP-1404, a small molecule inhibitor of prostate-specific membrane antigen (PSMA), is used to detect clinically significant disease in prostate cancer. The goal of this example is to develop a deep learning model for automated detection and quantification of prostate MIP-1404 uptake using SPECT / CT images.

[0386] A deep learning algorithm for automatic segmentation of the prostate and pelvic bone using CT images was developed based on a convolutional neural network. The algorithm is designed to handle high-dose and low-dose CT images and whole-body and partial-body fields of view without manual interaction. The training material consists of 100 diagnostic CT images (all male) that are completely and differently segmented for the relevant anatomical regions. The algorithm was validated on the MIP-1404 Phase II study including 102 high-risk prostate cancer patients, who all underwent PSMA imaging before radical prostatectomy. All validation scans were manually quantified using the OsiriX medical image viewer (Pixmeo SARL) by measuring the maximum absorption in the circular ROI placed inside the prostate in the slice, and the area with the highest absorption value was visually determined. The automatic algorithm uses its volume segmentation to measure the absorption of each voxel in the prostate, and temporarily stores the maximum absorption. The Pearson correlation coefficient is used to evaluate the consistency between the manual and automated quantification of prostate absorption.

[0387] The algorithm based on the training material has 2.7 million parameters and is optimized using Adam (a variant of gradient descent). In the test set, 1404 images from 34 patients (33%; 34 / 102) were excluded due to excessive CT artifacts, incomplete data, and / or data format issues. The computational time for evaluable patients (N=68) was 13 seconds (per case) on commercial hardware. The automated maximum absorption values ​​were significantly correlated with the manually obtained values ​​in the prostate (r=0.95, 95% CI=[0.91, 0.97]; slope=0.89, 95% CI=[0.80, 0.98]; p<0.0001). The algorithm is fully automated and deterministic, resulting in 100% reproducibility.

[0388] This example demonstrates the feasibility of objective and automated measurement of MIP-1404 uptake in the prostate.

[0389] H. Example 2: Technical and Clinical Performance of Automated Segmentation and Absorption Index Determination

[0390] Example 2 demonstrates the evaluation of the technical and clinical performance of an embodiment of the automated image segmentation and absorption index determination method described herein implemented via an exemplary cloud-based application called Pioneer. In order to establish the hypothesis generation algorithm of Pioneer, MIP-1404 SPECT / CT image data from two clinical studies were used, including healthy volunteers (Phase 1 study: MIP-1404-1301) and patients with prostate cancer and with histopathological data available after radical prostatectomy (Phase 2 study: MIP-1404-201). Data comparison of receiver operating characteristic curves (ROC), sensitivity, specificity, positive and negative predictive values ​​generated by Pioneer with those obtained by traditional manual readings demonstrates that Pioneer can be a useful tool to assist radiologists in MIP-1404 image interpretation; thereby facilitating the evaluation of patients with prostate cancer.

[0391] i. Technical performance

[0392] As detailed in OTHA-2262 and OTHA-2263, segmentation and quantification performance criteria and motivation analysis tests of the algorithm were prospectively defined. A total of 61 1404 SPECT / CT images and standard of care readings by nuclear medicine physicians were used as a benchmark for evaluating analytical performance. All manual readings were completed independently and blindly according to technical performance acceptance criteria.

[0393] The prostate segmentation performance test showed an average dice of 0.77 and a standard deviation of 0.012. The lower end point of the unilateral 95% confidence interval was 0.75, which is higher than the predefined threshold (0.70) in OTHA-2262. The background (left gluteus maximus) segmentation performance test showed an average dice of 0.94 and a standard deviation of 0.002. The lower end point of the unilateral 95% confidence interval was 0.94, which is higher than the predefined threshold (0.80) in OTHA-2262.

[0394] Only four of a total of 61 (6.6%) target locations predicted by the automated Pioneer software were classified as defective by the human experts (nuclear medicine readers). The results were better than those specified in OTHA-2263 (10% rejection).

[0395] ii. Clinical performance

[0396] In a retrospective specific analysis of clinical performance, 14 healthy volunteers (Phase 1 study MIP-1404-1301) and 105 prostate cancer subjects (Phase 2 study MIP-1404-201) were combined in a single cohort for analysis using Pioneer. Subjects who received prior therapy for prostate cancer in MIP-1404-201 were excluded.

[0397] Images with CT artifacts or that could not be reconstructed for automated analysis were also excluded.A total of 75 images were evaluated from 61 prostate cancer patients and 14 normal volunteers.

[0398] The automated evaluation of Pioneer is evaluated for the histopathological true standard of those who have been diagnosed with prostate cancer and have undergone radical prostatectomy after MIP-1404 SPECT / CT imaging. For healthy volunteers, it is assumed that they do not have prostate cancer based on their normal PSA and pelvic MRI when studying. Spearman's rho is used to evaluate the correlation with Gleason score. The area under the receiver operating characteristic (ROC) curve is used to determine the performance of the algorithm for detecting prostate cancer in the prostate gland. Sensitivity and specificity are measured from the most preferred threshold / cutoff value of the ROC curve.

[0399] Automated quantitative evaluation of 1404 images correlated with Gleason score (rho: 0.54; p<0.0001). The ROC curve showed an AUC of 0.80 (95% CI: 67-94). The optimal threshold for distinguishing clinically significant prostate cancer from clinically insignificant cancer or normal prostate based on binary TBR was determined to be 25. Using this threshold, Pioneer used 1404 SPECT / CT images from the MIP-1404-1301 and MIP-1404-201 studies to distinguish clinically significant prostate cancer from clinically insignificant prostate cancer or normal prostate with a sensitivity of 75% and a specificity of 80%.

[0400] Thus, Pioneer improves upon the manual read by providing (i) more objective and reproducible read performance across all diagnostic endpoints, and (ii) point estimates of key diagnostic performance parameters (e.g., sensitivity, specificity) that demonstrate >70% consistency using histopathology as the true standard.

[0401] I. Example 3: For 1404 An example cloud-based software for automated and semi-automated analysis of CT / SPECT images Item

[0402] Example 3 is an example of a cloud-based software platform, called Pioneer, which implements an embodiment of the automated image analysis method described herein. Pioneer is a cloud-based software platform implemented in accordance with regulatory and data security standards, where users can upload 1404 SPECT / CT image data, view it using two-dimensional and three-dimensional medical image viewers, evaluate it via the user's ordinary Internet browser, and review and export TBR values. The software also provides a quality control workflow, where users can evaluate the quality of the analysis with the option to reject and / or adjust the automated analysis.

[0403] Therefore, under the supervision of the user, the software uses artificial intelligence algorithms to automatically identify and analyze the region of interest (ROI). Pioneer extracts image data from the ROI to provide an objective analysis based on absorption in the prostate and background - target to background (TBR). Since the signal in the prostate gland is often masked by the signal from the bladder (MIP-1404 is excreted in the urine), the ability to segment the bladder and inhibited bladder-related signals is also built into the software, enabling more accurate measurement of prostate signals.

[0404] Pioneer's nonclinical performance data includes a Verification and Validation (V&V) evaluation, including the definition of test methods. Pre-determined acceptance criteria are designed to minimally ensure equivalent performance to the state-of-the-art (human evaluation). Verification includes software unit testing, integration testing, and software system testing with functional testing of all software requirements. A validation process is performed to ensure that the system meets the user requirements specification. V&V test results demonstrate that Pioneer meets its intended use, user, and software requirements.

[0405] J. Example 4: Selection of TBR Thresholds for Clinically Significant Findings

[0406] Example 4 is an example showing how a TBR threshold for classifying a patient's prostate cancer pathology as clinically significant and clinically insignificant can be determined.

[0407] Combine two data sets of SPECT / CT images to select appropriate thresholds. The first data set includes images of healthy individuals taken from the Phase I study of 1404 drugs. This data set originally contained 14 images. According to the method described herein, the segmentation of the prostate in the image is performed, and two images in which the prostate segmentation obviously fails are excluded, resulting in 12 remaining images. The second data set includes images of individuals with prostate cancer derived from the Phase II study of 1404 drugs. Images are divided based on the Gleason grade of the individual of the histopathology from radical prostatectomy. Total Gleason score is 7 points or more and is considered clinically significant, and total Gleason score is 6 points or less and is considered clinically insignificant. This data set originally contained 65 images (63 clinically significant, 2 clinically insignificant), and after excluding an image in which the image does not cover the entire pelvic region and an image in which the prostate segmentation obviously fails, 63 (61 clinically significant, 2 clinically insignificant) remain.

[0408] In total, 14 images without clinically significant pathology and 61 images with clinically significant pathology were used.

[0409] A software package (ctseg package version 1.0.0rcIII) implementing an embodiment of the automated image segmentation and absorption index determination method described herein is used to calculate the TBR value of the image.

[0410] Fig.35A Cluster plots showing clinically insignificant images (<=6 on the x-axis) and clinically significant images (>=7 on the x-axis). A threshold of 25 indicated a sensitivity of 0.77 (lower end point of Jeffreys' one-sided 95% confidence interval was 0.67) and a specificity of 0.71 (lower end point of Jeffreys' one-sided 95% confidence interval was 0.50) on the test data. Fig.35BThe ROC curve based on changing TBR threshold is shown. The point with TBR threshold of 25 is marked as 3502 (red circular marker).

[0411] like Fig.35A and 35B As shown in , a threshold value of 25 provides a value for distinguishing a patient with clinically significant prostate cancer and a patient without clinically significant prostate cancer. However, since there are almost no data points very close to the threshold value, the estimation of sensitivity and specificity is not robust for the small random variation of data. In order to obtain a more robust estimation, R packages 'shape-constrained kernel density estimation (scdensity)' are used to estimate the smooth unimodal density of the log TBR value distribution of clinically insignificant and significant images respectively. Thus, sensitivity and specific estimates are calculated for different TBR thresholds, referring to Table 1.

[0412] Table 1. Robust estimates of sensitivity and specificity for different TBR thresholds.

[0413] TBR threshold Specificity Sensitivity 22 0.68 0.79 23 0.70 0.78 24 0.73 0.76 25 0.75 0.75 26 0.76 0.73 27 0.78 0.72 28 0.79 0.70

[0414] Therefore, based on the analysis described in this example, a TBR threshold of 25 was selected based on the desired specificity and sensitivity.These preliminary studies based on Phase 1 and Phase 2 data demonstrate that the diagnostic accuracy of MIP-1404 SPECT / CT is improved using automated methods compared to manual readings.

[0415] K. Example 5: Using AI to analyze images from a Phase 3 study

[0416] This example demonstrates an embodiment of using the artificial intelligence (AI) based image analysis systems and methods described herein in the context of a Phase 3 multi-center, multi-reader, open-label study. Specifically, this example describes the results obtained via study MIP-1404-3302 (hereinafter referred to as the "3302 study"), which is entitled "Evaluation of 99m TC-MIP-1404 is a Phase 3 study of the safety and efficacy of SPECT / CT imaging and Pioneer as a combination product to detect clinically significant prostate cancer in men with biopsy-proven low-grade prostate cancer (proSPECT-A1). In the 3302 study, a cloud-based implementation of the image analysis technology described herein (referred to as “Pioneer”) was used to assist physicians in the diagnosis and treatment of prostate cancer based on the use of 99m Analysis and interpretation of SPECT / CT images obtained using TC-MIP-1404 as an imaging agent (for SPECT imaging) distinguishes individuals with and without clinically significant prostate cancer. As used herein, the phrases "1404 SPECT / CT images" and "1404 SPECT / CT imaging" refer to images obtained using 99mA SPECT / CT image obtained using TC-MIP-1404 as an imaging agent for SPECT, or an imaging process for obtaining the image.

[0417] The 3302 study builds on the Phase 3 study MIP-1404-3301 (hereinafter referred to as the "3301 study"), which was entitled "A Phase 3 Study to Evaluate the Safety and Efficacy of 1404 SPECT / CT Imaging to Detect Clinically Significant Prostate Cancer in Men with Biopsy-Confirmed Low-Grade Prostate Cancer Who Are Candidates for Active Surveillance." The 3301 study also evaluated the use of 1404 SPECT / CT imaging to differentiate individuals with and without clinically significant prostate cancer. However, in the 3301 study, diagnosis was made by a physician using conventional manual reading methods (described in further detail herein) without the assistance of a pioneer.

[0418] This example summarizes the results obtained via both approaches and compares the results of the AI-assisted diagnoses performed in the 3302 study with the conventional manually read diagnoses of the 3301 study.

[0419] Briefly, the 3301 and 3302 studies utilized a common dataset of 1404 SPECT / CT images obtained from individuals for whom histopathology results were also available. In both the 3301 and 3302 studies, each individual's SPECT / CT images were analyzed to assign prostate cancer status as clinically significant or clinically insignificant, that is, positive or negative for clinically significant prostate cancer. The results obtained from the 1404 SPECT / CT image analysis were compared with the underlying true histopathology results to assess the sensitivity and specificity of 1404 SPECT / CT imaging for identifying clinically significant prostate cancer.

[0420] In the 3301 study, the 1404 SPECT / CT images were analyzed via a conventional manual reading method, in which the physician used standard imaging viewing software to scroll through the 2D CT and SPECT image slices, manually position markers to identify areas of interest, and calculate TBR values. In contrast, in the 3302 study, the physician performed an AI-assisted review using embodiments of the systems and methods described herein. Specifically, in the artificial intelligence AI-assisted method of the 3302 study, a computer program similar to Fig.25 The workflow shown in FIG. 1 is a flow chart showing a physician performing his image analysis with the assistance of Pioneer software. For each image, Pioneer software performs a 3D segmentation of the prostate, calculates an initial TBR value, and assigns individuals to an initial prostate cancer status in a fully automated manner based on a comparison of the initial TBR value with a predefined threshold. Figures 26A-29HThe results of this automated analysis are provided to the physician for review using a slightly modified version of the GUI described above. The physician can accept the results of the fully automated analysis, or reject the results and use the GUI to update the target and / or background values ​​and recalculate the TBR in a semi-automated manner.

[0421] As described in detail below, the results of the AI-assisted method used in Study 3302 were found to be more accurate than the conventional manual reading method of Study 3301. In addition, by reducing or substantially eliminating reliance on subjective human judgment, the AI-assisted method of Study 3302 provided highly consistent and reproducible results that were obtained in a rapid manner, with the physician merely approving the fully automated analysis in most cases.

[0422] i.3301 Research, study population, imaging, and manual reading protocols.

[0423] Study 3301 is a multicenter, multi-reader, open-label study. Study 3301 started in December 2015. Study 3301 completed enrollment and follow-up in December 2017. The analysis of the results is included in this article.

[0424] The objective of the 3301 study was to evaluate 1404 CT / SPECT imaging for the detection of clinically significant prostate cancer in men who had a diagnostic transrectal ultrasound (TRUS)-guided biopsy with histopathological findings of a total Gleason score less than or equal to 3+4 and / or were candidates for active surveillance.

[0425] Individuals of two cohorts were enrolled. The first cohort (referred to as "cohort A") was men with biopsy-confirmed low-grade to intermediate-grade prostate cancer (i.e., whose biopsy was assigned a total Gleason score of 3+3 or 3+4) who were candidates for active surveillance but chose to undergo radical prostatectomy (RP). The second cohort (referred to as "cohort B") was patients with biopsy-confirmed very low-risk prostate cancer who were scheduled to undergo routine re-biopsy as part of routine active surveillance.

[0426] The study objectives were to evaluate (1) the specificity of 1404 SPECT / CT imaging in identifying individuals without clinically significant prostate cancer (e.g., total Gleason score less than or equal to 3+4) and (2) the sensitivity of 1404 SPECT / CT imaging in identifying individuals with clinically significant prostate cancer (e.g., total Gleason score greater than 3+4).

[0427] 1404 SPECT / CT imaging was performed, in which all enrolled subjects received a single dose of 20±3mCi (740±111MBq) 99mTC-MIP-1404 injection, followed by SPECT / CT and whole body planar imaging within 3-6 hours after injection. According to standard care procedures, individuals undergo voluntary RP surgery (cohort A) or prostate biopsy (cohort B) within 42 days after study drug administration, followed by histological evaluation of samples. Prostate specimens (cohort A) or biopsy (cohort B) are subjected to histopathology to establish the underlying true prostate cancer status of each patient. Central pathologists are blinded to all clinical data (including imaging results).

[0428] The images were analyzed using conventional manual reading methods. Image analysis and diagnosis performed in this manner are also referred to as "unassisted" hereinafter. Unassisted analysis was performed using conventional SPECT / CT imaging workstation software as commonly found in most hospitals (e.g., MIMvista from MIM Software in this case). Fig.38A A screenshot showing the MIMvista GUI as used in a conventional unassisted workflow. In the unassisted workflow, an image reader (e.g., a physician or technologist) views CT and SPECT images in 2D slices. Fig.38A , a set of CT slices 3802a, 3802b, and 3802c and corresponding SPECT image slices 3804a, 3804b, and 3804c are shown. The reader locates the prostate (e.g., manually) in the CT image slice (e.g., 3802a). The reader then manually scrolls through the slices to identify the maximum absorption slice in the SPECT image where the reader senses maximum absorption. Once the reader identifies this maximum absorption slice, the reader (manually) places a fixed-size circular marker 3806 in the corresponding slice in the CT image to identify the area of ​​interest, as shown in FIG. Fig.38B The maximum absorption in the region of interest (ie, within the circle) is then recorded (conventional software such as MIMvista only displays this number, along with other values ​​such as the average absorption).

[0429] See also Fig.38C , the reader performs a similar procedure to determine the background value. The reader places a circular marker in the obturator muscle near the prostate to identify the area of ​​interest 3808, and records the average absorption in the area as the background value. The TBR is then calculated as the ratio of the maximum absorption value to the background value. It is worth noting that in this conventional method, automated 3D segmentation is not performed. Instead, the reader examines the slices of the SPECT / CT image piece by piece to ultimately identify the two areas of interest within the CT image slices. In addition, in contrast to using the gluteal muscle as a reference tissue for determining the background value (as done using the Pioneer system), the obturator muscle is used as a reference tissue area.

[0430] In the 3301 study, three independent blinded physician readers analyzed the 1404 SPECT / CT images to assign each individual a clinically significant or clinically insignificant prostate cancer status based on the determined TBR value. Central histopathology review was used as the ground truth for evaluating the diagnostic performance of 1404 SPECT / CT imaging, and the image-based analysis results were compared with the histopathology results.

[0431] ii.3302 Research

[0432] The 3302 study is based on the 3301 study, using the same patient population and image data, but evaluating the performance of the AI-assisted image analysis method described in this article to detect clinically significant prostate cancer. Specifically, the 3302 study is a prospectively defined retrospective study based on the analysis of the data set obtained from the 3301 study (SPECT / CT images and histopathology). As described in the 3301 study and in this article, the central histopathology assessment is used as the basic fact for evaluating performance. The two main endpoints of the 3302 study are as follows:

[0433] The combination product is used to detect clinically significant prostate cancer compared with histopathology after radical prostatectomy (cohort A) or prostate biopsy (cohort B) 99m Specificity of Tc-MIP-1404 and Pioneer; and

[0434] The combination product is used to detect clinically significant prostate cancer compared with histopathology after radical prostatectomy (cohort A) or prostate biopsy (cohort B) 99m Sensitivity of Tc-MIP-1404 and Pioneer.

[0435] This article describes the specific workflow used, analysis protocol, and results comparing the performance of the AI-assisted approach from Study 3302 with the unassisted manual approach from Study 3301.

[0436] iii. AI-assisted image analysis workflow

[0437] Use the quality control and reporting workflow described herein (e.g. Fig.25 ), image analysis was performed with the assistance of Pioneer in the 3302 study. Fig.39 A block flow diagram showing a specific workflow 3900 used in study 3302 is shown. As shown, a technician performs a set of steps 3902 to upload and initiate an initial automated analysis through the Pioneer software, followed by a physician reviewing the analysis and steps performed in a quality control and reporting workflow 3904.

[0438] The technician is responsible for uploading the 1404 SPECT / CT image data and ensuring that the imaging requirements are met. The automated analysis is then initiated, leading to one of three results, which are determined by Fig.40A - The window presentation of the analysis GUI shown in FIG. Fig.40A Screen shot showing a window of the analysis GUI indicating that the automated analysis completed successfully and the image is ready for review (eg, by a physician). Fig.40B Screen shot showing a window of the analysis GUI indicating that the automated analysis was not completed due to an analysis error, in which case manual guidance by a physician was used to complete the analysis (which was also considered ready for review). Fig.40C Screen shot showing a window of the analysis GUI indicating that the automated analysis was not completed due to incomplete data and that a technician is required to correct the imaging problem and / or re-upload the data.

[0439] To review the images, the physician uses a GUI similar to that described herein to follow Fig.39 workflow. Figures 41-44 Screen capture showing the various windows of the GUI that a physician uses to analyze images and make a diagnosis. Fig.41 Screen shot showing a list of patients to be reviewed by a physician. The physician reader selects a specific patient to review his data and assign a prostate cancer status.

[0440] After patient selection, a physician reader reviews the results of the automated analysis. Fig.42A -E A screen shot showing a view of the analysis GUI used by physicians to review the results of automated analysis performed by Pioneer software. The analysis GUI implemented in the Pioneer software used in the 3302 study is similar to the one described in this article. Fig.29A -H Analysis GUI shown and described. Fig.42A is a screenshot of the physician presented view of the analysis GUI. Fig.42A The screenshot in is similar to Fig.29A , and displays a 2D slice with SPECT image data superimposed on CT image data and a segmentation mask identifying the prostate. Fig.29A Same, Fig.42A The GUI view in also displays the automatically determined TBR value and the assignment of prostate cancer status (e.g., assignment as "clinically significant") based on comparison of the TBR value to a threshold as described herein. Fig.42B exhibit Fig.42A Another screenshot of the viewer shown in , where the physician switches to a 3D volume view of the SPECT / CT image data and the segmentation mask.

[0441] like Fig.42A shown in and Fig.42CAs further shown in detail in , the analysis GUI used in the 3302 study also includes a likelihood severity scale for assigning an individual's results to four categories of clinical significance. Specifically, in addition to the binary classification of clinically significant or clinically insignificant, an individual's prostate cancer status is assigned to one of four categories based on a comparison of three thresholds of calculated TBR values. On the likelihood severity scale, a "very unlikely" status is assigned for TBR values ​​below a first threshold, "unlikely" is assigned for TBR values ​​greater than or equal to the first threshold and below a second threshold, "likely" is assigned for TBR values ​​ranging from the second threshold to the third threshold, and "very likely" is assigned for values ​​greater than the third threshold.

[0442] Using the ROC analysis described in Example 4, reference Fig.35A and 35B , three TBR thresholds for assigning individuals to categories on the likelihood severity scale were determined. Each of the three thresholds was selected to produce a specific sensitivity and specificity value pair. Table 2 below shows the same data as in Table 1, but expanded to show specificity and sensitivity data for additional TBR thresholds. Fig.45A and 45B Repaint Fig.35A and 35B Figure 2 shows the data shown in Figure 3, but three TBR thresholds were used to assign individuals to the categories on the likelihood severity scale indicated. Fig.45A and 45B As shown in , a threshold of 12 was selected as the first lower threshold to provide a target sensitivity of 0.95; a threshold of 25 was selected as the second intermediate threshold to provide a sensitivity and specificity of 0.75 (for both), and a threshold of 45 was selected as the third upper threshold to provide a specificity of 0.9. Table 3 below shows the lower endpoint of the Jeffreys one-sided 95% confidence interval (CI) for each of these three thresholds.

[0443] Table 2. Robust estimates of sensitivity and specificity for different TBR thresholds

[0444] TBR threshold Specificity Sensitivity 10 0.17 0.98 11 0.22 0.96 12 0.28 0.95 13 0.33 0.94 14 0.38 0.92 … … … 23 0.70 0.78 24 0.73 0.76 25 0.75 0.75 26 0.76 0.73 27 0.78 0.72 … … … 43 0.89 0.51 44 0.89 0.50 45 0.90 0.49 46 0.90 0.48 47 0.90 0.47

[0445] Table 3. Direct estimates of sensitivity and specificity for three TBR thresholds and the lower end point of Jeffreys one-sided 95% confidence intervals

[0446]

[0447] Fig.45C and 45D It is shown how these thresholds, and the corresponding categories on the likelihood severity scale, relate to the total Gleason score (ranging from 2 to 10) of individuals in the Phase I and Phase II study populations used to determine the thresholds. Fig.45Cand 45D The results shown in show the likelihood severity scale with the indicated four-category labeling scheme, which is then updated to Fig.42A and 42C The four equivalent categories shown in Fig.42C The “likely negative” (in the scale shown in Fig.42C The category 4502 (referred to as "Impossible" in the scale shown in FIG) corresponds to TBR values ​​ranging from 12 to 25. Fig.45A and 45B As shown in , patients assigned to the "definite negative" 4501 and "likely negative" 4502 categories based on TBR values ​​generally have a total Gleason score below 7. Fig.42C Category 4503 corresponds to TBR values ​​ranging from 25 to 45 and captures total Gleason scores of 7 and 8. Patients with TBR values ​​above 45 are assigned a "definite positive" (in Fig.42C ) category 4504 on the scale shown in , and will typically have an overall Gleason score of 9 or 10.

[0448] As previously described herein, the physician is given the option to approve the automated evaluation or to override the automated evaluation in its entirety and update the target (maximum absorbance) value and background value used to calculate the TBR. Fig.42D A screenshot showing a GUI view used by a physician to update target and background values ​​for calculating TBR is shown. The physician is also given updates such as Fig.42E Options for clinically significant classification as shown in .

[0449] like Fig.39 As shown in , once the physician completes the quality control workflow, a report 3903 is generated. Fig.43A Shown is a report generated via fully automated review, where the physician approved only the TBR calculation and clinical significance status as determined via automated analysis by the Pioneer software. Fig.43B Shows a report generated via semi-automated analysis, where the physician is assisted by the software, but updates target and background values. Fig.44 Displays reports generated for unevaluable situations.

[0450] IV.3302 Research Results

[0451] The 3302 study used three independent readers (new readers, different from the readers that performed data analysis in the 3301 study) to analyze 1404 SPECT / CT images via the workflow described herein, and to identify whether the patient had a clinically significant prostate cancer state. Of the 464 cases in the 3301 study, 13 were excluded due to missing pathology (3 cases) or technical problems (10 cases), leaving a total of 451 cases for analysis in the 3302 study. The first reader (referred to as "AI Reader 1") analyzed 451 cases, the second reader (referred to as "AI Reader 2") analyzed 450 cases, and the third reader (referred to as "AI Reader 3") analyzed 441 cases.

[0452] like Fig.46 As shown in , the AI-assisted analysis method described in this article allows physicians to quickly assess the clinical significance and severity of prostate cancer in each patient based largely on a fully automated evaluation generated by the pioneering software. Fig.46 The histogram in shows the number of cases that each reader can handle in a specific amount of time. The reader using the Pioneer software embodiment in the 3302 study took an average of 3.5 minutes per case. In contrast, in the manual reading method of the 3301 study, for each case, the technician spent approximately 5 to 10 minutes manually placing the area of ​​interest in the prostate and background (obturator), as described above, and the radiologist then spent another 5 to 10 minutes to evaluate and then fill out a case report form. Therefore, the AI-assisted method provides a significant efficiency improvement, reducing the time spent on each case from up to 20 minutes for a single case to an average of 3.5 minutes per case.

[0453] In addition, if Fig.47 As shown in the , the AI-assisted method of the 3302 study for image analysis and prostate cancer status diagnosis can obtain highly consistent and reproducible results. The consistency of the results refers to the similarity of the results between different readers. Reproducibility refers to the variability within the reader, that is, the degree of similarity between the results produced by repeated analyses with a single reader.

[0454] Fig.47 The set of figures in demonstrates the consistency and reproducibility of the results obtained via the AI-assisted method of the 3302 study. Notably, as shown in the histogram 4702 of the figure, the majority (between 69% and 85%) of the cases were processed in a fully automated manner, and the physician reader simply approved the binary classification performed by the Pioneer system. The remaining cases were processed in a semi-automated manner, and the physician intervened to update the maximum absorption target value.

[0455] The main reason for the reader to intervene to select a new maximum intensity image voxel of the target value is due to bladder intensity overflow into the prostate region in the SPECT image. It is believed that this is mainly caused by Compton scattering. The Pioneer system includes a bladder overflow correction feature as described herein. However, in some cases (particularly in the low-grade disease population examined in the 3301 and 3302 studies, where absorption in the prostate is very small), the reader disagrees with the automated analysis (e.g., believes that the automated selection of the target voxel is inaccurate) and updates the target voxel selection with the assistance of the Pioneer system.

[0456] The high rate of acceptance of fully automated results is significant because fully automated diagnostics eliminate intra- and inter-reader variability (allowing for fully consistent and reproducible results).

[0457] The scatter plots 4710a, 4710b, and 4710c (collectively 4710) and 4720a, 4720b, and 4720c (collectively 4720) on the right side of the figure show the variability between readers. Each figure compares the TBR results obtained via a specific AI-assisted reader with those obtained by an internal reference reader (which also uses Pioneer software and follows the same protocol as the three readers of the 3302 study). Each point represents a case, where the y-axis value gives the TBR value calculated by the internal reader and the x-axis value gives the TBR value calculated by the specific reader. Scatter plot 4700 shows cases performed via fully automated evaluation, as evidenced by the complete (r=1) correlation coefficient, which eliminates variability between readers and allows completely consistent results. Cases analyzed in a semi-automated manner as described herein are shown in Figure 4720. The scatter plot and high correlation coefficient (better than r=0.95) show that highly consistent results are obtained even when the reader intervenes to update the target voxel selection. Therefore, even though the placement of this target voxel is subjective, the assistance of the Pioneer system can achieve highly consistent results.

[0458] Demonstrating the performance of three AI-assisted readers analyzing images in the 3302 study and from Fig.48A and Fig.48B Performance of three manual readers in the 3301 study. Fig.48AReceiver operating characteristic (ROC) curves for each of the three AI-assisted readers of the 3302 study and the three manual readers of the 3301 study are shown. The ROC curves show how sensitivity and specificity change with changes in the TBR threshold used to binary classify patients as being positive or negative for clinically significant prostate cancer. As described herein, in both the 3301 and 3302 studies, histopathology was used as the basis for sensitivity and specificity analysis. Table 4 below shows how the 1404 SPECT / CT imaging results are compared with histopathology to determine whether the imaging result is a true positive (TP), true negative (TN), false positive (FP) or weak negative (FN) in each particular case.

[0459] Table 4. Classification of 99mTc-MIP-1404 and Pioneer Imaging Results by Histopathology

[0460]

[0461] Positive prostate histopathology results were defined as clinically significant disease reported on the electronic case report form after MIP-1404 SPECT / CT imaging: (i) in at least one prostate lobe (for Cohort A) or (ii) in a prostate biopsy (Cohort B). For Cohort A, central histopathology laboratory evaluation of clinically significant disease was supported by at least one of the following findings:

[0462] Any Gleason grade disease in the prostate greater than 3+4;

[0463] Any Gleason grade 3+4 disease in greater than or equal to 10% of the prostate that is grade 4; and

[0464] Any disease within the prostate that is greater than or equal to stage T3 (pT3, according to the prostate cancer staging system).

[0465] In the event that an individual in Cohort B subsequently underwent RP, the histopathological results obtained from the prostate specimen were then used as the reference standard, with the remaining individuals in Cohort B used for subgroup analysis.

[0466] A positive imaging evaluation indicates the presence of prostate cancer as determined by the imaging reader (ie, based on an evaluation of the 1404 SPECT / CT imaging data).

[0467] Specificity is defined as the proportion of true negatives (TN) among individuals with histopathologically defined non-clinically significant prostate cancer (ie, with a total Gleason score of less than or equal to 3+4). Specificity is calculated as follows:

[0468] Specificity = TN / (FP+TN).

[0469] Sensitivity is the proportion of true positives (TP) among individuals with histopathologically defined clinically significant prostate cancer (ie, with a total Gleason score greater than 3+4). Sensitivity is calculated as follows:

[0470] Sensitivity = TP / (TP+FN).

[0471] Changing the TBR threshold gives different ratios of TP, FP, FN and TN, and therefore different sensitivities and specificities. For each reader, Fig.48A The ROC curves shown in Figure 1 plot the resulting changes in sensitivity and specificity when the TBR threshold is changed. As shown in the figure, all AI-assisted reader curves are above the curves of the manual reader, indicating that the performance of the AI-assisted method is improved. In addition, as shown in Figure 2, the sensitivity and specificity of the AI-assisted method are improved. Fig.48B As shown in , the area under the curve (AUC) value for each reader was calculated, and the performance of the AI-assisted reader was improved compared to the performance of the manual reader (for five of the nine possible comparisons between one of the three manual readers and one of the three AI-assisted readers, the p value was p<0.05). In addition, for a specific TBR threshold, the TBR value calculated by the AI-assisted method of the 3302 study was compared with the TBR threshold, thereby providing a high degree of specificity in making a binary classification of positive or negative for clinically significant prostate cancer. In particular, for patients with very low, low, and moderate risk of prostate cancer, specificities of 95%, 96%, and 96%, respectively, were found, with a sensitivity of 21%.

[0472] Fig.49A -C Compare internal reference reader ( Fig.47 The performance of the three AI-assisted readers studied in 3302 is compared with the performance of each of the human readers studied in 3301. Fig.47 As demonstrated by the data shown in , the performance of the internal reference reader is highly correlated with and representative of the three AI-assisted readers of the 3302 study. Fig.49A -C Comparison of individual ROC models for binary classification (classification as positive or negative for clinically significant disease based on comparison of TBR values ​​to a threshold) between the AI-assisted internal reference reader (referred to as "Pioneer Read" in the figure) and each manual reader of Study 3301. In each case, the improvement in the AUC metric for the AI-assisted reader compared to the manual reader was statistically significant (p-values ​​of 0.01, 0.02, and 0.045).

[0473] like Fig.49A-C, the AUC of the AI-assisted internal reference reader was 0.66 (95% CI: 0.61-0.71). Logistic regression indicated that the optimal TBR threshold was 14.4, with a sensitivity of 65% (95% CI: 61%-70%) and a specificity of 62% (95% CI: 58%-67%). Automated performance was significantly better than manual reader 1 (AUC 0.63; p=0.0445), manual reader 2 (AUC 0.62; p=0.0201), and manual reader 3 (AUC 0.62; p=0.0122). In addition, when a multivariate model including baseline covariates (taking into account patient age, weight, and PSA score) was used, the performance of the AI-assisted method of the 3302 study was compared with the manual reading method of the 3301 study. For the multivariate model, the AI-assisted internal reference reader demonstrated higher accuracy in predicting clinically significant disease (AUC 0.80) than the manual reading method (AUC 0.77), p=0.0014.

[0474] Thus, this example demonstrates that the combination of physician-assisted pioneer evaluation and 1404 imaging data indicates significant improvements in specificity and sensitivity for evaluating the clinical significance of prostate cancer relative to manual readers. Additionally, pioneer-assisted physicians are very efficient in analyzing imaging data and produce highly reproducible results, as well as useful information about the clinical severity of the diagnosis.

[0475] L. Example 6: Implemented by the first machine learning module (localization machine) and the second machine learning module (segmentation machine) Training and validation of existing convolutional neural networks

[0476] Example 6 is an example of training and validation of a CNN module for segmenting CT images to identify various tissue volumes (including prostate volume) according to aspects and embodiments described herein. In this example, a Keras framework with a Tensorflow backend is used to define and train a neural network.

[0477] i. Training and validation data

[0478] The training and validation data consisted of CT images coupled with semi-automated segmentation corrected by a radiologist, depicting all or some of the following body parts: (i) prostate; (ii) bladder; (iii) rectum; (iv) left gluteus maximus; (v) right gluteus maximus; (vi) left hip bone; (vii) right hip bone; and (viii) sacrum and coccyx.

[0479] To train the localization CNN, 90 high-quality CT images were used to segment all of the above body parts. These and 10 images of the same type were used for training the segmentation CNN for high-resolution segmentation of the left gluteus maximus. For training the segmentation CNN for high-resolution segmentation of the prostate, an additional dataset of 73 low-quality images was used, which had more anatomical variations of the prostate and bladder (due to disease). These additional images were matched only to the segmentation of the prostate and bladder. For these 73 images, pseudo labels were generated for the rectum, left and right hip bones, and sacrum and coccyx. Pseudo labels are predicted labels generated using a previously trained network (a network trained on 100 high-quality CT images). Prior to training, pseudo labels were merged with manual labels.

[0480] ii. Training Configuration

[0481] Each training run is defined by a configuration file named config.json. The configuration file includes the following parameters: (i) which dataset to use; (ii) data preprocessing; (iii) partitioning the data into training and validation sets; (iv) model architecture; and (v) training hyperparameters.

[0482] This configuration file is used when the trained model is used for inference, to ensure that the same image preprocessing during training is done during inference.

[0483] iii. Preprocessing

[0484] The configuration file defines which preprocessing steps are applied during training. Several preprocessing steps are included. A cropping preprocessing step is performed to remove surrounding air for localization training, and to crop out bounding boxes defined by the pelvic bones, bladder, rectum, and prostate for segmentation. A fixed intensity normalization step is performed by subtracting a fixed value and then dividing by another fixed value, where the value is selected so that the intensity of the voxels in the image has an average intensity of 0 and a standard deviation of 1 on average. The image is also reshaped to a fixed size by resampling. For localization, for images with a median size in each direction, the resolution in each direction is reduced by a factor of 4. For high-resolution segmentation, for images with a median size in each dimension, the resolution does not change. Other preprocessing steps include one-hot-encoding of the segmentation labels, which are reweighted as defined in Section iv. "Optimization", below, and data augmentation, as defined in Section iv. "Optimization", below.

[0485] Some preprocessing steps are done offline to speed up training, resulting in a preprocessed dataset. The config.json belonging to training only defines those preprocessing steps that are done online.

[0486] For localization training, the preprocessing steps used for offline preprocessing are stored in a separate file meta_data / prep_config.json in the trained CNN directory.

[0487] iv. Optimization

[0488] The loss function used for training is the voxel-wise categorical cross entropy, weighted so that all voxels belonging to one class (in the true label) together have the same weight as all voxels belonging to any other class (when the label frequency is equal to the median frequency in the dataset). This approach balances the classes and is important for prostate segmentation because the prostate is very small compared to (for example) the background.

[0489] The loss is optimized for about 2000-3000 epochs using mini-batch gradient descent with the Adam optimizer. The batch size is equal to 1. A batch size of 1 means that batch normalization becomes instance normalization (this is because the batch is only one sample). To improve performance, instance normalization is also performed during inference.

[0490] By having a very low learning rate (e.g. 1×10 -5 ) and then in steps from a high learning rate (e.g. 1×10 -2 ) decreases. After 250 epochs, the learning rate is halved, then again after 500 epochs, and again after 1000 epochs. To reduce overfitting, dropout is used. The dropout rate varies between 0.2 and 0.5 for different networks (but the same ratio is applied everywhere for a given network, e.g. Fig.13 and Fig.14 ). The appropriate dropout rate is based on experience with multiple training runs and how it affects the performance difference between the training and validation sets.

[0491] A training method that uses training image data augmented by adding random distortions to prevent the neural network from focusing on fine details in the image. Training a neural network (e.g., localization CNN and segmentation CNN) in this manner allows it to process images with image artifacts that are not present or uncommon in the training data (e.g., effectively localize and / or segment them).

[0492] The random distortions added to augment the training data included additive or multiplicative noise and added smooth salt noise. These random distortions were applied independently with certain probabilities. These random distortions were scheduled not to be used for the first few hundred generations so that the network first learns to process artifact-free images and then gradually increases the noise level. Intensity distortion was only applied to the 100 high-quality CT images and not to the 73 low-quality images because they already had artifacts.

[0493] The localization CNN is trained to process (eg, receive as input) full body images as well as different partial body images. To this end, an increase in random crops of the image is used (always keeping the entire pelvis bounding box).

[0494] v. Computing resources

[0495] The positioning training is done on an Nvidia GeForce GTX 1050, which takes several hours.

[0496] High-resolution segmentation training was done on an Invida GeForce GTX 1080 Ti, which took 2-3 days.

[0497] vi. Model selection

[0498] In order to optimize the neural network architecture and select hyperparameters for training, it is necessary to have a metric to measure performance. The main metric for evaluating training progress is the Sorenson-Dyess ( - Dice) score (hereinafter referred to as "Dice score"): a weighted score (weighted as described in section iv. "Optimization" above) or a score for an individual body part (prostate or left gluteus maximus).

[0499] When augmenting results over multiple images, the average of the evaluated Dyess scores is most often used (e.g. as an indicator). In some cases, the frequency of images with a Dyess score below a certain level is used.

[0500] For each positioning training run, the model with the most preferred average weighted Dye score of the training image is selected. For the segmentation training for training the segmentation neural network to segment the prostate, the Dye score of the prostate is used as the basis for selection. For the segmentation training for training the segmentation neural network to segment the left gluteus maximus, the Dye score of the left gluteus maximus is the basis for selection. For the segmentation training with auxiliary prediction, the Dye score is based on the prediction from the main output-that is, the output that also exists when there is no auxiliary prediction.

[0501] When training the localization CNN, 30% of the 90 CT images were reserved for validation so that the performance of the localization CNN could be evaluated using untrained images. To train the segmentation CNN for high-resolution prostate segmentation, 23 low-quality CT images were set aside for validation, leaving only 50 low-quality images for training (resulting in a total of 150 CT images for training).

[0502] When choosing between different localization training, a metric corresponding to the precision in cropping is used. Cropping precision is evaluated based on several metrics / aspects. Specifically, one metric is that as few as possible of the 90 training and validation images should require a cropping margin of more than 0.1 to cover the entire pelvic bone. Another metric is that the distance error to the bounding box wall should be as small as possible (assessed by viewing the error box plot). Another metric is that on 102 low-quality CT images (no ground truth segmentation available), the 2D projection of the coarse segmentation overlapping the final bounding box should show good agreement between the segmentation and the anatomy, especially covering the appropriate area.

[0503] When choosing between different high-resolution segmentation training, the following metrics / aspects were considered: (i) Dye scores for training and validation data; (ii) precision and recall of the prostate for training and validation data; (iii) overlap between the underlying true bladder and the predicted prostate; and (iv) examples of the segmentation superimposed on the CT image in the CT image viewer, where slices can be scrolled in the sagittal, axial, and coronal planes.

[0504] Therefore, this example provides an example of a method that can be used to train the neural network models used in the positioning machine learning module (the first machine learning module) and the segmentation machine learning modules (the second machine learning module and any auxiliary segmentation machine learning modules) described in this article.

[0505] M. Imaging Agents

[0506] In certain embodiments, the 3D functional image is a nuclear medicine image using an imaging agent containing a radiopharmaceutical. Nuclear medicine images are obtained after administration of a radiopharmaceutical to a patient and provide information about the distribution of the radiopharmaceutical in the patient. A radiopharmaceutical is a compound containing a radionuclide.

[0507] Nuclear medicine images (e.g., PET scans; e.g., SPECT scans; e.g., whole body scans; e.g., composite PET-CT images; e.g., composite SPECT-CT images) detect radiation emitted from the radionuclide of the radiopharmaceutical to form an image. The distribution of a particular radiopharmaceutical within a patient can be determined by biological mechanisms (e.g., blood flow or perfusion) and by specific enzymatic or receptor binding interactions. Different radiopharmaceuticals can be designed to exploit different biological mechanisms and / or specific enzymatic or receptor binding interactions, and therefore, when administered to a patient, selectively concentrate in specific types of tissues and / or regions within the patient. The amount of radiation emitted from regions within a patient with a higher concentration of radiopharmaceutical is greater than other regions, causing these regions to appear brighter in the nuclear medicine image. Therefore, intensity variations in nuclear medicine images can be used to map the distribution of the radiopharmaceutical within a patient. This mapped distribution of the radiopharmaceutical within a patient can be used, for example, to infer the presence of cancerous tissue in various regions of the patient's body.

[0508] For example, when administered to a patient, technetium-99m methylene diphosphonate ( 99m Tc MDP) selectively accumulates in the patient's skeletal regions, particularly at sites of abnormal bone formation associated with malignant bone lesions. The selective concentration of the radiopharmaceutical at these sites produces identifiable hot spots - localized areas of high intensity in nuclear medicine images. Therefore, the presence of malignant bone lesions associated with metastatic prostate cancer can be inferred by identifying the hot spots within the patient's whole-body scan. Risk indices associated with the patient's overall survival and other prognostic indicators indicating disease status, progression, treatment efficacy, and the like can be determined based on the number of patients given the radiopharmaceutical. 99m The calculation is based on automated analysis of intensity changes in whole body scans obtained after Tc MDP. In certain embodiments, other radiopharmaceuticals may also be similar to 99m Tc MDP is used.

[0509] In some embodiments, the particular radiopharmaceutical used depends on the particular nuclear medicine imaging modality being used. 18 Sodium fluoride (NaF) also accumulates in bone lesions, similar to 99m Tc MDP, but can be used with PET imaging. In certain embodiments, PET imaging can also utilize a radioactive form of the vitamin choline, which is readily taken up by prostate cancer cells.

[0510] In certain embodiments, radiopharmaceuticals may be used that selectively bind to specific proteins or receptors of interest, particularly those whose expression is increased in cancer tissue. Such proteins or receptors of interest include, but are not limited to, tumor antigens such as CEA, which is expressed in colorectal cancer; Her2 / neu, which is expressed in a variety of cancers; BRCA 1 and BRCA2, which are expressed in breast and ovarian cancers; and TRP-1 and -2, which are expressed in melanoma.

[0511] For example, human prostate-specific membrane antigen (PSMA) is upregulated in prostate cancer (including metastatic disease). PSMA is expressed by almost all prostate cancers, and its expression is further increased in poorly differentiated, metastatic, and hormone-resistant cancers. Therefore, radiopharmaceuticals corresponding to PSMA binders (e.g., compounds with high affinity to PSMA) labeled with one or more radionuclides can be used to obtain nuclear medicine images of patients, according to which the status and / or state of prostate cancer in various regions of the patient (e.g., including but not limited to bone regions) can be evaluated. In certain embodiments, when the disease is in a localized state, nuclear medicine images obtained using PSMA binders are used to identify the presence of cancerous tissue in the prostate. In certain embodiments, nuclear medicine images obtained using radiopharmaceuticals containing PSMA binders are used to identify the presence of cancerous tissue in various regions including not only the prostate but also other organs and tissue regions (e.g., lungs, lymph nodes, and bones), which is relevant when the disease is metastatic.

[0512] Specifically, when administered to a patient, the radionuclide-labeled PSMA-binding agent selectively accumulates in cancer tissue based on its affinity for PSMA. 99m In a manner similar to that of Tc MDP, the selective concentration of radionuclide-labeled PSMA binders at specific sites within a patient creates detectable hot spots in nuclear medicine images. As the PSMA binders concentrate in various cancer tissues and regions expressing PSMA in vivo, localized cancer within the patient's prostate and / or metastatic cancer in various regions of the patient's body can be detected and assessed. As described below, risk indices associated with overall patient survival and other prognostic indicators indicative of disease status, progression, efficacy of treatment, and the like can be calculated based on automated analysis of intensity changes in nuclear medicine images obtained after administration of PSMA binder radiopharmaceuticals to patients.

[0513] Various radionuclide-labeled PSMA binding agents can be used as radiopharmaceutical imaging agents for nuclear medicine imaging to detect and evaluate prostate cancer. In certain embodiments, the specific radionuclide-labeled PSMA binding agent used depends on factors such as the specific imaging modality (e.g., PET; e.g., SPECT) and the specific area (e.g., organ) of the patient to be imaged. For example, certain radionuclide-labeled PSMA binding agents are suitable for PET imaging, while others are suitable for SPECT imaging. For example, certain radionuclide-labeled PSMA binding agents are beneficial for imaging the prostate of a patient and are primarily used when localizing the disease, while others are beneficial for imaging organs and regions throughout the patient's body and can be used to evaluate metastatic prostate cancer.

[0514] Various PSMA-binding agents and radionuclide-labeled versions thereof are described in U.S. Pat. Nos. 8,778,305, 8,211,401, and 8,962,799, each of which is incorporated herein by reference in its entirety.

[0515] i. PET imaging radionuclide-labeled PSMA binders

[0516] In certain embodiments, the radionuclide-labeled PSMA binding agent is a radionuclide-labeled PSMA binding agent suitable for PET imaging.

[0517] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises [18F]DCFPyL (also referred to as PyL TM ; also known as DCFPyL-18F):

[0518]

[0519] or a pharmaceutically acceptable salt thereof.

[0520] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises [18F]DCFBC:

[0521]

[0522] or a pharmaceutically acceptable salt thereof.

[0523] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 68 Ga-PSMA-HBED-CC (also known as 68 Ga-PSMA-11):

[0524]

[0525] or a pharmaceutically acceptable salt thereof.

[0526] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises PSMA-617:

[0527]

[0528] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 68 Ga-PSMA-617 (which is 68 In certain embodiments, the radionuclide-labeled PSMA-617 comprises 177 Lu-PSMA-617 (which is 177 Lu-labeled PSMA-617) or a pharmaceutically acceptable salt thereof.

[0529] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises PSMA-I&T:

[0530]

[0531] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 68 Ga-PSMA-I&T (which is 68 Ga-labeled PSMA-I&T) or a pharmaceutically acceptable salt thereof.

[0532] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises PSMA-1007:

[0533]

[0534] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 18 F-PSMA-1007 (which is 18 F-labeled PSMA-1007) or a pharmaceutically acceptable salt thereof.

[0535] ii. SPECT imaging of radionuclide-labeled PSMA binders

[0536] In certain embodiments, the radionuclide-labeled PSMA binding agent is a radionuclide-labeled PSMA binding agent suitable for SPECT imaging.

[0537] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 1404 (also referred to as MIP-1404):

[0538]

[0539] or a pharmaceutically acceptable salt thereof.

[0540] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 1405 (also referred to as MIP-1405):

[0541]

[0542] or a pharmaceutically acceptable salt thereof.

[0543] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 1427 (also known as MIP-1427):

[0544]

[0545] or a pharmaceutically acceptable salt thereof.

[0546] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 1428 (also referred to as MIP-1428):

[0547]

[0548] or a pharmaceutically acceptable salt thereof.

[0549] In certain embodiments, the radionuclide is combined with a radioisotope of a metal, such as a radioisotope of technetium (Tc), such as technetium-99m ( 99m Tc)); for example, radioactive isotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium-186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g. 90 Y); for example, radioactive isotopes of lutetium (Lu) (such as 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; for example, 67 Ga); for example, radioactive isotopes of indium (e.g. 111 In); for example, radioactive isotopes of copper (Cu) (such as 67 The PSMA-binding agent was labeled with a radionuclide by chelation of Cu].

[0550] In certain embodiments, 1404 is radionuclide-labeled (e.g., chelated to a radioisotope of a metal). In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 99m Tc-MIP-1404, which is 99m Tc-labeled (e.g. chelated) 1404:

[0551]

[0552] or a pharmaceutically acceptable salt thereof. In certain embodiments, 1404 can be combined with other metal radioisotopes [e.g., radioisotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium-186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g. 90 Y); for example, radioactive isotopes of lutetium (Lu) (such as 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g. 111 In); for example, radioactive isotopes of copper (Cu) (such as 67 Cu)] chelate to form a 99m Compounds of the structure shown in Tc-MIP-1404, wherein the radioactive isotopes of other metals are substituted 99m Tc.

[0553] In certain embodiments, 1405 is radionuclide-labeled (e.g., chelated to a radioisotope of a metal). In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 99m Tc-MIP-1405, which is 99m Tc-labeled (e.g. chelated) 1405:

[0554]

[0555] or a pharmaceutically acceptable salt thereof. In certain embodiments, 1405 can be combined with other metal radioisotopes [e.g., radioisotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium-186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g. 90 Y); for example, radioactive isotopes of lutetium (Lu) (such as 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; e.g. 67 Ga); for example, radioactive isotopes of indium (e.g. 111 In); for example, radioactive isotopes of copper (Cu) (such as 67 Cu)] chelate to form a 99m Compounds of the structure shown in Tc-MIP-1405, wherein the radioactive isotopes of other metals are substituted 99m Tc.

[0556] In certain embodiments, 1427 is labeled with (eg, chelated to) a radioisotope of a metal to form a compound according to the following formula:

[0557]

[0558] or a pharmaceutically acceptable salt thereof, wherein M is a metal radioisotope [e.g., a radioisotope of technetium (Tc) (e.g., technetium-99m ( 99m Tc)); for example, radioactive isotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium-186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g. 90 Y); for example, radioactive isotopes of lutetium (Lu) (such as 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; for example, 67 Ga); for example, radioactive isotopes of indium (e.g. 111 In); for example, radioactive isotopes of copper (Cu) (such as 67 Cu)] and marked it with 1427.

[0559] In certain embodiments, 1428 is labeled with (eg, chelated to) a radioisotope of a metal to form a compound according to the following formula:

[0560]

[0561] or a pharmaceutically acceptable salt thereof, wherein M is a metal radioisotope [e.g., a radioisotope of technetium (Tc) (e.g., technetium-99m ( 99m Tc)); for example, radioactive isotopes of rhenium (Re) (e.g., rhenium-188 ( 188 Re); for example, rhenium-186 ( 186 Re)); for example, radioactive isotopes of yttrium (Y) (e.g. 90 Y); for example, radioactive isotopes of lutetium (Lu) (such as 177 Lu); for example, radioactive isotopes of gallium (Ga) (such as 68 Ga; for example, 67 Ga); for example, radioactive isotopes of indium (e.g. 111 In); for example, radioactive isotopes of copper (Cu) (such as 67 Cu)] and marked it with 1428.

[0562] In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises PSMA I&S:

[0563]

[0564] or a pharmaceutically acceptable salt thereof. In certain embodiments, the radionuclide-labeled PSMA-binding agent comprises 99m Tc-PSMA I&S (which is 99m Tc-labeled PSMA I&S) or a pharmaceutically acceptable salt thereof.

[0565] N. Computer system and network architecture

[0566] like Fig.36 As shown in , an implementation of a network environment 3600 for providing the systems, methods, and architectures described herein is shown and described. In brief summary, now refer to Fig.36 , a block diagram of an exemplary cloud computing environment 3600 is shown and described. The cloud computing environment 3600 may include one or more resource providers 3602a, 3602b, 3602c (collectively referred to as 3602). Each resource provider 3602 may include computing resources. In some embodiments, computing resources may include any hardware and / or software for processing data. For example, computing resources may include hardware and / or software capable of executing algorithms, computer programs, and / or computer applications. In some embodiments, exemplary computing resources may include application servers and / or databases with storage and retrieval capabilities. Each resource provider 3602 may be connected to any other resource provider 3602 in the cloud computing environment 3600. In some embodiments, the resource providers 3602 may be connected via a computer network 3608. Each resource provider 3602 may be connected to one or more computing devices 3604a, 3604b, 3604c (collectively referred to as 3604) via a computer network 3608.

[0567] The cloud computing environment 3600 may include a resource manager 3606. The resource manager 3606 may be connected to resource providers 3604 and computing devices 3608 via a computer network 3602. In some embodiments, the resource manager 3606 may facilitate the provisioning of computing resources from one or more resource providers 3602 to one or more computing devices 3604. The resource manager 3606 may receive a request for computing resources from a particular computing device 3604. The resource manager 3606 may identify one or more resource providers 3604 that can provide the computing resources requested by the computing device 3602. The resource manager 3606 may select a resource provider 3602 to provide the computing resources. The resource manager 3606 may facilitate the connection between the resource provider 3602 and the particular computing device 3604. In some embodiments, the resource manager 3606 may establish the connection between the particular resource provider 3602 and the particular computing device 3604. In some embodiments, the resource manager 3606 may redirect a specific computing device 3604 to a specific resource provider 3602 with the requested computing resources.

[0568] Fig.37 Examples of computing device 3700 and mobile computing device 3750 that can be used to implement the techniques described in this disclosure are shown. Computing device 3700 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Mobile computing device 3750 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, and other similar computing devices. The components shown here, their connections and relationships, and their functions are meant to be examples only and are not meant to be limiting.

[0569] The computing device 3700 includes a processor 3702, a memory 3704, a storage device 3706, a high-speed interface 3710 connected to the memory 3708 and a plurality of high-speed expansion ports 3704, and a low-speed interface 3706 connected to a low-speed expansion port 3712 and a storage device 3714. Each of the processor 3702, the memory 3704, the storage device 3706, the high-speed interface 3708, the high-speed expansion port 3710, and the low-speed interface 3712 are interconnected using various buses and may be mounted on a common motherboard or otherwise appropriately mounted. The processor 3702 may process instructions for execution within the computing device 3700, including instructions stored in the memory 3704 or on the storage device 3706 to display graphical information for a GUI on an external input / output device (e.g., a display 3716 coupled to the high-speed interface 3708). In other embodiments, multiple processors and / or multiple buses may be used appropriately along with multiple memories and several types of memories. Furthermore, multiple computing devices may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a blade server group, or a multi-processor system). Thus, as the terminology is used herein, if multiple functions are described as being performed by a "processor," this encompasses embodiments in which the multiple functions are performed by any number of processor(s) of any number of computing devices(s). Furthermore, if a function is described as being performed by a "processor," this encompasses embodiments in which the function is performed by any number of processor(s) of any number of computing devices(s), such as in a distributed computing system.

[0570] The memory 3704 stores information within the computing device 3700. In some implementations, the memory 3704 is one or more volatile memory units. In some implementations, the memory 3704 is one or more non-volatile memory units. The memory 3704 may also be another form of computer-readable media, such as a magnetic disk or optical disk.

[0571] Storage device 3706 is capable of providing mass storage for computing device 3700. In some embodiments, storage device 3706 may be or contain a computer-readable medium, such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory, or other similar solid-state memory device or device array, including devices in a storage area network or other configuration. Instructions may be stored in an information carrier. Instructions, when executed by one or more processing devices (e.g., processor 3702), perform one or more methods, such as those described above. Instructions may also be stored by one or more storage devices (e.g., computer or machine-readable media (e.g., memory 3704, storage device 3706, or memory on processor 3702)).

[0572] High-speed interface 3708 manages bandwidth-intensive operations of computing device 3700, while low-speed interface 3712 manages less bandwidth-intensive operations. This functional allocation is merely an example. In some embodiments, high-speed interface 3708 is coupled to memory 3704, display 3716 (e.g., via a graphics processor or accelerator), and to high-speed expansion ports 3710 that can accept various expansion cards (not shown). In an embodiment, low-speed interface 3712 is coupled to storage device 3706 and low-speed expansion ports 3714. Various communication ports (e.g., USB, A low-speed expansion port 3714 for Ethernet, wireless Ethernet) can be coupled to one or more input / output devices, such as a keyboard, a pointing device, a scanner, or a network connection device (such as a switch or a router), for example, via a network adapter.

[0573] As shown in the figure, computing device 3700 can be implemented in several different forms. For example, it can be implemented as a standard server 3720 or multiple times in a group of such servers. In addition, it can be implemented in a personal computer, such as a laptop computer 3722. It can also be implemented as part of a rack server system 3724. Alternatively, components from computing device 3700 can be combined with other components in a mobile device (not shown), such as mobile computing device 3750. Each of the devices can contain one or more of computing device 3700 and mobile computing device 3750, and the entire system can be composed of multiple computing devices that communicate with each other.

[0574] The mobile computing device 3750 includes, among other components, a processor 3752, a memory 3764, an input / output device (e.g., a display 3754), a communication interface 3766, and a transceiver 3768. The mobile computing device 3750 may also be provided with a storage device (e.g., a microdrive or other device) to provide additional storage. Each of the processor 3752, the memory 3764, the display 3754, the communication interface 3766, and the transceiver 3768 are interconnected using various buses, and several components may be mounted on a common motherboard or otherwise if appropriate.

[0575] The processor 3752 may execute instructions within the mobile computing device 3750, including instructions stored in the memory 3764. The processor 3752 may be implemented as a chipset including separate or multiple analog and digital processor chips. For example, the processor 3752 may provide coordination of other components of the mobile computing device 3750, such as control of a user interface, applications executed by the mobile computing device 3750, and wireless communications performed by the mobile computing device 3750.

[0576] The processor 3752 may communicate with the user via a control interface 3758 and a display interface 3756 coupled to the display 3754. For example, the display 3754 may be a TFT (thin film transistor liquid crystal display) display or an OLED (organic light emitting diode) display or other appropriate display technology. The display interface 3756 may include appropriate circuits for driving the display 3754 to present graphics and other information to the user. The control interface 3758 may receive commands from the user and convert them for submission to the processor 3752. In addition, the external interface 3762 may provide communication with the processor 3752 to enable near-area communication of the mobile computing device 3750 with other devices. For example, the external interface 3762 may provide wired communication in some embodiments or wireless communication in other embodiments, and multiple interfaces may also be used.

[0577] The memory 3764 stores information within the mobile computing device 3750. The memory 3764 may be implemented as one or more of one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. An expansion memory 3774 may also be provided and connected to the mobile computing device 3750 via an expansion interface 3772, which may include, for example, a SIMM (Single In-line Memory Module) card interface. The expansion memory 3774 may provide additional storage space for the mobile computing device 3750 or may also store applications or other information for the mobile computing device 3750. Specifically, the expansion memory 3774 may include instructions to perform or supplement the above-described processes, and may also include security information. Thus, for example, the expansion memory 3774 may be provided as a security module for the mobile computing device 3750 and may be programmed with instructions to permit secure use of the mobile computing device 3750. Additionally, a security application may be provided via a SIMM card along with additional information, such as placing authentication information on a SIMM card in a non-hacking manner.

[0578] For example, the memory may include flash memory and / or NVRAM memory (non-volatile random access memory), as discussed below. In some embodiments, the instructions are stored in an information carrier. The instructions, when executed by one or more processing devices (e.g., processor 3752), perform one or more methods, such as those described above. The instructions may also be stored by one or more storage devices (e.g., one or more computer or machine readable media (e.g., memory 3764, expansion memory 3774, or memory on processor 3752)). In some embodiments, the instructions may be received, for example, on a transceiver 3768 or external interface 3762 in the form of a propagated signal.

[0579] The mobile computing device 3750 may communicate wirelessly via a communication interface 3766, which may include digital signal processing circuitry if desired. The communication interface 3766 may provide for communication in various modes or protocols, such as GSM voice calls (Global System for Mobile Communications), SMS (Short Message Service), EMS (Enhanced Communication Service) or MMS communications (Multimedia Communication Service), CDMA (Code Division Multiple Access), TDMA (Time Division Multiple Access), PDC (Personal Digital Cellular System), WCDMA (Wideband Code Division Multiple Access), CDMA2000 or GPRS (General Packet Radio Service), among others. For example, such communication may occur using radio frequencies via the transceiver 3768. Additionally, short-range communications may be performed using, for example, Wi-Fi TM Or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module 3770 can provide additional navigation and location-related wireless data to mobile computing device 3750, which can be used by applications running on mobile computing device 3750, where appropriate.

[0580] The mobile computing device 3750 may also communicate audio using an audio codec 3760, which may receive spoken information from a user and convert it into usable digital information. The audio codec 3760 may likewise produce audible sounds to the user, such as via a speaker, in a handset of the mobile computing device 3750. Such sounds may include sounds from voice telephone calls, may include recorded sounds (e.g., voice messages, music files, etc.), and may also include sounds produced by applications operating on the mobile computing device 3750.

[0581] As shown in the figure, the mobile computing device 3750 can be implemented in several different forms. For example, it can be implemented as a cellular phone 3780. It can also be implemented as part of a smart phone 3782, a personal digital assistant, or other similar mobile device.

[0582] Various implementations of the systems and techniques described herein can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor that can be special or general purpose (coupled to receive data and instructions from a storage system and transmit data and instructions to a storage system), at least one input device, and at least one output device.

[0583] These computer programs (also referred to as programs, software, software applications or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages ​​and / or in assembly / machine language. As used herein, the terms machine-readable medium and computer-readable medium refer to any computer program product, device and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term machine-readable signal refers to any signal for providing machine instructions and / or data to a programmable processor.

[0584] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a presentation device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for presenting information to the user and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input from the user may be received in any form (including sound, voice, or tactile input).

[0585] The systems and techniques described herein may be implemented in a computing system that includes a back-end component (e.g., as a data server); or includes a middleware component (e.g., an application server); or includes a front-end component (e.g., a client computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein) or any combination of such back-end, middleware, or front-end components. The components of the system may be interconnected by any digital data communication form or medium (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0586] A computing system may include clients and servers. Clients and servers are generally remote from each other and generally interact via a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0587] In some embodiments, the various modules described herein may be separated, combined, or incorporated into a single or combined module. The modules depicted in the figures are not intended to limit the systems described herein to the software architectures shown therein.

[0588] The elements of the different embodiments described herein may be combined to form other embodiments not explicitly described above. Elements may be disregarded from the processes, computer programs, databases, etc. described herein without adversely affecting their operation. In addition, the logical flows depicted in the figures do not require the particular order or sequential order shown to achieve the desired results. Various individual elements may be combined into one or more individual elements to perform the functions described herein.

[0589] Throughout this description, where devices and systems are described as having, including, or comprising particular components, or where processes and methods are described as having, including, or comprising particular steps, it is further contemplated that there are devices and systems of the present invention that consist essentially of or consist of the components, and that there are processes and methods of the present invention that consist essentially of or consist of the process steps.

[0590] It should be understood that the order of the steps or the order in which a particular action is performed is not important as long as the present invention remains operable. In addition, two or more steps or actions may be performed simultaneously.

[0591] While the present invention has been particularly shown and described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the following claims.

Claims

1. A method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a prostate of an individual and determining one or more absorption indices indicative of an amount of radiopharmaceutical absorbed therein, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of the individual obtained using an anatomical imaging modality, wherein the 3D anatomical image includes a graphical representation of tissue within the individual, at least a portion of which corresponds to a pelvic region of the individual; (b) receiving, by the processor, a 3D functional image of the individual obtained using a functional imaging modality, wherein the 3D functional image comprises a plurality of voxels, each voxel representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represent a physical volume within a pelvic region of the individual; (c) determining, by the processor using a first module, an initial volume of interest (VOI) within the 3D anatomical image, the initial VOI corresponding to tissue within a pelvic region of the individual and excluding tissue outside the pelvic region of the individual; (d) identifying, by the processor using a second module, a prostate volume within the initial VOI corresponding to a prostate of the individual; and (e) determining, by the processor, the one or more absorption indices using the identified prostate volume within the initial VOI of the 3D functional image and the 3D anatomical image, wherein the first module is a CNN module that receives as input and operates on a downsampled version of the 3D anatomical image having a first resolution, and wherein the second module is a CNN module that receives as input and operates on a high-resolution version of the 3D anatomical image cropped to the initial VOI and having a second resolution that is higher than the first resolution.

2. The method of claim 1, wherein the first module receives the 3D anatomical image as input and outputs a plurality of coordinate values ​​representing relative corners of a rectangular volume within the 3D anatomical image.

3. The method of claim 1 or claim 2, wherein step (c) comprises determining a 3D pelvic bone mask using the first module to thereby identify a volume within the 3D anatomical image corresponding to the individual's pelvic bone.

4. The method according to claim 1 or claim 2, wherein the first module is a convolutional neural network (CNN) module.

5. The method of claim 1 or claim 2, wherein step (d) comprises using the second module to identify one or more additional tissue volumes within the 3D anatomical image, each volume corresponding to a specific tissue region within the subject, wherein the one or more additional tissue volumes correspond to one or more specific tissue regions selected from the group consisting of: a pelvic bone of the individual; the individual’s bladder; the individual's rectum; and A gluteal muscle of the individual.

6. The method of claim 1 or claim 2, wherein step (d) comprises using the second module to classify each voxel within the initial VOI as corresponding to a specific tissue region from a set of different tissue regions within the subject.

7. The method of claim 6, wherein classifying each voxel within the initial VOI comprises: determining, via the second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, wherein the set of likelihood values ​​includes a respective likelihood value for each of one or more tissue regions of the set of tissue regions, the likelihood value representing a likelihood that the voxel represents a physical volume within the tissue region; and For each of the plurality of voxels within the initial VOI, the voxel is classified as corresponding to the particular tissue region based on the set of likelihood values ​​determined for the voxel.

8. The method of claim 6, wherein the second module receives the initial VOI as input and outputs a plurality of values, the plurality of values ​​comprising, for each voxel within the initial VOI, at least one of the following (i), (ii), and (iii): (i) a value for classifying the voxel; (ii) a set of likelihood values ​​for the voxels; and (iii) identifying values ​​of said voxels that do not correspond to any of the predetermined set of different tissue regions.

9. The method of claim 6, wherein the set of different tissue regions comprises one or more tissue regions selected from the group consisting of: a prostate gland of the individual; a pelvic bone of the individual; the individual’s bladder; the individual's rectum; and A gluteal muscle of the individual.

10. The method of claim 5, wherein step (d) comprises using the second module to identify a set of one or more base tissue volumes, the one or more base tissue volumes comprising the identified prostate volume and the one or more additional tissue volumes, and wherein the method further comprises: identifying, by the processor using one or more auxiliary modules, one or more auxiliary tissue volumes within the 3D anatomical image, each auxiliary tissue volume corresponding to the base tissue volume identified by the second module; and Each auxiliary tissue volume is merged, by the processor, with a corresponding base tissue volume identified by the second module.

11. The method according to claim 1 or claim 2, comprising: identifying, by the processor, a reference volume within the 3D anatomical image, the reference volume corresponding to a reference tissue region within the individual; and In step (e), at least one of the one or more absorption indices is determined using the reference volume identified within the 3D functional image and the 3D anatomical image.

12. The method of claim 11, wherein the at least one of the one or more absorption indices determined using the 3D functional image and the reference volume comprises a tumor-to-background ratio (TBR) value, and wherein determining the TBR value comprises: determining a target intensity value using intensity values ​​of one or more voxels in the 3D functional image corresponding to the prostate volume identified within the initial VOI of the 3D anatomical image; determining a background intensity value using intensity values ​​of one or more voxels in the 3D functional image corresponding to the reference volume identified within the 3D anatomical image; and The ratio of the target intensity value to the background intensity value is determined as the TBR value.

13. The method of claim 12, comprising determining the prostate cancer status of the individual based on a comparison of the TBR value to one or more threshold values. The method of claim 13 , wherein the one or more threshold values ​​are determined using a plurality of reference TBR values.

15. The method of claim 13, wherein the one or more thresholds are determined using a receiver operating characteristic (ROC) curve.

16. The method of claim 13, wherein the one or more thresholds include a plurality of thresholds such that comparison of the TBR value with the plurality of thresholds is capable of determining the prostate cancer status according to a level on a non-binary scale.

17. The method of claim 13, comprising determining the prostate cancer status of the individual as follows: (i) clinically significant if the TBR value is above a cutoff threshold; or (ii) clinically insignificant if the TBR value is below the cutoff threshold.

18. The method according to claim 1 or claim 2, comprising: identifying, by the processor, a bladder volume within the 3D anatomical image corresponding to a bladder of the individual; and In step (e), crosstalk from the bladder is corrected using the intensity of voxels in the 3D functional image corresponding to the identified bladder volume in the 3D anatomical image.

19. The method of claim 18, wherein correcting for crosstalk from the bladder comprises: determining one or more bladder intensity dumping functions that model a contribution of an intensity of a radiopharmaceutical originating from within the bladder of the individual to an intensity of one or more voxels of the 3D functional image, the one or more voxels corresponding to one or more regions in the 3D anatomical image that are outside of the identified bladder volume, wherein the one or more bladder intensity dumping functions model the contribution as a function of distance from the identified bladder volume; and For each of one or more voxels of the 3D functional image corresponding to the identified prostate volume within the 3D anatomical image, the intensity of the voxel is adjusted for bladder crosstalk using the one or more bladder intensity dumping functions.

20. The method of claim 1 or claim 2, comprising: identifying, by the processor, a bladder volume within the 3D anatomical image corresponding to a bladder of the individual; determining, by the processor, an expanded bladder volume by applying a morphological expansion operation to the identified bladder volume; and In step (e), the one or more absorption indices are determined using intensity values ​​of voxels of the 3D functional image, the intensity values ​​of the voxels (i) corresponding to the prostate volume identified within the VOI of the 3D anatomical image, but (ii) not corresponding to a region of the 3D anatomical image that is within the expanded bladder volume.

21. The method of claim 1 or claim 2, wherein the 3D functional image is a nuclear medicine image of the subject after the radiopharmaceutical is administered to the subject.

22. The method of claim 21, wherein the radiopharmaceutical comprises a PSMA binder.

23. The method of claim 21, wherein the nuclear medicine image is a single photon emission computerized tomography (SPECT) scan of the subject obtained after the radiopharmaceutical is administered to the subject.

24. The method of claim 21, wherein the radiopharmaceutical comprises 99m Tc-MIP-1404.

25. The method of claim 1 or claim 2, comprising determining one or more diagnostic or prognostic values ​​for the subject based on at least a portion of the one or more absorption indices.

26. The method of claim 25, wherein determining at least one of the one or more diagnostic or prognostic values ​​comprises comparing an absorption index to one or more threshold values.

27. The method of claim 25, wherein at least one of the one or more diagnostic or prognostic values ​​estimates the individual's risk of developing clinically significant prostate cancer.

28. The method of claim 1 or claim 2, comprising: (f) causing display, by the processor, of an interactive graphical user interface GUI to present a visual representation of the 3D anatomical image and / or the 3D functional image to the user; and (g) generating, by the processor, a graphical representation of the 3D anatomical image and / or the 3D functional image within the GUI as selectable and overlappable layers so that either one can be selected for display and rendered separately, or both can be selected for display and rendered together by overlaying the 3D anatomical image with the 3D functional image.

29. The method of claim 28, wherein step (g) comprises generating a graphical representation of a selectable and overlappable segmentation layer comprising one or more identified specific tissue volumes within the 3D anatomical image, wherein upon selection of the segmentation layer for display, a graphic representation of the one or more specific tissue volumes is superimposed on the 3D anatomical image and / or the 3D functional image.

30. The method of claim 29, wherein the one or more specific tissue volumes comprises the identified prostate volume.

31. The method according to claim 28 comprises, in step (g), causing the 2D cross-sectional view of the 3D anatomical image and / or the 3D functional image to be reproduced in an interactive 2D viewer so that the user can adjust the position of the 2D cross-sectional view.

32. The method of claim 28, comprising, in step (g), causing an interactive 3D view of the 3D anatomical image and / or the 3D functional image to be rendered.

33. The method of claim 28, comprising causing display of a graphical element within the GUI, the graphical element indicating locations of voxels corresponding to the identified prostate volume, thereby facilitating user review and / or quality control of the method.

34. The method of claim 28, comprising causing text and / or graphics to be displayed within the GUI, and a quality control graphic widget, the text and / or graphics representing the one or more absorption indices determined in step (e), the quality control graphic widget for guiding the user through a quality control and reporting workflow to review and / or update the one or more absorption indices.

35. The method of claim 34, comprising: receiving user input via the quality control graphical widget, the user input corresponding to an approved automated determination of one or more absorbance indices; and The processor, in response to the receipt of the user input corresponding to the approved automated determination of the one or more absorption indicators, generates a report for the individual, the report including a representation of the one or more automated determined absorption indicators.

36. The method of claim 34, comprising: receiving, via the quality control graphical widget, a user input corresponding to an unapproved automated determination of one or more absorbance indices; the processor, in response to receipt of the user input corresponding to the unauthorized automated determination of the one or more absorption indices, causing a voxel selection graphical element to be displayed for user selection of one or more voxels of the 3D functional image for determining updated values ​​of the one or more absorption indices; receiving, via the voxel selection graphical element, one or more voxels of the 3D functional image selected by the user for determining updated values ​​of the one or more absorption indices; updating, by the processor, values ​​of the one or more absorption indices using the user-selected voxels; and A report is generated, by the processor, for the individual, the report including a representation of the one or more updated absorption indicators.

37. The method of claim 34, comprising: receiving, via the quality control graphical widget, a user input corresponding to an automated determination of the one or more absorption indices being unapproved; receiving, via the quality control widget, user input corresponding to rejecting quality control; and A report for the individual is generated by the processor, wherein the report includes an identification of quality control rejections.

38. The method of claim 1 or claim 2, wherein voxels of the 3D functional image are related to voxels of the 3D anatomical image via a known relationship.

39. The method of claim 1 or claim 2, comprising performing steps (a) and (c) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each initial VOI being within one of the plurality of 3D anatomical images, wherein a size variability of the initial VOI is less than a size variability of the 3D anatomical image.

40. A method according to claim 1 or claim 2, wherein the first module receives at least a portion of the 3D anatomical image as input, and wherein the physical volume represented by the input to the first module is (i) at least twice the physical volume represented by the initial VOI and / or (ii) twice the physical volume represented by the initial VOI in at least one dimension.

41. A method according to claim 1 or claim 2, wherein the first module is a CNN module trained to identify graphical representations of the pelvic region within a 3D anatomical image and wherein the second module is a CNN module trained to identify graphical representations of prostate tissue within a 3D anatomical image.

42. A method for automatically processing a 3D image to identify a 3D volume within the 3D image corresponding to a target tissue region and determining one or more absorption indices indicative of an amount of radiopharmaceutical absorbed therein, the method comprising: (a) receiving, by a processor of a computing device, a 3D anatomical image of an individual obtained using an anatomical imaging modality, wherein the 3D anatomical image comprises a graphical representation of tissue within the individual; (b) receiving, by the processor, a 3D functional image of the individual obtained using a functional imaging modality, wherein the 3D functional image comprises a plurality of voxels, each voxel representing a specific physical volume within the individual and having an intensity value representing detected radiation emitted from the specific physical volume, wherein at least a portion of the plurality of voxels of the 3D functional image represent a physical volume within a specific anatomical region including the target tissue region; (c) determining, by the processor using a first module, an initial volume of interest (VOI) within the 3D anatomical image, wherein the initial VOI corresponds to the specific anatomical region; (d) identifying, by the processor using a second module, a target volume within the initial VOI corresponding to the target tissue region of the individual; and (e) determining, by the processor, the one or more absorption indices using an identified target volume within the initial VOI of the 3D functional image and the 3D anatomical image, wherein the first module is a CNN module that receives as input and operates on a downsampled version of the 3D anatomical image having a first resolution, and wherein the second module is a CNN module that receives as input and operates on a high-resolution version of the 3D anatomical image cropped to the initial VOI and having a second resolution, which is higher than the first resolution.

43. The method of claim 42, wherein the first module receives the 3D anatomical image as input and outputs a plurality of coordinate values ​​representing relative corners of a rectangular volume within the 3D anatomical image.

44. A method according to claim 42 or 43, wherein the first module is a convolutional neural network (CNN) module.

45. The method of claim 42 or 43, wherein step (c) comprises identifying, using the second module, one or more additional tissue volumes within the 3D anatomical image, each additional tissue volume corresponding to a specific tissue region within the individual.

46. ​​The method of claim 42 or 43, wherein step (c) comprises using the second module to classify each voxel within the initial VOI as corresponding to a specific tissue region from a set of different tissue regions within the subject.

47. The method of claim 46, wherein classifying each voxel within the initial VOI comprises: determining, via the second module, a set of likelihood values ​​for each of a plurality of voxels within the initial VOI, wherein the set of likelihood values ​​includes a respective likelihood value for each of one or more tissue regions of the set of tissue regions, the likelihood value representing a likelihood that the voxel represents a physical volume within the tissue region; and For each of the plurality of voxels within the initial VOI, the voxel is classified as corresponding to the particular tissue region based on the set of likelihood values ​​determined for the voxel.

48. The method of claim 46, wherein the second module receives the initial VOI as input and outputs a plurality of values, the plurality of values ​​comprising, for each voxel within the initial VOI, at least one of (i), (ii), and (iii): (i) a value for classifying the voxel; (ii) a set of likelihood values ​​for the voxels; and (iii) identifying values ​​of said voxels that do not correspond to any of the predetermined set of different tissue regions.

49. The method of claim 42 or 43, wherein step (c) comprises using the second module to identify a set of one or more base tissue volumes, the one or more base tissue volumes comprising the identified target volume and the one or more additional tissue volumes, and wherein the method further comprises: identifying, by the processor using one or more auxiliary modules, one or more auxiliary tissue volumes within the 3D anatomical image, each auxiliary tissue volume corresponding to the base tissue volume identified by the second module; and Each auxiliary tissue volume is merged, by the processor, with a corresponding base tissue volume identified by the second module.

50. The method of claim 42 or 43, comprising: (e) causing display, by the processor, of an interactive graphical user interface GUI to present a visual representation of the 3D anatomical image to the user; and (f) generating, by the processor within the GUI, a graphical representation of the 3D anatomical image and selectable and overlappable segmentation layers comprising one or more identified specific tissue volumes within the 3D anatomical image, wherein when the segmentation layer is selected for display, graphics representing the one or more specific tissue volumes are superimposed on the 3D anatomical image.

51. The method of claim 50, wherein the one or more specific tissue volumes comprises the identified target volume.

52. The method of claim 50, comprising, in step (f), causing the 2D cross-sectional view of the 3D anatomical image to be reproduced in an interactive 2D viewer so that the user can adjust the position of the 2D cross-sectional view.

53. The method of claim 50, comprising, in step (f), rendering an interactive 3D view of the 3D anatomical image.

54. The method of claim 42 or 43, comprising performing steps (a) and (b) for each of a plurality of 3D anatomical images to determine a plurality of initial VOIs, each initial VOI being within one of the plurality of 3D anatomical images, wherein a size variability of the initial VOI is less than a size variability of the 3D anatomical image.

55. A method according to claim 42 or 43, wherein the first module receives at least a portion of the 3D anatomical image as input, and wherein the physical volume represented by the input to the first module is (i) at least twice the physical volume represented by the initial VOI and / or (ii) twice the physical volume represented by the initial VOI in at least one dimension.

56. A method according to claim 42 or 43, wherein the first module is a CNN module trained to identify a graphical representation of the specific anatomical region within a 3D anatomical image and wherein the second module is a CNN module trained to identify a graphical representation of the target tissue region within a 3D anatomical image.

57. The method of claim 42, wherein voxels of the 3D functional image are related to voxels of the 3D anatomical image via a known relationship.

Images