Method and uses for region of interest registration between imaging modalities

Abstract

The disclosure relates to systems and methods for locating region of interest (ROI) within a predetermined boundaries of a scene of interest (SOI) among different imaging modalities used in various medical procedures, regardless of the originating image quality. Specifically, the disclosure is directed to methods and systems for locating ROI in the intraoperative (or second) imaging modality, wherein the ROI is accurately discernable only in the preoperative (or first) imaging modality of the SOI and not in the intraoperative (or second) imaging modality, by registering a collection of extracted, patient-specific stochastic elements within the boundaries of the scene of interest (SOI) detectable by each imaging modality used.

Description

37005PCMETHOD AND USES FOR REGION OF INTEREST REGISTRATION BETWEEN IMAGING MODALITIES BACKGROUND

[0001] The present disclosure is directed to systems and methods for locating a region of interest (ROI) within a predetermined boundaries of a scene of interest (SOI) among different imaging modalities used in various medical procedures. Specifically, the disclosure is directed to methods and systems for locating ROI in the intraoperative (or second) imaging modality, wherein the ROI is accurately discernable only in the preoperative (or first) imaging modality of the SOI and not in the intraoperative (or second) imaging modality, by registering a collection of extracted, patient-specific stochastic elements within the boundaries of the scene of interest (SOI) detectable by each imaging modality used.

[0002] The fusion of MRI and CT images typically demonstrates spatial registration errors ranging from 0.5 to 2.5mm, representing a percentage error of 1-3% in most applications. This error margin becomes particularly significant in neurosurgery and stereotactic procedures where submillimeter precision is crucial. The primary contributors to these discrepancies include patient positioning variations, respiratory motion, and inherent differences in image acquisition parameters between modalities.

[0003] When fusing ultrasound with either MRI or CT, the registration error typically increases to 2-4mm (3-5% error) due to tissue deformation during ultrasound probe contact and the real-time nature of ultrasound imaging. This presents particular challenges in soft tissue procedures where organ shift and deformation are common. The development of electromagnetic tracking systems has helped reduce these errors, but the fundamental challenge of tissue elasticity remains.

[0004] X-ray fusion with 3D modalities like CT or MRI demonstrates varying accuracy depending on the anatomical region and patient positioning. In spinal procedures, registration errors typically range from l-3mm (2-4% error), while in peripheral skeletal procedures, the error may increase to 2-4mm (3-6% error) due to limb positioning variations and projection differences.

[0005] In clinical practice, these fusion technologies are being actively employed across various surgical specialties. In neurosurgery, multimodal image fusion guides tumor resection by combining the superior soft tissue contrast of MRI with the bone detail of CT, enabling more precise surgical navigation. The percentage error becomes particularly critical when operating near eloquent brain areas, where even a 2mm deviation could have significant consequences.37005PC

[0006] Cardiothoracic surgery has employed fusion imaging for complex procedures such as transcatheter aortic valve replacements (TAVR). Here, real-time fluoroscopy is fused with preprocedural CT angiography, allowing for more accurate valve positioning. The fusion error in these cases typically ranges from 2-3mm (2-4% error), which is generally acceptable given the larger anatomical structures involved yet additional registration accuracy will help in reducing healing time.

[0007] Likewise, in hepatobiliary surgery, the fusion of contrast-enhanced CT with intraoperative ultrasound has revolutionized liver tumor resection planning. This combination provides real-time guidance while maintaining awareness of critical vascular structures. The registration error in liver procedures tends to be higher (3 -5mm, or 4-7% error) due to significant organ deformation during breathing and manipulation. Also, orthopedic surgery has particularly benefited from image fusion in spine procedures, where CT-fluoroscopy fusion guides pedicle screw placement with reported accuracy rates of 96-98%. The remaining 2-4% error margin typically relates to patient movement and registration accuracy between preoperative and intraoperative imaging.

[0008] In prostate procedures, MRI-ultrasound fusion has become increasingly important for targeted biopsies and focal therapy. Current systems demonstrate targeting errors of 2-3mm (3-5% error), which has significantly improved the detection rates of clinically significant cancers compared to conventional ultrasound-guided procedures.

[0009] Primary error sources include patient movement, tissue deformation, organ shift between imaging sessions, and tracking system limitations in addition to the challenge of interpreting the same scene of interest as it is imaged / processed differently by each different imaging modality, according to the different mechanism by which each imaging modality gathers information. The advancement of artificial intelligence and deep learning algorithms is gradually improving these fusion accuracies. Modern neural networks can now predict and compensate for organ deformation, potentially reducing registration errors by 20-30% compared to traditional rigid registration methods. However, these improvements are still in various stages of clinical validation and implementation.

[0010] Real-time updating of fused images remains a significant challenge, particularly in procedures involving soft tissue deformation. The time lag between image acquisition and fusion processing, though improved with modern computing power, still introduces additional uncertainty in the 1 -2% range, which must be considered during surgical planning and execution.

[0011] Accordingly, there is a need to develop solutions to assist the current fusion shortcoming of the preoperative and intraoperative imaging modalities used.37005PCSUMMARY

[0012] Disclosed, in various exemplary implementations, are methods and systems for performing imaging fusion and / or alignment between preoperative and intraoperative imaging modalities by registering a collection of extracted, patient-specific stochastic elements within a predetermined boundaries of a scene of interest (SOI) detectable by each imaging modality.

[0013] In an exemplary implementation, provided herein is method of registering a plurality of images of a predetermined scene of interest (SOI), wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, the method comprising: for each imaging modality, identifying a plurality of patient-specific stochastic elements within the SOI; extracting the identified patient-specific stochastic elements from each image of the plurality of images, thereby forming patient-specific SOI fingerprint; and registering the patient specific SOI fingerprint across the plurality of images, forming at least one fused and / or aligned image, and / or calculating ROI coordinates.

[0014] In another exemplary implementation, provided herein is a system for registering a plurality of images of a predetermined region of interest, wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, the system comprising: a preoperative, or first imaging modality; an intraoperative, or second imaging modality; a registration module; and an operating room user interface.

[0015] These and other features of the methods and systems for performing imaging fusion and / or alignment between preoperative (or first) and intraoperative (or second) imaging modalities by registering a collection of extracted, patient-specific stochastic elements within a predetermined boundaries of a scene of interest (SOI) detectable by each imaging modality will become apparent from the following detailed description when read in conjunction with the drawings, which are exemplary, not limiting.BRIEF DESCRIPTION OF THE FIGURES

[0016] The figures along with the detailed description are incorporated and form part of the specification and serve to further illustrate examples, exemplary implementations, and the like, and explain various principles and advantages, in accordance with this disclosure, in which:

[0017] FIG. 1, is an example of the difference between the contour of the prostate in MRI and US imaging modalities;37005PC

[0018] FIG. 2, is an example of using sporadically occurring elements inside the prostate to accurately align and fuse the MRI and US images in a cognitive manner;

[0019] FIGs. 3A-3E, are exemplary implementations of the systems disclosed; and

[0020] FIG. 4A depicts a base US image, with FIG. 4B depicting a base MRI image, with FIG.4C depicting binding box around the identified prostate.

[0021] FIG. 5, is a flowchart showing the steps involved in modality-dedicated use of AI / ML for extraction of stochastic elements, and their selective integration into the fusion process; and

[0022] FIG. 6, is a schematic illustration of an exemplary implementation of the system for deploying modality-dedicated AI / ML for extraction of stochastic elements, and their selective integration into the fusion process.DETAILED DESCRIPTION

[0023] Provided herein are exemplary implementations of methods and systems for registering scenes of interest (SOIs) across different imaging modalities by extracting a collection of patientspecific stochastic elements within a predetermined boundaries of the scene of interest (SOI) detectable by each imaging modality.

[0024] Many different procedures, for example biopsies and minimally invasive surgeries, require the operating medical professional to locate the intended region of interest (ROI) precisely during the operation, using various imaging technologies. Some of these medical procedures, require the use of more than one imaging modality. This is due to the fact that the preoperative imaging modality with which the ROI is identifiable, cannot be used intraoperatively during the necessary medical operation to locate the ROI in real-time, and the intraoperative imaging modality used to guide the operating medical professional cannot be used to directly identify the exact location of the ROI.

[0025] An example for one of these medical scenarios is in the diagnosis and in some treatments of prostate cancer. Prostate cancer is the leading cancer diagnosis among men and the second most common cancer diagnosis overall (in both men and women) with about 300,000 annual new diagnosed prostate cancer cases in the US alone as of 2024. The diagnosis process for prostate cancer starts with initial testing, for example a prostate-specific antibodies (PSA) blood test, which, under certain circumstances when viewed through testing history, can indicate a significant risk for prostate cancer. To diagnose prostate cancer properly and to assess the grade of the cancer, a biopsy must remove tissue of the suspected tumor (i.e. biopsy) successfully, which will then be analyzed.37005PCRemoving tissue from the intended region accurately, is essential because if the removed tissue is not of the suspected tumor but of a nearby, healthy prostate tissue, the results of the biopsy will be a false negative, which will not indicate anything regarding the potential tumor and the results will be of a normal healthy tissue.

[0026] To biopsy the suspected tumor effectively, an MRI scan will be conducted. The suspected tumor will show up on an MRI scan as having different tissue properties relative to the rest of the healthy prostate. Among the many imaging modalities available in the medical field, an MRI is required specifically for locating the suspected tumor because of the requirement to differentiate between the healthy tissue and potential tumor. However, using MRI imaging modality intraoperatively is not possible, as the biopsy procedure itself cannot be conducted while the patient is being imaged inside an MRI machine. Alternatively, a PET scan may be used with similar limitations. Consequently, another imaging modality must be used intraoperatively to guide the surgeon conducting the biopsy, enabling the surgeon to ascertain that they are indeed removing tissue from the suspected tumor (the Region of Interest - ROI).

[0027] This intraoperative imaging modality, in the case of prostate cancer biopsies, is an ultrasound (US) probe inserted transrectally (TRUS). The fact that the ROI cannot be distinguished from healthy prostate tissue using US and can only be identified when observed via MRI raises a challenge. Consequently, fusing of the preoperative MRI image and the intraoperative US image is needed, the MRI image to find the exact location of the ROI (in other words, the suspected tumor), and the US image to be able to guide the surgeon in real-time to that region. This process of initially imaging the prostate using MRI, and then during the operation using US to be able to guide the operating medical professional is also necessary during minimally invasive surgeries that are used to treat prostate cancer (for example, during seed implantation, High-Intensity Focused Ultrasound (HIFU), cryotherapy, Focal Irreversible Electroporation (IRE), Brachytherapy, Focal laser ablation (FLA), and the like).

[0028] Typically, the technologies for fusing these two images consists of matching the contour (the outline bounding the shape of the organ) of the prostate, in each of the two modalities, and adjusting the US contour and shape to fit the MRI contour and shape so that, in theory, the ROI in the MRI aligns with the correct place in the “new” US-fused image. This “new” US-fused image is then used to guide the surgeon to the ROI. There is an inherent deficiency when using this approach to fuse the images (the contour-led approach), which results in a misalignment between the correct position of the ROI and the position represented in the “new” US-fused image which guides the surgeon.37005PC

[0029] This significantly increases the risk of missing the suspected tumor (the ROI) and taking a biopsy of healthy tissue instead (again, potentially resulting in a false negative diagnosis, or a wrong grade given to the tumor) requiring another biopsy. The cause of this inherent misalignment of the fused image when using the contour-based approach is the difference between the efficacy of each one of the different imaging modalities (MRI and US) when imaging the edges of the organ (the edges, or boundaries of the prostate when it connects to the connective tissue and surrounding organs). This approach is used by many systems to direct the operating surgeon to the estimated location of the ROI after fusing the images.

[0030] As illustrated in FIG. 1, depicting the difference between the contour of the prostate in MRI and US imaging modalities. In each of these images (A, B and C), the difference between the contour of the prostate based on the MRI image (internal circle) and based on the US image (external circle). These differences will lead to a misaligned fused image, where the new proposed location of the ROI does not correspond to the location which was found on the MRI (before the fusion process). This may lead the surgeon to biopsy healthy tissue instead of the ROI and result in a misdiagnosis.

[0031] Provided herein, in an exemplary implementation, is a novel approach to solve this challenge of identifying the location of a ROI (which is identifiable in one image from one imaging modality) in another image (from an imaging modality in which the ROI is not readily, and / or accurately identifiable) of the same organ / tissue / area. This approach utilizes one or more stochastic morphological / tissue elements / irregularities / abnormalities which appear and develop in sporadic, random, chaotic or unexpected manner in the target tissue / organ / area and are discernable in both imaging modalities (hereinafter referred to as “stochastic elements”).

[0032] Accordingly, and as illustrated in FIG. 5, provided herein, is a method of registering a plurality of images of a predetermined scene of interest (SOI), wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, the method comprising: for each image 100, identifying patient specific stochastic element(s) (in other words, the minimal number of stochastic elements ensuring accuracy of the SOI registration) 106, within the SOI; extracting 108 the identified patient-specific stochastic element(s) from each image of the plurality of images, thereby forming patient-specific SOI fingerprint; and registering 116 the patient specific SOI fingerprint across the plurality of images. Or, in other words, aligning the images according to the fingerprint comprised of the collection of the patient-specific stochastic elements.37005PC

[0033] It is noted, that in certain exemplary implementation of the methods and systems disclosed, the term “scene of interest” (SOI) refers to the tissue or organ where the region of interest (ROI, e.g., tumor, implant site, biopsy target and the like) is subsumed. In other exemplary implementations of the methods and systems disclosed, the term “scene of interest” (SOI) can refer to the tissue or organ where the region of interest (ROI, e.g., tumor, implant site, biopsy target and the like) is subsumed but also to other surrounding relevant tissues or organs.

[0034] These patient-specific, stochastic elements which appear and develop in a sporadic, random, chaotic or unexpected manner are stochastic in their aspect (which specific morphological / tissue element), their location (they can appear throughout the tissue / organ / area), their shape, size or volume. These elements are distinct from morphological elements / anatomical features which anatomical literature states are predetermined (the structure of an organ), or are generally constant (e.g. the expected general shape / size of an organ e.g., the contour of the prostate, or the location of different organs with respect to one another), or that their attributes or their known differentiation / variation between different people or over a person’s lifetime can be predicted (the width of the muscular wall of the left ventricle of the heart).

[0035] For example, numerous patient-specific stochastic elements can appear within the SOI. For example, inflammatory changes can appear randomly throughout tissues, manifesting as areas of edema, infiltrates, or local tissue reactions. These can vary significantly in size and distribution, often showing different characteristics across imaging modalities. Likewise, calcifications represent another common stochastic element, appearing as focal densities in various organs. These can range from microcalcifications in breast tissue to larger dystrophic calcifications in chronic inflammatory conditions (e.g., atherosclerotic plaques, damaged heart valves, and lymph nodes in the presence of tuberculosis infection). In addition, fibrotic changes and scarring occur unpredictably following injury, infection, or therapeutic interventions. These elements can distort normal tissue architecture and create variable signal characteristics across different imaging modalities. Moreover, post-surgical scarring is especially challenging as it can change over time. Cystic formations, can develop sporadically in various organs. These fluid-filled structures range from simple cysts to complex septated lesions, appearing with different signal intensities depending on their content and the imaging modality used.

[0036] Neovascularization and vascular malformations represent yet another category of stochastic elements. These can develop randomly and show complex branching patterns that may be difficult to identify consistently across different imaging modalities. So does tissue metaplasia and37005PCdysplasia, which occur unpredictably and can present with varying imaging characteristics. These cellular changes may be subtle on some modalities while more apparent on others.

[0037] Then there are foreign bodies, such as surgical clips, stents, or other implanted materials, which create imaging artifacts that vary between modalities. Their presence can significantly impact image fusion accuracy in the surrounding tissues. Furthermore, focal areas of necrosis or tissue degeneration can develop randomly within organs, presenting as regions of altered signal intensity or density. Their appearance often varies significantly between imaging modalities. Likewise hemorrhagic foci can appear spontaneously or post-intervention, showing different characteristics depending on their age and the imaging modality used. Their evolution over time adds another layer of complexity to image fusion but also are beneficial for creating the patient-specific SOI fingerprint.

[0038] Granulomas represent yet another stochastic element, forming in response to various stimuli and appearing as focal lesions with varying imaging characteristics depending on their composition and stage of development, which add to their uniqueness in forming the patient-specific SOI fingerprint. Local tissue reactions to therapeutic interventions, such as radiation changes or postablation effects, create areas of altered tissue architecture that may appear differently across imaging modalities as do fatty infiltration or degeneration, which can occur sporadically within organs, creating areas of altered signal intensity that may be more or less apparent depending on the imaging technique used.

[0039] These stochastic elements are beneficial in creating the procedure-specific, and the patient-specific SOI fingerprint. The formation of the patient-specific SOI fingerprint, will therefore, and in an exemplary implementation, be a function of the specific surgical procedure, the preoperative imaging modality used, and the intraoperative modality used.

[0040] For example, variations in the width of the muscular walls of the left ventricle of the heart can be expected (e.g. in hypertrophic cardiomyopathy - HCM). Conversely, there are variations and elements which anatomical study determines are unexpected and cannot be predicted. This collection, or “constellation” of patient specific stochastic elements form together a unique “fingerprint” which is different between any two patients, but is also different and changes throughout each patient’s own lifetime.

[0041] In an exemplary implementation, the patient-specific stochastic elements used to form the SOI fingerprint used in fusing the preoperative image is: inflammatory changes, and the intraoperative image, inflammatory infiltrates, micro calcifications, macro calcifications, fibroses, scarring, simple cysts, complex cysts, neovascularization, vascular malformations, tissue metaplasia,37005PCtissue dysplasia, surgical clips, stents, implants, necrotic areas, hemorrhagic foci, granulomas, radiation induced tissue change, ablation induced morphological changes, fatty infiltration, fatty degeneration, edema, abscesses, tissue mineralization, lymphoid aggregates, focal atrophy, post-traumatic changes, adhesions, microlithiasis, focal hypertrophy, amyloid tissue deposits, iron deposits, focal fibrosis, or patient-specific stochastic elements comprising one or more of the foregoing.

[0042] As illustrated in an exemplary implementation in FIG. 2, stochastic elements within the prostate (the SOI), can be used to accurately align (interchangeable with “register”, or “map”) and fuse preoperative MRI image and intraoperative US image in a cognitive manner. As shown, in each row, (i) and (ii), includes three images of a prostate (one prostate per row). The left-most images (MRI(i) and MRI(ii)) are the preoperative MRI images of each prostate. The ROI (the subsumed circles), is marked on each MRI image. The stochastic elements, in this case (numbered) cysts, are also marked on the images. The middle images (US(i) an US(ii)) are intraoperative US images. The stochastic elements seen in the MRI are marked on the US images with their respective numbers. The right-most images (Fusion(i) and Fusion(ii)) are the fused images including markers for the ROI, and the edges of the prostate (e.g., the SOI) as seen in the MRI.

[0043] In an exemplary implementation, the preoperative imaging modality 100 used in the methods and systems disclosed can be, for example: Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Imaging, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy, X-ray, positron emission tomography (PET), PET-CT, single-photon emission computerized tomography (SPECT), or a combination of imaging modality comprising one or more of the foregoing, while the intraoperative imaging modality can be, for example, Fluoroscopy, Real-time 3D Ultrasound (4D US), Contrast-Enhanced Ultrasound (CEUS), Intraoperative CT, Intraoperative MRI, Near-Infrared Fluorescence Imaging, Surgical Navigation, Electromagnetic Navigation Systems, Augmented Reality Systems, Optical Surface Scanning, Intraoperative OCT, Photoacoustic Imaging, US-based Elastography, MR-based Elastography, Laser Speckle Contrast Imaging, or a combination of imaging modality comprising one or more of the foregoing. Additional imaging modalities that can be used include, for example, Cerenkov Luminescence Imaging, Raman Spectroscopy Imaging, Hyperspectral Imaging, Molecular Imaging techniques, and various Optical Imaging modalities being currently developed for intraoperative use.37005PC

[0044] In an exemplary implementation, the surgical operation is transcatheter aortic valve replacement (TAVR), the intraoperative imaging modality is fluoroscopy, the preoperative image modality is CT Angiography, and the stochastic elements used to fuse the CT-fluoroscopy images can be calcifications in the aortic valve and annulus, which are prominently visible on both imaging modalities. In CTA, calcifications are quantified and mapped to assess procedural risks and guide device selection. During fluoroscopy, these calcifications serve as anatomical landmarks for valve positioning. Likewise anomalous coronary ostia, e.g., abnormal number of coronary ostia, malposition of a coronary ostium, and stenosis or atresia of a coronary ostium visible on both imaging modalities, can be used to form the SOI fingerprint.

[0045] In another exemplary implementation, the surgical operation is a hepatobiliary surgery, the intraoperative imaging modality is ultrasound, the pre-surgical operation image modality is contrast-enhanced CT, and the stochastic elements used to form the SOI fingerprint can be, for example, calcifications in the liver parenchyma, biliary tree, or vascular structures, all which are detectable on both contrast-enhanced CT and intraoperative ultrasound. These may include calcified biliary stones, vascular plaques, or calcifications associated with chronic infections or fibrosis. Cysts, whether simple or complex (such as hydatid cysts), are also visible on both modalities, with features like septations or calcified walls being identifiable. Fibrotic changes within the liver parenchyma, which may result from prior inflammation or injury, can appear as hypodense regions on CT and hyperechoic areas on ultrasound. Biliary stones, including gallstones or intrahepatic stones, are seen as hyperdense structures on CT and echogenic foci with posterior acoustic shadowing on ultrasound. Additionally, foreign bodies or metallic surgical clips from prior procedures can be identified as dense artifacts on CT and echogenic foci with shadowing on ultrasound. These stochastic morphological elements can be used to form the SOI fingerprint and used to accurately and precisely fuse the intra-and preoperative images.

[0046] In yet another exemplary implementation, the surgical operation is a pedicle screw placement, the real time imaging modality is fluoroscopy, the pre-surgical operation image modality is CT, and the stochastic elements can be, for example; calcifications, such as those in the vertebral body or surrounding structures, which can be identified as dense areas on CT and appear as radiopaque regions on fluoroscopy. Bone density variations, including sclerotic or osteoporotic changes, are also detectable, with CT providing detailed density measurements and fluoroscopy showing differences in radiopacity. Additionally, cortical breaches or irregularities caused by prior trauma or degenerative37005PCchanges may also be visible on both modalities, with CT offering precise localization and fluoroscopy providing real-time confirmation during screw placement.

[0047] In an exemplary implementation, the surgical operation is a prostate operation, the intraoperative imaging modality is trans rectal US (TRUS), and the preoperative imaging modality is MRI. For example, each prostate (as with other organs) develops physiological changes over the lifetime of the individual, in a stochastic manner. These changes also may include anatomical landmarks as prostate cysts, prostate calcifications, prostate arteries and veins, post operative defects as after TURP, HOLEP etc., Post operative scarring as after cryotherapy, Irreversible electroporation, High Intensity Focal Ultrasound, Laser ablation etc., artificial elements as marker seeds (inserted prior to radiation), Brachytherapy seeds etc., Indwelling catheter, Gel Spacer, stents, Uro-lift tackers etc., cancerous lesion, post radiation changes.

[0048] Many of these elements can be successfully spotted in more than one imaging modality, for example these elements can be identified both using MRI and using US. Accordingly, and in an exemplary implementation, instead of solely relying on the contour and edges of the prostate to fuse between the MRI and US images, these elements (which are visible in both modalities) can be used to register the US image to the MRI images, placing the ROI in the correct position in the new fused image assisted by a superimposed grid or for example to align between corresponding images of the two imaging modalities. It is noted that in this exemplary implementation of the system, these sporadic elements vary greatly and are very different between each patient; for example, one patient may have 3 visible cysts which will be used by the system, while another will have 1 cyst, 2 calcifications and a blood-vessel junction which will be used by the system. Moreover, these elements might change over the course of a patient’s lifetime. This specific exemplary implementation of the system, might have 2 cysts and 1 calcification to use for a specific patient at one point of his life, but at a significantly later point of his life, the system might be able to only find 1 cyst but 3 calcifications.

[0049] Since medical images are two dimensional (2D), while the body is three dimensional (3D), many medical images are taken and presented as “slices” of different depths into the organ. When imaging an organ (in other words, imaging the SOI in order to locate the ROI) the ROI might appear in one or more of the slices (depending on factors such as imaging modality, tumor size and scanning angle), but the organ (the SOI) itself may be made up of more slices.

[0050] For example, an MRI system is configured to generate a preoperative image set of a typical prostate in a 60-year-old male patient, wherein the imaging controller automatically selects an axial imaging volume encompassing the prostate and seminal vesicles and prescribes a series of37005PCapproximately 20-30 contiguous axial slices extending from the urogenital diaphragm to a superior level above the seminal vesicle tips, each slice having a thickness of about 3 mm such that the interslice distance corresponds substantially to the slice thickness with no intervening gap, in accordance with or exceeding PI-RADS-compliant multiparametric prostate MRI guidelines for spatial resolution and anatomical coverage, while still allowing dynamic adjustment of field of view, exact slice count, and angulation based on patient-specific prostate size, orientation, and scanner constraints to ensure complete coverage and optimal depiction of relevant anatomical structures for preoperative planning. With 3 mm contiguous slices, a typical 60-year-old male prostate (about 3-5.5 cm craniocaudal length) will usually be covered by roughly 10-18 slices dedicated to the gland itself, with the remaining slices in the 20-30-slice stack covering tissue inferior to the apex and superior toward and through the seminal vesicles for preoperative context and planning.

[0051] In an exemplary implementation, the systems and methods disclosed do not necessarily require that the ROI will appear (in the initial / preoperative imaging modality) in the same “slice” as the patient-specific stochastic elements (which are identifiable in both preoperative and intraoperative imaging modalities). Likewise, the systems and methods disclosed do not require that all the elements appear in the same “slice”. For example, out of 12 “slices” of a specific organ or SOI imaged by the initial imaging modality; one may contain the ROI, a second slice will contain two such stochastic elements, and a third will contain a final stochastic element. Each one of those elements, will be identifiable in the other imaging modality too (while the ROI may not be) though the other imaging modality might image the entire organ or SOI using 18 “slices”. The number of slices proportional to the size of the ROI and the imaging angle.

[0052] Different imaging modalities may have a different “thickness” of each “slice”.Accordingly, the other imaging modality in this example may have 18 “slices” (each one, two thirds the thickness of the slices from the initial imaging modality). The system could then use the aforementioned stochastic elements in the target, or predefined SOI (e.g., prostate) and which are discernable in both imaging modalities to align the two imaging modalities of the entire organ (i.e. the entire two sets of “slices”) and in such a manner find the location of the ROI in the imaging modality where the ROI may not discernable.

[0053] In an exemplary implementation, the steps of identifying 106 and extracting 108 the patient-specific stochastic elements, forming the SOI fingerprint and registering the fingerprint across the images obtained from the different imaging modalities is carried out using machine learning (ML) (see e.g., FIG. 5, 200), and / or non-specialized or specialized neural networks (Al) trained on data37005PCwhich is labeled / aligned / annotated by uniquely expert medical professionals and practitioners (in other words, supervised learning - initially). In an exemplary implementation of the systems and methods disclosed, this process will include receiving two sets of images (i.e. two sets of “slices”) of the same SOI / organ / tissue / area from the same patient. These images will be aligned (so that each image from one modality will have a corresponding aligned image from another modality) by uniquely expert medical professionals and practitioners. This database will allow the neural network to learn how to locate the elements which appear in both imaging modalities on each set of corresponding images, and which correspond with each other.

[0054] This ability can be used in certain exemplary implementations by the neural network to align (or register) images from different modalities and output aligned images and / or fused images 120 (see e.g., FIG. 5). According to an exemplary implementation of the methods disclosed using the systems provided, due to the use of the sporadically and / or stochastically occurring morphological / anatomical elements (e.g. cysts, calcifications, arteries, veins, post operative defects (such as after TURP, HOLEP, etc.) or scars (such as following cryotherapy, electroporation, HIFU, laser ablation, etc.), artificial elements (such as marker seeds, Brachytherapy seeds, etc.), indwelling catheters, Gel Spacer stents, Uro-lift tackers, cancerous lesion, post radiation changes, topography, etc.), rather than specific method for identifying these elements within the SOI in the different imaging modalities, this approach may use specifically trained ML / Al module 30 (see e.g., FIG. 6) which were trained on sets of images from different imaging modalities 110, or in another exemplary implementation, ML / AI / algorithms from any source which identify such elements in the tissue of interest in a certain modality. Following this initial identification of such elements in each of the imaging modalities (e.g. separately) a “constellation” of identified elements will be acquired for each imaging modality 32 (see e.g., FIG. 6), another ML / AI / algorithm can be used 34 (see e.g., FIG. 6) to align 112, as accurately as possible, the two “constellations” from the two imaging modalities. Since the identification of these elements in each modality was conducted separately, while some of the stochastic elements identified will be shared between the two imaging modality “constellations”, is possible that the “constellation” of stochastic morphological elements in the first imaging modality will include some elements which were not found in the second imaging modality, and some of the stochastic morphological elements found in the “constellation” in the second imaging modality will not be identified in the first imaging modality. This is not expected to hinder the alignment between the imaging modality sets / identification of the ROI location 102, since as indicated in the disclosed methods only a minimal number of stochastic elements ensuring accuracy of the SOI registration can37005PCbe used. In addition to these stochastic elements, in an exemplary implementation, the ML / AI / algorithms may also utilize 36 (see e.g., FIG. 6) the contour 114, or parts of it, to assist the alignment based on the stochastic elements.

[0055] In an exemplary implementation, some of the evaluation metrics which can be used to quantify the accuracy of some of these algorithms which output the calculated location of the ROI, both in relation to themselves and in relation to other fusion technologies, can include the following; The “distance between the center of the baseline ROI location and the center of the calculated ROI location” that is below a predetermined threshold, for example, no more than ±3.0mm or a dynamic threshold, for example, a percentage of the size of the ROI or SOI. The assumed baseline location of the ROI will be the location stated by a uniquely expert medical professional who performed cognitive fusion, in lieu of the ability to surgically find the location of the ROI. Then, any competing algorithms or technologies, will state their calculated location of the ROI. The distance between the center of the baseline ROI location and the center of the calculated ROI location will be the score of this evaluation metric; as an absolute distance, the distance on each of the three planes, the distance in each of the three axes, and each of these distances in relation to the size or volume of the ROI and the size or volume of the SOI. In this evaluation metric there could be several methods by which to calculate the center of a three-dimensional ROI (especially when it can be a concave three-dimensional ROI). These aforementioned metrics can be relevant to algorithms which output a location on a grid in which to conduct the operation, together with a Boolean success / fail metric - whether the calculated location on the grid is inside the volume of the ROI or not. Another evaluation metric for algorithms which output the calculated location of the ROI could be the “calculated ROI volume percentage overlap”, where the baseline three-dimensional location of the ROI will be once again decided by a uniquely expert medical professional who performed cognitive fusion, and the competing algorithms will calculate the three-dimensional location of the ROI. The percentage of the volume of the calculated ROI which overlaps with the baseline three-dimensional location of the ROI is the score of this metric. Another similar evaluation metric could be the “baseline ROI volume percentage overlap” where the process is similar but the percentage of the volume of the baseline ROI which overlaps with the calculated three-dimensional location of the ROI is the score of this metric. For algorithms which output sets of images from different modalities which are correctly aligned the evaluation metric can be a Boolean success / fail metric in relation to the baseline alignment set forth by a uniquely expert medical professional, or a percentage of the success rate over a testing pool. Another evaluation metric for these algorithms can be the distance between the baseline alignment and the alignment calculated by the37005PCalgorithm, wherein an alignment identical to the baseline will be a 0 mm or 0 “slices” difference. Other standard evaluation metrics which can be relevant are: Fl-Score, Boundary Fl-Score, Intersection over Union (loU), Dice Similarity Coefficient (DSC), Pixel Accuracy, Peak Signal to Noise Ratio (PSNR), Root Mean Square Error (RMSE), Structural Similarity (SSIM), Mutual Information (MI), Universal Quality Index (UQI), Correlation Coefficient (CC), Standard Deviation (SD), Entropy (EN), Spatial Frequency (SF), Gradient-Based Index (QAB / F) etc. It is noted, that although referring to a “plurality of patient-specific stochastic elements”, a single stochastic element within the ROI that would result in an alignment accuracy (or a predetermined target registration error (IRE), e.g., of less than 2mm) according to the predetermined parameter is also contemplated. Additionally, different accuracy requirements may apply depending if the images organ / tissue is rigid like bone or the brain (before significant brain shift), or soft, deformable organs such as the prostate, liver, or kidney, where there can be ongoing deformation and motion (e.g., respiration, bladder / rectal filling, probe pressure, and the like).

[0056] Since no fusion process is mathematically perfect, it may be necessary to account for the discrepancy between where a specific anatomical point appears on our fused display versus its actual physical location within the patient. This discrepancy is referred herein as the Target Registration Error, or TRE. Specifically, TRE is defined as the distance between the transformed position of a region of interest — the "target," such as a tumor or a specific vessel — and its true position as determined by a gold-standard reference (e.g., “ground truth). While Fiducial Registration Error (FRE) may sometimes be used to measure how well the identified stochastic elements identified in the various image modalities line up during the initial setup, TRE can be the better metric for clinical success because it measures the accuracy at the actual site of the intervention. It is calculated by taking the Euclidean distance between the target coordinates in the intraoperative image after registration and the corresponding coordinates in the preoperative image. Because errors in rotation and translation can amplify a function of the distance from the registration markers (the corresponding stochastic element(s), the TRE can vary significantly throughout the surgical field, making it the measure of whether the virtual map in the fused image is truly guiding the needle or scalpel to the correct location. Accordingly, the predetermined TRE can be, for example, a function of the ROI size, the distance between the fused stochastic element(s) and the ROI, and can be, for example between about 0.5 mm and about 10 mm, or between about 1 mm, and about 5mm, for example, between 1 mm and 3 mm, or less or equal to about 2mm in certain implementations.37005PC

[0057] In an exemplary implementation, a method to reduce the work required by the uniquely expert medical professional to just align / register one image from one modality to one image of the other modality, will use an additional algorithm. For example, the medical expert will identify an image captured by the first imaging modality that can be aligned to an image from the other imaging modality and align them. The algorithm will then, according to the “thickness” of each slice of each modality and the distance from the aligned set of images (information available as meta-data for each image), find the corresponding image for each imaging modality in the other imaging modality and register them accordingly.

[0058] In another exemplary implementation, the data (image sets from both modalities or from only one modality) provided by the uniquely expert medical professional to the neural network would include labels / annotations on the aforementioned stochastic elements (which are identifiable in both modalities), which the neural network learns to differentiate and recognize / identify and then further align the images. The labels / annotations could include location of the element, the type, size, shape etc. Additional information which could be supplied with each image could include the medical information regarding the patient (age, sex, ethnicity, etc.). Data augmentation could be used for training any of the machine learning algorithms.

[0059] Many medical images obtained by the imaging modalities disclosed, include areas outside of the relevant SOI, which are not necessarily required for the registration of the images and could include other organs / parts of proximal tissues in the frame of the image. The images provided to the system for the learning dataset, include in an exemplary implementation, a border (directed by the uniquely expert medical professionals and practitioners, which will be used to train an algorithm (which will be trained per organ) to crop the image around the “relevant” area / organ / tissue (see e.g., FIG.s 4A-4C). This “cropped” image (containing only the “relevant” organ / tissue / area) is supplied to the neural network which identifies the stochastic elements in the bound SOI. In another exemplary implementation, the uncropped image is provided together with the cropped image. Over time, the neural network decides whether to use only the cropped image for alignment, or whether to use the uncropped image and the additional information available there too.

[0060] In yet another exemplary implementation of the system and methods disclosed, the neural network receives the images already cropped by a medical professional (this will require the images provided during each operation to be cropped according to the same guidelines).

[0061] Some operating medical professionals of such medical procedures currently use other systems to conduct these medical procedures (whether only as an interface to view the medical images,37005PCto conduct the fusion of the different medical images according to other methods already in use, or even to guide operating medical personnel where to conduct the surgeries). In a yet another exemplary implementation, the neural network provides existing systems used to conduct the operation, with the more accurately aligned output image. This will require an interface between the existing systems and the registration module.

[0062] Some of these other systems employ primarily rigid, affine (transform), or deformable registration algorithms for MRI-ultrasound fusion. The URONAV™ system (Philips / Invivo) utilizes electromagnetic tracking with rigid and elastic registration, with reported fusion (registration) errors of 2.4-5.3mm. Conversely, KOELIS UROSTATION™ employs elastic registration with image-based tracking, reportedly achieving 2.8-4.1mm errors. Yet another system, BIOJET™ (D&K Engineering) uses mechanical tracking with rigid registration, claiming 2.3 -3.6 mm errors. Other systems, such as BK Medical's bkFusion™ implement electromagnetic tracking with deformable registration, reporting 2.5-4.7 mm errors, with ARTEMIS (Eigen), which combines mechanical tracking arm technology with elastic registration algorithms, demonstrated 1.9-3.8mm errors. Likewise, Hitachi's Hi-RVS™ utilizes magnetic position sensors with rigid and non-rigid registration, showing errors of 3.1 -5.0mm.Specifically for MRI-TRUS fusion, real-time systems like Toshiba's Smart Fusion employs optical tracking with deformable registration techniques, reporting 2.6-4.2 mm errors. Similarly, Esaote's VIRTUAL NAVIGATOR™ implements electromagnetic tracking with both rigid and non-rigid algorithms, showing 2.9-4.8 mm errors.

[0063] Accordingly, in an exemplary implementation illustrated schematically in FIG. 6, provided herein is a system for fusing a plurality of images of a predetermined scene of interest (SOI), wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, the system comprising: a preoperative imaging modality 10; an intraoperative imaging modality 12; a registration module 20; and an operating room user interface 40.

[0064] According to another exemplary implementation, the neural network supplies the final location of the ROI, or in yet another example, just coordinates of where to conduct the operation (e.g. coordinates of locations inside the ROI where it is preferable to conduct the operation, the location on the grid, etc.). In another exemplary implementation the system could be completely stand-alone, where all the imaging modalities are supplied (whether preoperative or intraoperative) to the standalone system directly, and the standalone system includes an interface for the operating medical personnel to view (some or all of the foregoing): the aligned image of the different modalities, the location of the37005PCROI on the intraoperative imaging modality, the coordinate of where to conduct the operation, and / or locations on the grid to conduct the operation.

[0065] In certain circumstances, actual processing of the images provided for each medical operation are conducted, by a centralized neural network (e.g. operating in the cloud 50 (see e.g., FIG.6)). According to this exemplary implementation , each operating room / each existing system used to conduct the operation will be connected to this cloud-based neural network which will receive the medical images in real-time, process them (the retrieval and processing of the images can be, either in real time - for example, the real-time images used to guide the location of the procedure must be retrieved in real-time, or could be prior to the operation - for example, the initial images conducted before the operation to locate the ROI could be retrieved and partially processed well before the operation), and send out the relevant output (some or all of the following: the aligned image of the different modalities, the location of the ROI on the real-time imaging modality, the coordinate of where to conduct the operation, and / or locations on the grid to conduct the operation). This could allow for a significantly increased amount of computing power and for the central (e.g. cloud- based) system to be continually supplied with additional data images and which will be capable of being stored in an anonymized manner - providing an ever-growing database of image sets and aligned images.

[0066] In yet another exemplary implementation, is a locally based system, whereby the trained neural network, along with any necessary algorithms and interface software is installed locally still storing the received images locally, and only possibly exporting the images (in an anonymized manner), to a central (e.g. cloud-based) location, for example only during update periods of the system. In yet another exemplary implementation, a locally based system could supplement a mainly centralized (e.g. cloud-based) system. This will among other benefits, allow for the increased computing power of the cloud-based system (when there is a steady internet-connection) together with the connectivity reliability of a locally-based system (a locally-based system isn’t as reliant on a constant internet connection as the centralized system)

[0067] It is noted that while software solutions for the registration of two different images do exist (and are sometimes used in the medical field, and might even include neural networks) there is a significant difference between these approaches and the approaches and methods described herein.

[0068] The goal of the networks / algorithms provided herein is not (as it is in most of those cases) to enhance one modality image using a different modality image, but rather to provide a solution for a specific use - locating an ROI 118c (see e.g., FIG. 5) discernable only in one modality image (usually preoperatively) in a different modality image where the ROI is non-discernable usually37005PCintraoperatively. These two different scenarios (merging several images for an enhanced picture and using one modality image to locate a region in the other modality image) while seemingly using similar tools - the inputs, outputs and the processing models (the algorithms and neural networks) are inherently different.

[0069] It is important to note that while software solutions for fusing of two different images do exist (and are sometimes used in the medical field, and might even include neural networks) there is a significant difference between these approaches and the approach and method described here. The goal of the networks / algorithms disclosed is not (as it is in most of those cases) to enhance an image from one modality, using a different modality image or generate a new fused image using different images from different modalities, but rather supplying a solution for a specific use - locating the ROI, that is discernable only in one modality, in another imaging modality in which the ROI is not discernable. The methods and systems disclosed do not require locating and registering the maximal number of stochastic elements but rather the minimal number of stochastic elements ensuring accuracy of the SOI registration. Nor are they aimed to generate a new holistic image (with attributes of each of the input modalities), but can in an exemplary implementation output only a location on a grid in which to conduct the medical procedure. It is further important to note, that the systems and methods disclosed do not seek to improve the resolution of the registered (fused) images. In other words, using the systems and methods disclosed will not, nor does these systems and methods rely on smaller pixels or voxels, but rather to register an ROI within an SOI, where the SOI is detectable on more than one image modality (pre-operational, and intra-operational while the ROI is not detectable in all these modalities. The methods and systems disclosed can be effective in achieving an error that is lower than 3 mm between all imaging modalities typically used.

[0070] Turning now to FIG.s 3, 6, illustrating certain exemplary implementations of the systems used to implement the methods disclosed. Currently, in operating rooms (whether having some form of Registration / Fusion module 20 (see e.g., FIG. 3B, FIG. 6) or not (see e.g., FIG. 3A)) there is an existing communication between the intraoperative imaging modality 2 (US imaging apparatus) and the operating room module. In exemplary implementation, where a registration module as disclosed herein is added to an existing Fusion technology setup, a local software module will be situated in between, on the same existing connection (see e.g., FIG. 3C, 3D), of FIG.s 3C, 3D with the light-gray "Fusion system" located between local software module and the operating room module.

[0071] In an exemplary implementation, the term “module” refers to any currently known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of37005PChardware and software that is capable of performing the functionality associated with that element. In another exemplary implementation, the term module should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner and / or to perform certain operations described herein. Considering exemplary implementations in which modules or components are temporarily configured (e.g., programmed), each of the modules or components need not be configured or instantiated at any one instance in time. For example, where the modules or components comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different modules at different times. Software may accordingly configure the processor to constitute a particular module at one instance of time and to constitute a different module at a different instance of time. In exemplary implementations where there is no pre-existing Fusion technology then the light-gray "Fusion system" is eliminated from FIG.s 3C, 3D.

[0072] FIG. 3C and FIG. 6 illustrates exemplary implementations of a fully functional, locally installed / situated system as disclosed and claimed within the operation room. All system operations are performed locally on the system itself. This local module combination may require periodic updates, and may save the images it receives / generates and during these updates is transferred from a locally saved position to a cloud position (in an anonymous manner). Furthermore, this data is retrieved in certain implementations by a backend management server (BMS) 50 (see e.g., FIG. 6), in communication with a learning dataset database 30 (see e.g., FIG. 6).

[0073] Turning now to FIG. 3D, illustrating a locally installed interface (e.g., a transceiver in communication with a communication network used as a remote connector), using a stable internet connection sends the intraoperative images to the BMS through a wide area network (the cloud). The calculations in this case will be conducted on the BMS and retransmitted back to the local software module, and rendered on a display device included with the operating room, where the local software module can render 104 the SOI and the ROI on a grid 42 (see e.g., FIG. 6), providing the physician with the option of deciding whether to accept the calculated location of the ROI rendered on the fused images 118a . In an exemplary implementation, preoperative images can be uploaded to the cloud, or the local software module ahead of obtaining the intraoperative images to reduce processing time, or in another example, be uploaded during the operation.

[0074] The system disclosed in FIG.s 3C-3E sends in certain exemplary implementations, instructions to the operating surgeon (e.g., to capture additional intraoperative images), and will also37005PCrender the output (e.g. location of the ROI) either on a display included with the local software module and user interface, or on the already existing user interface used in the operating room module.

[0075] In yet another exemplary implementation illustrated in FIG. 3E, the local software module is integrated into the registration / fusion module. This implementation sends the supplied / generated images for analysis and / or storage to the cloud-based algorithm / Data-Bases via communication with the BMS and can receive the calculation output via this connection, or conduct the calculation and store the supplied / generated images and calculation outputs locally while during any necessary updates transfer them from a locally saved position to a cloud position (in an anonymous manner).

[0076] The calculation and processing of the images themselves, the generation of the output, and the saving of the data (if there is anonymized image & data saving) will take place in yet another exemplary implementation - in the BMS, whether using local area network (LAN) or wide area network (cloud computing). In certain implementations, the "client" side uploads the preoperative images and meta-data (modality 1 e.g. MRI), which is transmitted to the BMS, providing the connection between the intraoperative imaging system (modality 2 e.g. US), and receives the output of the results produced by the local software module operable to carry out the methods disclosed herein.

[0077] In another exemplary implementation of the system used to implement the methods disclosed, more than two imaging modalities are used. If the ROI is discernable in one (preoperative) imaging modality, but no corresponding stochastic elements are discernable in a first intraoperative imaging modality, a second intraoperative or preoperative imaging modality is used to bridge the gap. The imaging modality used to locate the ROI might contain stochastic elements which are discernable in it and in another imaging modality (this second imaging modality may also not be applicable intraoperatively). The location of the ROI may be located in this additional imaging modality (the bridging modality) using the systems and methods disclosed herein. Consequently, the systems and methods disclosed herein are used again, using the stochastic elements discernable in both the bridging modality and in the final intraoperative imaging modality. Accordingly, the system is configured to locate the ROI in the intraoperative imaging modality, based on the calculated / inferred location of the ROI in the bridging modality and the shared stochastic elements of the two modalities.

[0078] The terms “first,” “second,” and the like, herein do not denote any order, quantity, or importance, but rather are used to denote one element from another. The terms “a”, “an” and “the” herein do not denote a limitation of quantity, and are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The suffix “(s)” as used37005PCherein is intended to include both the singular and the plural of the term that it modifies, thereby including one or more of that term (e.g., the user(s) includes one or more user). Reference throughout the specification to “one exemplary implementation”, “another exemplary implementation”, “an exemplary implementation”, and so forth, means that a particular element (e.g., feature, structure, and / or characteristic) described in connection with the exemplary implementation is included in at least one exemplary implementation described herein, and may or may not be present in other exemplary implementations. In addition, it is to be understood that the described elements may be combined in any suitable manner in the various exemplary implementations.

[0079] The term "plurality", as used herein, is defined as two or as more than two. The term "another", as used herein, is defined as at least a second or more. The terms "including" and / or "having", as used herein, are defined as comprising (i.e., open language).

[0080] The term “communication” and its derivatives (e.g., “in communication”) may refer to a shared bus configured to allow communication between two or more devices, or to a point to point communication link configured to allow communication between only two (device) points.

[0081] Although exemplary implementations of the invention are not limited in this regard, discussions utilizing terms such as, for example, “processing,” “computing,” “calculating,” “determining,” “establishing”, “analyzing”, “checking”, or the like, may refer to operation(s) and / or process(es) of a computer, a computing platform, a computing system, or other electronic computing device, that manipulates and / or transforms data represented as physical (e.g., electronic) quantities within the computer’s registers and / or memories into other data similarly represented as physical quantities within the computer’s registers and / or memories or other information non-transitory storage medium 24 (see e.g., FIG. 6) that may store instructions to perform operations and / or processes.

[0082] Furthermore, in the context of the disclosure, the term ‘operable’ means the system and / or the device and / or the program, or a certain element or step is fully functional, sized, adapted and calibrated, comprises elements for, and meets applicable operability requirements to perform a recited function when activated, coupled, implemented, actuated, effected, realized, or when an executable program 26 (see e.g., FIG. 6) is executed by at least one processor 22 (see e.g., FIG. 6) associated with the system and / or the device. In relation to systems and circuits, the term "operable" means the system and / or the circuit is fully functional and calibrated, comprises logic for, having the hardware and firmware necessary, as well as the circuitry for, and meets applicable operability requirements to perform a recited function when executed by at least one processor.37005PC

[0083] Additionally, “Operating Room user interface module” broadly refers to any visual, graphical, tactile, audible, sensory, or other means of providing information to and / or receiving information from a user or other entity. For example, a set of instructions which enable presenting a graphical user interface (GUI) on a display module to a user for displaying and changing and or inputting data associated with a data object in data fields. In an embodiment, the user interface module is capable of displaying any data that it reads from the imaging module 108. In addition, the term ‘module’, as used herein, means, but is not limited to, a software or hardware component, such as a Field Programmable Gate- Array (FPGA) or Application- Specific Integrated Circuit (ASIC), which performs certain tasks. A module may advantageously be configured to reside on an addressable storage medium and configured to execute on one or more processors. Thus, a module may include, by way of example, components, such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables. The functionality provided for in the components and modules may be combined into fewer components and modules or further separated into additional components and modules.

[0084] Unless specifically stated otherwise, as apparent from the description, it is appreciated that throughout the specification discussions utilizing terms such as “using”, “processing,” “loading,” “in communication,” “detecting,” “calculating,” “determining”, “analyzing,” “presenting”, “retrieving” or the like as generally used herein, refer to the action and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical, such as the radiation dosage adsorbed by the target material into other data similarly represented as series of numerical values, such as the fused stochastic elements’ coordinates.

[0085] As indicated, provided herein is a computer program, comprising program code means for carrying out the steps of the methods described herein, implementable in the systems provided, as well as a computer program product (e.g., a micro-controller) comprising program code means stored on a medium that can be read by a computer, such as a hard disk, USB, SSD, memory stick, or a storage medium that can be accessed via a data network, such as the Internet or Intranet, when the computer program product is loaded in the main memory of a computer [or micro-controller] and is carried out by the computer [or micro controller],

[0086] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be,37005PCfor example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc readonly memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0087] Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, python, Java, C#, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the system’s computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.

[0088] In addition, the memory medium may be located in a first computer in which the programs are executed, and / or may be located in a second different computer [or micro controller]37005PCwhich connects to the first computer over a network (or an integrated second computer within the system network), such as the Internet [or, they might be even not connected and information will be transferred using USB], In the latter instance, the second computer may further provide program instructions to the first computer for execution.

[0089] The term “coupled”, including its various forms such as “operably coupling”, "coupling" or "couplable", refers to and comprises any direct or indirect, structural coupling, connection or attachment, or adaptation or capability for such a direct or indirect structural or operational coupling, connection or attachment, including integrally formed components and components which are coupled via or through another component or by the forming process. Indirect coupling may involve coupling through an intermediary member or adhesive, or abutting and otherwise resting against, whether frictionally or by separate means without any physical connection. Likewise, “operably coupled” refers to the joining of two members directly or indirectly to one another. Such joining may be stationary in nature or moveable in nature. Such joining may be achieved with the two members (or the two members and any additional intermediate) being integrally formed as a single unitary body with one another or with the two members or the two members and any additional members being attached to one another. Such joining may be permanent in nature or may be removable or releasable in nature.

[0090] Furthermore, "communicate" (and its derivatives e.g., a first component "communicates with" or "is in communication with" a second component) and grammatical variations thereof are used to indicate a structural, functional, mechanical, electrical, optical, or fluidic relationship, or any combination thereof, between two or more components or elements. As such, the fact that one component is said to communicate with a second component is not intended to exclude the possibility that additional components can be present between, and / or operatively associated or engaged with, the first and second components

[0091] The term "comprising" and its derivatives, as used herein, are intended to be open-ended terms that specify the presence of the stated features, elements, components, groups, integers, and / or steps, but do not exclude the presence of other unstated features, elements, components, groups, integers and / or steps. The foregoing also applies to words having similar meanings such as the terms, "including", "having" and their derivatives.

[0092] All ranges disclosed herein are inclusive of the endpoints, and the endpoints are independently combinable with each other. “Combination” is inclusive of blends, mixtures, alloys, reaction products, and the like. The terms “a”, “an” and “the” herein do not denote a limitation of37005PCquantity, and are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The suffix “(s)” as used herein is intended to include both the singular and the plural of the term that it modifies, thereby including one or more of that term (e.g., the element(s) includes one or more element). Reference throughout the specification to “one exemplary implementation”, “another exemplary implementation”, “an exemplary implementation”, and so forth, when present, means that a particular element (e.g., feature, structure, and / or characteristic) described in connection with the exemplary implementation is included in at least one exemplary implementation described herein, and may or may not be present in other exemplary implementations. In addition, it is to be understood that the described elements may be combined in any suitable manner in the various exemplary implementations.

[0093] Accordingly, and in an exemplary implementation, provided herein is a method of registering a region of interest (ROI) within a scene of interest (SOI), the SOI being detectable in a plurality of images, wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, wherein the ROI is non-discernable in the different imaging modality, the method comprising: for each image, identifying a plurality of a patient-specific stochastic elements within the SOI 106 (see e.g., FIG. 5), extracting the identified patient-specific stochastic elements from each image of the plurality of images 108 (see e.g., FIG. 5), thereby forming patient-specific SOI fingerprint, and registering the patient specific SOI fingerprint across the plurality of images 116 (see e.g., FIG. 5), the method further comprising (i) forming at least one fused image 118a (see e.g., FIG. 5), and / or registering the at least one image from one modality with the image from another modality 118b (see e.g., FIG. 5), and / or calculating the location of the ROI in the different imaging modality where the ROI is non-discernable 118c (see e.g., FIG. 5), wherein (ii) at least one image of the plurality of images is a preoperative image 100 (see e.g., FIG. 5) and at least another image of the plurality of images is an intraoperative image 12 (see e.g., FIG. 6), (iii) the step of identifying a plurality of a patient-specific stochastic elements within the SOI 106 (see e.g., FIG. 5), is preceded by a step of identifying a region of interest (ROI) on the preoperative imaging modality 102 (see e.g., FIG. 5), and rendering the ROI on the at least one image captured by the preoperative imaging modality 104 (see e.g., FIG. 5), while (iv) the step of forming at least one fused image, is followed by a step of rendering the ROI on the at least one fused image 120 (see e.g., FIG. 5), (v) the step of aligning the images from different modalities 118b (see e.g., FIG. 5), is followed by a step of rendering the ROI on at least on image of the intraoperative imaging modality 120 (see e.g., FIG. 5), wherein (vi) the patient-specific stochastic elements comprise: inflammatory37005PCchanges, inflammatory infiltrates, micro calcifications, macro calcifications, fibroses, scarring, simple cysts, complex cysts, neovascularization, vascular malformations, stochastic blood vessel junctions, tissue metaplasia, tissue dysplasia, surgical clips, stents, implants, necrotic areas, hemorrhagic foci, granulomas, radiation induced tissue change, ablation induced morphological changes, fatty infiltration, fatty degeneration, edema, abscesses, tissue mineralization, lymphoid aggregates, focal atrophy, post-traumatic changes, adhesions, microlithiasis, focal hypertrophy, amyloid tissue deposits, iron deposits, focal fibrosis, prostate cysts, prostate calcifications, prostate arteries and veins, post operative defects, post operative scarring, irreversible electroporation, marker seeds, Brachytherapy seeds, indwelling catheter, gel spacer, stents, Uro-lift™ tackers, cancerous lesion, post radiation changes, or patientspecific stochastic elements comprising one or more of the foregoing, wherein (vii) the image modalities used comprise: a digital camera, magnetic resonance imaging (MRI), ultrasound (US), X-ray, positron emission tomography (PET), PET-CT, single-photon emission computerized tomography (SPECT), Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy, Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy or an imaging technology combination comprising one or more of the foregoing, (viii) the steps of identifying, extracting, and registering the patient-specific stochastic elements, and rendering the ROI on the at least one fused image or on at least one image of the intraoperative imaging modality, is carried out using machine learning (ML) and / or artificial intelligence (Al) 30 (see e.g., FIG. 6), (ix) wherein the surgical operation is transcatheter aortic valve replacement (TAVR), the intraoperative imaging modality is fluoroscopy, and the preoperative image modality is CT Angiography, (x) the surgical operation is a hepatobiliary surgery, the intraoperative imaging modality is ultrasound, and the preoperative imaging modality is contrast-enhanced CT, and / or (xi) wherein the surgical operation is a pedicle screw placement, the intraoperative imaging modality is fluoroscopy, and the preoperative imaging modality is CT, and / or (xi) wherein the surgical operation is a prostate operation, the intraoperative imaging modality is ultrasound or trans rectal US (TRUS), and the preoperative imaging modality is MRI, or PET-CT, wherein (xii) the step of registering the patient-specific SOI fingerprint37005PCacross the plurality of images 116 (see e.g., FIG. 5) further comprises: independently applying 110 (see e.g., FIG. 5), for each imaging modality, a machine learning model or algorithm configured to identify patient-specific stochastic elements within the SOI in that imaging modality separately from any other imaging modality, thereby generating, for each imaging modality, a respective modality-specific constellation of identified stochastic elements, applying 112 (see e.g., FIG. 5) an alignment machine learning model or algorithm, aligning the modality-specific constellation generated from the first imaging modality with the modality-specific constellation generated from the second imaging modality, wherein the alignment is optimized across at least a minimum subset of stochastic elements shared between the two constellations sufficient to achieve SOI registration accuracy having a predetermined target registration error (TRE), (xiii) the step of aligning 112 the modality-specific constellation generated from the first imaging modality with the modality-specific constellation generated from the second imaging modality, further comprises utilizing 114 a contour of the SOI, or a portion thereof, as an additional spatial constraint.

[0094] In another exemplary implementation, provided herein is a system for registering a region of interest (ROI) within a scene of interest (SOI), wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, the system comprising: a preoperative imaging modality 10 (see e.g., FIG. 6), an intraoperative imaging modality 12 (see e.g., FIG. 6), a registration module 20 (see e.g., FIG. 6), and an operating room user interface 40 (see e.g., FIG. 6), (xiv) wherein the registration module comprises at least one processor 22 (see e.g., FIG. 6) in communication with a non-transitory memory device 24 (see e.g., FIG. 6), storing thereon a computer- readable medium with a set of executable instructions 26 (see e.g., FIG. 6), configured when executed by the at least one processor 24 to: for each image, identify 26(i) a plurality of a patient-specific stochastic elements within the SOI, extract 26(ii) the identified patientspecific stochastic elements from each image of the plurality of images, thereby forming patientspecific SOI fingerprint, and register 26(iii) the patient specific SOI fingerprint across the plurality of images, (xv) the preoperative imaging modality 10 is: a digital camera, Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Imaging, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy, X-ray, positron emission tomography (PET), PET-CT, single- photon emission computerized tomography (SPECT), or a combination of imaging modality comprising one or more of the foregoing, wherein37005PC(xvi) the intraoperative imaging modality 12 comprise: a digital camera, Fluoroscopy, ultrasound, Realtime 3D Ultrasound (4D US), Contrast-Enhanced Ultrasound (CEUS), Intraoperative CT, Intraoperative MRI, Near-Infrared Fluorescence Imaging, Surgical Navigation, Electromagnetic Navigation Systems, Augmented Reality Systems, Optical Surface Scanning, Intraoperative OCT, Photoacoustic Imaging, US-based Elastography, MR-based Elastography, Laser Speckle Contrast Imaging, or a combination of imaging modality comprising one or more of the foregoing, wherein (xvii) the operating room user interface 40 (see e.g., FIG. 6) comprises a display and an input module 44 (see e.g., FIG. 6), wherein (xviii) the patient-specific stochastic elements comprise: inflammatory changes, inflammatory infiltrates, micro calcification(s), macro calcification(s), fibroses, stochastic blood vessel junction(s), scarring, simple cyst(s), complex cyst(s), neovascularization, vascular malformation(s), tissue metaplasia, tissue dysplasia, surgical clip(s), stent(s), implant(s), necrotic area(s), hemorrhagic foci, granuloma(s), radiation induced tissue change(s), ablation induced morphological change(s), fatty infiltration(s), fatty degeneration(s), edema, abscess(es), tissue mineralization, lymphoid aggregate(s), focal atrophy, post-traumatic change(s), adhesion(s), microlithiasis, focal hypertrophy, amyloid tissue deposit(s), iron deposit(s), focal fibrosis, prostate cysts, prostate calcifications, post operative defects ,post operative scarring, artificial elements, marker seeds, Brachytherapy seeds , indwelling catheter, gel spacer, stents, Uro-lift™ tackers, cancerous lesion, post radiation changes, or patient-specific stochastic elements comprising one or more of the foregoing, wherein (xix) the set of executable instruction 26 is further configured, when executed by the at least one processor 24, to: generate a grid on the display, wherein the display is configured to encompass the SOI, and display at least one registered image on the grid 26(iv) or at least one set of aligned images of which, at least one image is from one modality and another image is from another modality, with the grid overlaying each image, with the possibility of locating on the grid the location of the ROI, (xx) the set of executable instruction is further configured, when executed, to use machine learning and / or artificial intelligence 30 to: identify the plurality of a patient-specific stochastic elements, extract the identified patient-specific stochastic elements, register the patient specific SOI fingerprint across the plurality of images, (xxi) the set of executable instruction is further configured, when executed, to use machine learning to: locate a predetermined region of interest (ROI) discernable only on the preoperative imaging modality, and render the ROI on a display included with the operating room user interface, wherein (xxii) to register the patient specific SOI fingerprint across the plurality of images, (xxiii) the set of executable instruction 26 (see e.g., FIG. 6) is further configured, when executed by the at least one processor, to: independently apply, for each imaging modality, a37005PCmachine learning model or algorithm configured to identify patient-specific stochastic elements within the SOI in that imaging modality separately from any other imaging modality, thereby generating, for each imaging modality, a respective modality-specific constellation of identified stochastic elements 32, and applying an alignment machine learning model or algorithm, align the modality-specific constellation generated from the first imaging modality with the modality-specific constellation generated from the second imaging modality 34, wherein the alignment is optimized across at least a minimum subset of stochastic elements shared between the two constellations sufficient to achieve SOI registration accuracy having a predetermined target registration error (TRE), and wherein (xxiv) to align the modality-specific constellation generated from the first imaging modality with the modalityspecific constellation generated from the second imaging modality, the set of executable instruction is further configured, when executed by the at least one processor, to: use a contour of the SOI, or a portion thereof, as an additional spatial constraint 36.

[0095] While particular exemplary implementations have been described, alternatives, modifications, variations, improvements, and substantial equivalents that are or may be presently unforeseen may arise to applicants or others skilled in the art. Accordingly, the appended claims as filed and as they may be amended, are intended to embrace all such alternatives, modifications variations, improvements, and substantial equivalents.

Claims

37005PCCLAIMS1. A method of registering a region of interest (ROI) within a scene of interest (SOI), the SOI being detectable in a plurality of images, wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, wherein the ROI is non-discemable in the different imaging modality, the method comprising:a) for each image, identifying a plurality of a patient-specific stochastic elements within the SOI;b) extracting the identified patient-specific stochastic elements from each image of the plurality of images, thereby forming patient-specific SOI fingerprint; andc) registering the patient specific SOI fingerprint across the plurality of images.

2. The method of claim 1, further comprising:a) forming at least one fused image; and / orb) registering the at least one image from one modality with the image from another modality; and / orc) calculating the location of the ROI in the different imaging modality where the ROI is non-discernable.

3. The method of claim 1, wherein at least one image of the plurality of images is a preoperative image and at least another image of the plurality of images is an intraoperative image.

4. The method of claim 3, wherein the step of identifying a plurality of a patientspecific stochastic elements within the SOI, is preceded by a step of identifying a region of interest (ROI) on the preoperative imaging modality, and rendering the ROI on the at least one image captured by the preoperative imaging modality.

5. The method of claim 4, wherein the step of forming at least one fused image, is followed by a step of rendering the ROI on the at least one fused image.

6. The method of claim 4, wherein the step of aligning the images from different modalities, is followed by a step of rendering the ROI on at least one image of the intraoperative imaging modality.

7. The method of claim 1, wherein the patient-specific stochastic elements comprise: inflammatory changes, inflammatory infiltrates, micro calcifications, macro calcifications, fibroses, scarring, simple cysts, complex cysts, neovascularization, vascular malformations, stochastic blood vessel junctions, tissue metaplasia, tissue dysplasia, surgical clips, stents,37005PCimplants, necrotic areas, hemorrhagic foci, granulomas, radiation induced tissue change, ablation induced morphological changes, fatty infiltration, fatty degeneration, edema, abscesses, tissue mineralization, lymphoid aggregates, focal atrophy, post-traumatic changes, adhesions, microlithiasis, focal hypertrophy, amyloid tissue deposits, iron deposits, focal fibrosis, prostate cysts, prostate calcifications, prostate arteries and veins, post operative defects, post operative scarring, irreversible electroporation, marker seeds, Brachytherapy seeds, indwelling catheter, gel spacer, stents, URO-LIFT™ tackers, cancerous lesion, post radiation changes, or patientspecific stochastic elements comprising one or more of the foregoing.

8. The method of claim 3, wherein the image modalities used comprise: a digital camera, magnetic resonance imaging (MRI), ultrasound (US), X-ray, positron emission tomography (PET), PET-CT, single-photon emission computerized tomography (SPECT), Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy, Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy or an imaging technology combination comprising one or more of the foregoing.

9. The method of claim 5 or claim 6, wherein the steps of identifying, extracting, and registering the patient-specific stochastic elements, and rendering the ROI on the at least one fused image or on at least one image of the intraoperative imaging modality, is carried out using machine learning (ML) and / or artificial intelligence (Al).

10. The method of claim 9, wherein the surgical operation is transcatheter aortic valve replacement (TAVR), the intraoperative imaging modality is fluoroscopy, and the preoperative image modality is CT Angiography.

11. The method of claim 9, wherein the surgical operation is a hepatobiliary surgery, the intraoperative imaging modality is ultrasound, and the preoperative imaging modality is contrast-enhanced CT.

12. The method of claim 9, wherein the surgical operation is a pedicle screw placement, the intraoperative imaging modality is fluoroscopy, and the preoperative imaging modality is CT.37005PC13. The method of claim 9, wherein the surgical operation is a prostate operation, the intraoperative imaging modality is ultrasound or trans rectal US (TRUS), and the preoperative imaging modality is MRI, or PET-CT.

14. The method of claim 9, wherein the step of registering the patient-specific SOI fingerprint across the plurality of images further comprises:a) independently applying, for each imaging modality, a machine learning model or algorithm configured to identify patient-specific stochastic elements within the SOI in that imaging modality separately from any other imaging modality, thereby generating, for each imaging modality, a respective modality-specific constellation of identified stochastic elements;b) applying an alignment machine learning model or algorithm, aligning the modality-specific constellation generated from the first imaging modality with the modalityspecific constellation generated from the second imaging modality, wherein the alignment is optimized across at least a minimum subset of stochastic elements shared between the two constellations sufficient to achieve SOI registration having a predetermined target registration error (TRE).

15. The method of claim 14, wherein the step of aligning the modality-specific constellation generated from the first imaging modality with the modality-specific constellation generated from the second imaging modality, further comprises utilizing a contour of the SOI, or a portion thereof, as an additional spatial constraint.

16. A system for registering a region of interest (ROI) within a scene of interest (SOI), wherein at least one image of the plurality of images is obtained by a different imaging modality than at least another image in the plurality of images, the system comprising:a) a preoperative imaging modality;b) an intraoperative imaging modality;c) a registration module; andd) an operating room user interface.

17. The system of claim 16, wherein the registration module comprises at least one processor in communication with a non-transitory memory device, storing thereon a computer-readable medium with a set of executable instructions, configured when executed by the at least one processor to:a) for each image, identify a plurality of a patient-specific stochastic elements within the SOI;b) extract the identified patient-specific stochastic elements from each image of the plurality of images, thereby forming patient-specific SOI fingerprint; and37005PCc) register the patient specific SOI fingerprint across the plurality of images.

18. The system of claim 16, wherein the preoperative imaging modality is: a digital camera, Computed Tomography (CT), Computed Tomography Angiography (CTA), CT Perfusion, Magnetic Resonance Imaging, Magnetic Resonance Angiography (MRA), MR Spectroscopy, MR Perfusion, Diffusion Tensor Imaging (DTI), Functional MRI (fMRI), Digital Subtraction Angiography (DSA), Dual-Energy CT (DECT), Cone Beam CT (CBCT), Optical Coherence Tomography (OCT), Nuclear Medicine Scintigraphy, X-ray, positron emission tomography (PET), PET-CT, single-photon emission computerized tomography (SPECT), or a combination of imaging modality comprising one or more of the foregoing.

19. The system of claim 16, wherein the intraoperative imaging modality comprise: a digital camera, Fluoroscopy, ultrasound, Real-time 3D Ultrasound (4D US), Contrast-Enhanced Ultrasound (CEUS), Intraoperative CT, Intraoperative MRI, Near-Infrared Fluorescence Imaging, Surgical Navigation, Electromagnetic Navigation Systems, Augmented Reality Systems, Optical Surface Scanning, Intraoperative OCT, Photoacoustic Imaging, US-based Elastography, MR-based Elastography, Laser Speckle Contrast Imaging, or a combination of imaging modality comprising one or more of the foregoing.

20. The system of claim 18, wherein the operating room user interface comprises a display and an input module.

21. The system of claim 18, wherein the patient-specific stochastic elements comprise: inflammatory changes, inflammatory infiltrates, micro calcification(s), macro calcification(s), fibroses, stochastic blood vessel junction(s), scarring, simple cyst(s), complex cyst(s), neovascularization, vascular malformation(s), tissue metaplasia, tissue dysplasia, surgical clip(s), stent(s), implant(s), necrotic area(s), hemorrhagic foci, granuloma(s), radiation induced tissue change(s), ablation induced morphological change(s), fatty infiltration(s), fatty degeneration(s), edema, abscess(es), tissue mineralization, lymphoid aggregate(s), focal atrophy, post-traumatic change(s), adhesion(s), microlithiasis, focal hypertrophy, amyloid tissue deposit(s), iron deposit(s), focal fibrosis, prostate cysts, prostate calcifications, post operative defects ,post operative scarring, artificial elements, marker seeds, Brachytherapy seeds , indwelling catheter, gel spacer, stents, URO-LIFT™ tackers, cancerous lesion, post radiation changes, or patientspecific stochastic elements comprising one or more of the foregoing.

22. The system of claim 21, wherein the set of executable instruction is further configured, when executed by the at least one processor, to:a) generate a grid on the display, wherein the display is configured to encompass the SOI; and37005PCb) display at least one registered image on the grid or at least one set of aligned images of which, at least one image is from one modality and another image is from another modality, with the grid overlaying each image, with the possibility of locating on the grid the location of the ROI.

23. The system of claim 18 wherein the set of executable instruction is further configured, when executed, to use machine learning and / or artificial intelligence to: identify the plurality of a patient-specific stochastic elements, extract the identified patient-specific stochastic elements, register the patient specific SOI fingerprint across the plurality of images.

24. The system of claim 23, wherein the set of executable instruction is further configured, when executed, to use machine learning to: locate a predetermined region of interest (ROI) discernable only on the preoperative imaging modality; and render the ROI on a display included with the operating room user interface.

25. The system of claim 23, wherein to register the patient specific SOI fingerprint across the plurality of images, the set of executable instruction is further configured, when executed by the at least one processor, to:a) independently apply, for each imaging modality, a machine learning model or algorithm configured to identify patient-specific stochastic elements within the SOI in that imaging modality separately from any other imaging modality, thereby generating, for each imaging modality, a respective modality-specific constellation of identified stochastic elements; andb) applying an alignment machine learning model or algorithm, align the modality-specific constellation generated from the first imaging modality with the modalityspecific constellation generated from the second imaging modality, wherein the alignment is optimized across at least a minimum subset of stochastic elements shared between the two constellations sufficient to achieve SOI registration having a predetermined TRE.

26. The system of claim 16, wherein to align the modality-specific constellation generated from the first imaging modality with the modality-specific constellation generated from the second imaging modality, the set of executable instruction is further configured, when executed by the at least one processor, to: use a contour of the SOI, or a portion thereof, as an additional spatial constraint.

Images