Methods and systems for high performance and versatile molecular imaging

Abstract

To provide a method and system for high performance and versatile molecular imaging.SOLUTION: Embodiments provide improved imaging devices and methods. A portable SPECT imaging device may co-align with imaging modalities such as an ultrasound and the like. Gamma camera panels including gamma camera sensors may be connected to a mechanical arm. A coded aperture mask may be installed in front of a gamma-ray photon sensor, and used to construct a high-resolution three-dimensional map of radioisotope distributions inside a patient, which can be generated by scanning the patient from a reduced range of directions around the patient, and by using radiation sensors installed in close proximity to this patient. Boosted imaging sensitivity and resolution are provided. The SPECT imaging device can be used to guide medical interventions such as biopsies, ablation therapies and the like, and can also be used to guide surgeries.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of and priority to U.S. Application No. 62 / 831,504, filed April 9, 2019, and entitled "Methods And Systems For High Performance Spect Imaging," and U.S. Application No. 62 / 836,514, filed April 19, 2019, and entitled "Methods And Systems For Portable Spect And Ultrasound Imaging," the contents of which are incorporated herein by reference in their entirety for all purposes.

[0002] The present invention relates to gamma camera architectures and their use along with other co-registered medical imaging modalities such as ultrasound systems to enable new high performance and versatile imaging systems for diagnostic imaging, guidance of medical interventions such as percutaneous biopsy and ablation therapy, as well as for surgical guidance. [Background technology]

[0003] Single photon emission computed tomography (SPECT) is the main molecular imaging modality used for medical imaging, either alone or in combination with computed tomography (CT) (SPECT / CT). Most commonly, a SPECT imaging device comprises an array of gamma-ray sensors that either surrounds the patient's body or orbits around the patient. During the image scan, the patient most commonly lies on a table, and for some cardiac imaging systems may be seated on a custom-made chair. A parallel-hole collimator is commonly used in front of the detector array to constrain the direction that the gamma-ray photons can take before interacting with the position-sensitive sensor. This produces a parallel projection of the distribution of gamma-ray-emitting isotopes inside the patient. A computer program is used to reconstruct this distribution in 3D by using analytical or iterative image reconstruction algorithms.

[0004] The embodiments provide improved methods and systems for SPECT imaging. Summary of the Invention [Means for solving the problem]

[0005] SUMMARY OF THE DISCLOSURE Embodiments relate to systems and methods for single photon emission computed tomography (SPECT) imaging.

[0006] Some embodiments provide a portable single photon emission computed tomography (SPECT) imaging system for scanning a patient. The system includes a SPECT controller unit, the controller unit including a computer. The system further includes a mechanical articulated arm connected to the controller unit. The articulated arm can be positioned at a desired location by a user through applying a direct force. The system further includes at least one gamma camera panel connected to the articulated arm. The gamma camera panel includes a gamma camera sensor with position and energy sensing resolution. The gamma camera panel can provide an image field of view greater than 15 degrees. The system further includes a camera mounted in such a manner as to observe the entire area of ​​the patient. The system further includes at least one processor and a memory operatively coupled to the at least one processor, the camera, and the gamma camera sensor. The memory has instructions for execution by the at least one processor that cause the at least one processor to read a first gamma ray photon sensing event received from the gamma camera sensor. The processor further provides a first position and orientation of the gamma camera panel relative to the patient's body. The processor further co-locates the first gamma ray photon sensing event to the patient's body using the first position and orientation. The processor further reads a second gamma ray photon sensing event received from the gamma sensor. The processor further provides a second position and orientation of the gamma camera panel relative to the patient's body. The processor further co-locates the second gamma ray photon sensing event to the patient's body using the second position and orientation. And the processor reconstructs a 3D distribution of the gamma ray emitting radioisotope inside the patient by using the first and second co-registered sensing events.

[0007] Some embodiments provide a real-time multi-modality portable single photon emission computed tomography (SPECT) imaging system for scanning a patient. The system includes a SPECT controller unit, the unit includes a computer. The system further includes a mechanical joint arm connected to the controller unit, the joint arm can be positioned at a desired location by a user through applying a direct force. The system further includes at least one gamma camera panel connected to the joint arm. The gamma camera panel includes a gamma camera sensor with position and energy sensing resolution. The system further includes an ultrasound probe that is positionable in a manner to have a field of view that at least partially overlaps with the gamma camera field of view. The system further includes a tracking system that is capable of providing information about a relative position of the ultrasound transducer with respect to the gamma camera. The system further includes a visualization device. The system further includes at least one processor and a memory operatively coupled to the gamma camera sensor, the ultrasound probe, the tracking system, and the visualization device. The memory has instructions for execution by the at least one processor to cause the at least one processor to read a first gamma ray photon sensing event received from the gamma sensor. The processor further executes the instructions to read a second gamma ray photon sensing event received from the gamma sensor. The processor further executes the instructions to reconstruct a 3D distribution of gamma ray emitting radioisotopes inside the patient by using the first and second sensing events. The processor further executes the instructions to determine co-registration between an ultrasound transducer and a gamma sensor using the tracking information. The processor further executes the instructions to determine co-registration between the ultrasound transducer and a gamma sensor using the co-registration between the ultrasound transducer and a gamma sensor. The processor further executes the instructions to deliver an image to a visualization device comprising an enhancement of the 3D distribution of gamma ray emitting radioisotopes onto the ultrasound scan by using the co-registration between the 3D distribution of gamma ray emitting radioisotopes and the ultrasound scan.

[0008] Some embodiments provide a portable single photon emission computed tomography (SPECT) imaging system for scanning a body part of a patient. The system includes a SPECT controller unit, the unit includes a computer. The system further includes a mechanical articulated arm connected to the controller unit. In some embodiments, the articulated arm can be mounted on other objects such as on floors, ceilings, walls, rails, and other fixed objects instead of being mounted on the controller unit. The system further includes at least a gamma camera panel connected to the articulated arm, the gamma camera panel includes a gamma camera sensor with position and energy sensing resolution. The gamma camera panel provides an image field of view that is greater than 15 degrees. The image field of view can be defined as a range of angles relative to a direction in which the gamma camera has maximum image sensitivity, from which gamma photons can be detected and imaged by a gamma sensor consisting of a gamma camera panel with sensitivity greater than 1 / 100 of the maximum image sensitivity. The system further includes a tactile sensor mounted on the panel. The tactile sensor is operably coupled to at least one processor and a memory. Movement of the panel relative to the patient is modified in response to tactile sensor data acquired by the tactile sensor.

[0009] In some embodiments, the portable SPECT system uses data from an external computed tomography (CT) or another medical imaging scanner (such as magnetic resonance imaging) to improve the delivered molecular image quality by applying attenuation correction. In some embodiments, the CT image can be co-registered with the molecular image and a rendering of the combination may be sent to a visualization device for user interpretation. In some embodiments, co-registration between the CT image and the SPECT image may be performed by matching 3D contours of the patient. Co-registered ultrasound images may be used to aid in co-registration. In some other embodiments, tags may be used for co-registration.

[0010] In some embodiments, the portable SPECT system may be co-registered with a medical optical imaging device, such as an endoscope, laparoscope, or with an X-ray device, such as a fluoroscope, to guide a medical intervention, such as a biopsy, ablation, or surgery.

[0011] In an exemplary embodiment, the system comprises a gamma ray photon sensor with energy and position resolution sensing capabilities. The gamma ray photon sensor can provide a location of the photon interaction. The system further comprises a coded aperture mask placed in front of the photon sensor. The mask can comprise photon attenuation mask pixel elements shaped as bi-pyramids, with the physical space between the bi-pyramid mask pixel elements having a common edge being partially or fully occupied by material. The mask can generate an image field in front of the sensor. The system further comprises at least one processor and a memory operatively coupled to the sensor and the processor. The memory can store instructions for execution by the at least one processor that cause the processor to project a location of a first photon interaction onto a reference surface and generate a first projected interaction point. The processor can also read out a photon attenuation coefficient stored in the memory for the first projected interaction point with respect to a direction toward the image field. The processor can also project a location of a second photon interaction onto the reference surface and generate a second projected interaction point. The processor may also retrieve the photon attenuation coefficient stored in the memory for a second projected interaction point for a direction toward the image field of view. The processor may also reconstruct an image of the gamma ray source using the retrieved attenuation coefficients for the first and second photon interactions.

[0012] In some embodiments, the sensor provides the location of the photon interaction with a resolution better than 4 millimeters (mm) in all three dimensions. In some embodiments, the coded aperture mask is made of a material with a density greater than 10 grams per cubic centimeter (g / cc). In some embodiments, the mask pixel elements are shaped as bi-pyramids with at least a side that makes an angle greater than 3 degrees with respect to a normal on the bi-pyramid base. In some embodiments, the mask pixel elements are shaped as bi-pyramids with at least a side that makes an angle greater than 5 degrees with respect to a normal on the bi-pyramid base. In some embodiments, the material between the bi-pyramid mask pixel elements is of a density greater than 10 g / cc. In some embodiments, the bi-pyramid mask pixel elements have a base selected from the group containing a rectangular base, a triangular base, a hexagonal base. In some embodiments, the shape of the bi-pyramid mask pixel elements is approximated by a mask pixel element with curved sides. In some embodiments, the coded aperture mask extends across multiple planes. In some embodiments, the system further comprises a photon attenuating block in a direction around the sensor that is not covered by the coded aperture mask. In some embodiments, the coded aperture mask has an open fraction, defined as the ratio of the area of ​​the unattenuated mask area to the total area of ​​the mask, ranging from 0.1% to 70%. In some embodiments, the coded aperture mask is freestanding. In some embodiments, the coded aperture mask is constructed from multiple layers that are stacked together to approximate a bi-frustum shaping of the mask pixels.

[0013] In another exemplary embodiment, the method includes projecting a location of a first photon interaction detected by a gamma ray photon sensor onto a first reference plane to generate a first projected interaction point. In this embodiment, the gamma ray photon sensor has energy and position resolution sensing capabilities. The gamma ray photon sensor provides the location of the photon interaction. In this embodiment, a coded aperture mask is placed in front of the photon sensor. The mask comprises photon attenuation mask pixel elements shaped as a bi-frustum. In the mask, the physical space between the bi-frustum mask pixel elements having a common edge is partially or fully occupied by material. The mask generates an image field in front of the sensor. The method further includes retrieving the photon attenuation coefficient stored in the memory for the first projected interaction point with respect to a direction toward the image field. The method further includes projecting a location of a second photon interaction detected by the gamma ray photon sensor onto a second reference plane to generate a second projected interaction point. The method further includes retrieving the stored photon attenuation coefficient in the memory for a second projected interaction point for a direction toward the image field of view. The method further includes reconstructing an image of the gamma ray source using the retrieved attenuation coefficients for the first and second photon interactions.

[0014] In some embodiments, the mask used in front of the sensor to code the incident gamma ray photons has an adjustable geometry. The adjustment can be triggered by a computer sending commands to the actuators. The adjustment allows the mask to provide both wide and narrow fields of view. This allows wide field scanning and narrow focusing on a structure of interest when needed. Furthermore, the adjustment can change the distance between the mask and the detector. Furthermore, the adjustment can change the open percentage of the mask. In some embodiments, the mask can be made of overlapping parallel plates with partially or completely overlapping openings. In some embodiments, the mask is made of three overlapping layers, although any number of layers can be envisioned. In some embodiments, the layers are moved away from each other to increase the focusing ability or collimation. In some implementations, the computer controls the arrangement of the mask elements depending on the image task or depending on user input.

[0015] In some embodiments, the portable SPECT system may be used in both a scanning mode to generate more extended SPECT image data sets, and in a real-time imaging mode to generate actionable images. Adjustable masks may be used to optimize these imaging modes by allowing both wide field of view images, which are particularly useful for scanning, and narrow fields of view, which are particularly useful for real-time images of specific structures.

[0016] A further understanding of the nature and advantages of the embodiments may be gained by reference to the following detailed description and the accompanying drawings. The present invention provides, for example, the following: (Item 1) 1. A portable single photon emission computed tomography (SPECT) imaging system for scanning a patient, the system comprising: a single photon emission computed tomography controller unit, the controller unit comprising a computer; a mechanical joint arm connected to the controller unit, the joint arm being capable of being positioned at a desired location by a user through application of a direct force; at least one gamma camera panel connected to the joint arm, the gamma camera panel comprising a gamma camera sensor with position and energy sensing resolution, the gamma camera panel providing an image field of view greater than 15 degrees; a camera mounted in such a way as to observe the entire area of ​​the patient; at least one processor and memory, the memory operatively coupled to the at least one processor, the camera, and the gamma camera sensor, the memory having instructions for execution by the at least one processor, the instructions causing the at least one processor to: reading a first gamma ray photon sensing event received from the gamma camera sensor; providing a first position and orientation of the gamma camera panel relative to a body of the patient; co-registering the first gamma ray photon sensing event with the patient's body using the first position and orientation; reading a second gamma ray photon sensing event received from the gamma sensor; and providing a second position and orientation of the gamma camera panel relative to the patient's body; co-registering the second gamma ray photon sensing event with the patient's body using the second position and orientation; reconstructing a 3D distribution of a gamma ray emitting radioisotope inside the patient by using the first and second co-registered sensed events; at least one processor and memory for causing the A system comprising: (Item 2) 2. The system of claim 1, wherein the articulated arm is a computer-controllable six-axis robotic arm. (Item 3) 2. The system of claim 1, wherein the memory further comprises instructions for causing the at least one processor to operate the mechanical arm to perform a SPECT scan. (Item 4) 4. The system of claim 3, further comprising wheels actuated by a motor coupled to the at least one processor and the memory, the wheels being actuated to move the SPECT system during scanning of the patient. (Item 5) 4. The system of claim 3, further comprising a proximity sensor mounted on the panel, the proximity sensor operatively coupled to the at least one processor and the memory, and wherein movement of the panel relative to the patient is modified in response to proximity sensor data acquired by the proximity sensor. (Item 6) The memory further comprises: generating a 3D model of the patient's body at regular intervals using computer vision; Detecting body changes and deformations occurring from one 3D model of the body to another 3D model. 2. The system of claim 1, having instructions to: (Item 7) 7. The system of claim 6, wherein the memory further comprises instructions for causing the at least one processor to notify the user of significant bodily changes that may require a reset of the SPECT scan. (Item 8) The memory further comprises: determining and assigning a first 3D patient body model to a first sensor detected event; determining and assigning a second 3D patient body model to the second sensor detected event; generating a tissue deformation model from the first 3D patient body model to the second 3D patient body model; reconstructing a 3D distribution of a gamma ray emitting radioisotope inside the patient by using the first and second sensed events and the tissue deformation model; 7. The system of claim 6, having instructions to: (Item 9) 2. The system of claim 1, wherein the memory further comprises instructions for causing the at least one processor to stop movement of the articulated arm and gamma camera panel to avoid collision between components of the SPECT imaging system and the patient's body. (Item 10) The memory further comprises: monitoring a space within a projected path of said gamma camera panel for other objects or people; Stopping the movement of the joint arm and gamma camera panel to avoid collisions between any components of the SPECT imaging system with other objects or people. 10. The system of claim 9, having instructions to: (Item 11) An ultrasonic probe; a tracking system used to determine a position and orientation of the ultrasound probe relative to the patient's body; and A visualization device; at least one processor and memory operatively coupled to the ultrasound probe, a tracking system, a memory storing a 3D distribution of a gamma ray emitting radioisotope co-registered to the patient or a reference point, and the visualization device, the memory having instructions for execution by the at least one processor, the instructions including: tracking the ultrasound probe relative to the patient or relative to a reference point; determining a co-registration between the 3D distribution of gamma ray emitting radioisotopes and an ultrasound scan using ultrasound probe tracking data acquired by the ultrasound probe; delivering to the visualization device an image comprising an enhancement of a feature of the 3D distribution of gamma ray emitting radioisotopes onto the ultrasound scan; and at least one processor and memory for causing the Item 1. The system of item 1, further comprising: (Item 12) Item 12. The system of item 11, wherein the tracking system is selected from the group containing an optical tracking system, an electromechanical tracking system, an electromagnetic tracking system, an ultrasonic tracking system, a depth imaging tracking system, or combinations thereof. (Item 13) The memory further comprises: reading the SPECT image data set from the memory; generating a model of movement of features in the image from a first ultrasound frame to a second ultrasound frame; generating a modified SPECT image based on a model of the movement of the features within the image; enhancing the modified SPECT image onto the second ultrasound frame; and Item 12. The system of item 11, having instructions to: (Item 14) 1. A real-time multi-modality portable single photon emission computed tomography (SPECT) imaging system for scanning a patient, the system comprising: a SPECT controller unit, the unit comprising at least a computer; and a mechanical joint arm connected to the controller unit, the joint arm being capable of being positioned at a desired location by a user through application of a direct force; at least one gamma camera panel connected to the joint arm, the gamma camera panel comprising a gamma camera sensor with position and energy sensing resolution; an ultrasound transducer, the ultrasound transducer being positionable in a manner such that the ultrasound transducer has a field of view that at least partially overlaps with a gamma camera field of view; a tracking system capable of providing tracking information about a relative position of the ultrasound transducer with respect to the gamma camera; A visualization device; at least one processor and memory, the memory operatively coupled to the gamma camera sensor, the ultrasound transducer, the tracking system, and the visualization device, the memory having instructions for execution by the at least one processor, the instructions causing the at least one processor to: reading a first gamma ray photon sensing event received from the gamma sensor; reading a second gamma ray photon sensing event received from the gamma sensor; and reconstructing a 3D distribution of a gamma ray emitting radioisotope inside the patient using the first and second sensed events; determining a co-registration between the ultrasound transducer and the gamma sensor using the tracking information; determining a co-registration between the 3D distribution of gamma ray emitting radioisotopes and an ultrasound scan using the co-registration between the ultrasound transducer and the gamma sensor; using the co-registration between the 3D distribution of gamma-ray emitting radioisotopes and an ultrasound scan to deliver to the visualization device an image comprising an enhancement of the 3D distribution of gamma-ray emitting radioisotopes onto the ultrasound scan; at least one processor and memory for causing the A system comprising: (Item 15) Item 15. The system of item 14, wherein the gamma camera panel provides an image field of view greater than 15 degrees, the image field of view being defined as a range of angles relative to a direction in which the gamma camera has maximum image sensitivity, from which gamma photons can be detected and imaged by a gamma sensor constituted by the gamma camera panel with a sensitivity greater than 1 / 100 of the maximum image sensitivity. (Item 16) Item 15. The system of item 14, wherein the reconstructed 3D distribution of gamma ray-emitting radioisotopes is characterized by a finite depth image resolution of less than 20 millimeters (mm) from an essentially static position of the gamma camera panel relative to the patient along a direction in which the gamma camera is most sensitive for a range of distances covering at least 50 mm. (Item 17) Item 15. The system of item 14, wherein the ultrasound transducer is affixed to the patient's body without a user holding the transducer. (Item 18) at least one processor and memory, the memory operatively coupled to the gamma camera sensor, the ultrasound transducer, and the tracking system, the memory having instructions for execution by the at least one processor, the instructions causing the at least one processor to: reading gamma ray photon sensing events received from said gamma sensor, thereby detecting gamma ray photons; correlating an ultrasound image map with said gamma ray photon sensing events; providing a tissue model from the ultrasound image map; generating a gamma photon attenuation model from the tissue model; determining an attenuation probability within the patient for the detected gamma ray photons using the gamma photon attenuation model; Item 15. The system according to item 14, (Item 19) Item 15. The system of item 14, comprising first and second gamma camera panels mounted to a distal end of the jointed arm, the panels being actuable such that the distance between the panels may be modified, the panels being actuable such that the relative angle between the panels may be modified, and the panels being actuable to separate to allow another medical instrument to be positioned between the panels. (Item 20) 20. The system of claim 19, wherein the medical instrument is an ultrasound probe. (Item 21) 20. The system of claim 19, wherein the medical instrument is a biopsy needle. (Item 22) 20. The system of claim 19, wherein the medical instrument is an ablation treatment device. (Item 23) 1. A portable single photon emission computed tomography (SPECT) imaging system for scanning a body part of a patient, the system comprising: a SPECT controller unit, the controller unit comprising at least a computer; a mechanical joint arm connected to the controller unit; a gamma camera panel connected to the joint arm, the gamma camera panel comprising a gamma camera sensor with position and energy sensing resolution, the gamma camera panel providing an image field of view greater than 15 degrees, the image field of view being defined as a range of angles relative to a direction in which the gamma camera has a maximum image sensitivity, from which gamma photons can be detected and imaged by a gamma sensor constituted by the gamma camera panel with a sensitivity greater than 1 / 100 of the maximum image sensitivity; a tactile sensor mounted on the panel, the tactile sensor operatively coupled to at least one processor and memory, and wherein movement of the panel relative to the patient is modified in response to tactile sensor data acquired by the tactile sensor; A system comprising: (Item 24) 24. The system of claim 23, further comprising a second gamma camera panel mounted to a distal end of the joint arm, the panels being actuable such that the distance between the panels can be modified, the panels being actuable such that the relative angle between the panels can be modified, and the panels being positionable essentially opposite and allowing a body part of the patient to be positioned between panel faces. (Item 25) 1. A system comprising: a gamma ray photon sensor with energy and position resolution sensing capability, said gamma ray photon sensor providing a position of a photon interaction; a coded aperture mask placed in front of the photon sensor, the mask comprises photon attenuating mask pixel elements shaped as bi-frustums; the physical space between bi-frustum mask pixel elements having a common edge is partially or completely occupied by material; the mask creates an image field in front of the sensor; A coded aperture mask; At least one processor; A memory operatively coupled to the sensor and the processor, the memory storing instructions for execution by the at least one processor, the instructions causing the processor to: projecting a location of the first photon interaction onto a reference plane to generate a first projected interaction point; reading a photon attenuation coefficient stored in the memory for the first projected interaction point relative to a direction toward the image field of view; projecting the location of the second photon interaction onto a reference plane to generate a second projected interaction point; reading a photon attenuation coefficient stored in the memory for the second projected interaction point relative to a direction toward the image field of view; reconstructing an image of a gamma ray source using the retrieved attenuation coefficients for the first and second photon interactions; and memory and A system comprising: (Item 26) 26. The system of claim 25, wherein the sensor provides the position of the photon interaction with a resolution of better than 4 mm in all three dimensions. (Item 27) Item 26. The system of item 25, wherein the coded aperture mask is made from a material having a density greater than 10 grams per cubic centimeter (g / cc). (Item 28) Item 26. The system of item 25, wherein the mask pixel elements are shaped as bi-pyramids having at least one side that makes an angle of greater than 10 degrees with respect to a normal on the bi-pyramid base. (Item 29) Item 26. The system of item 25, wherein the material between the bi-pyramid mask pixel elements has a density greater than 10 grams per cubic centimeter (g / cc). (Item 30) Item 26. The system of item 25, wherein the bi-pyramid mask pixel elements have a base selected from the group including a rectangular base, a triangular base, and a hexagonal base. (Item 31) 26. The system of claim 25, wherein the shape of the bi-frustum mask pixel element is approximated by a mask pixel element with curved sides. (Item 32) Item 26. The system of item 25, wherein the coded aperture mask extends across multiple planes. (Item 33) 26. The system of claim 25, further comprising a photon-attenuating shield in a direction around the sensor that is not covered by the coded aperture mask. (Item 34) Item 26. The system of item 25, wherein the coded aperture mask has an open fraction defined as the ratio of the area of ​​the unattenuated mask area to the total area of ​​the mask ranging from 0.1% to 70%. (Item 35) Item 26. The system of item 25, wherein the coded aperture mask is freestanding. (Item 36) Item 26. The system of item 25, wherein the coded aperture mask is constructed from multiple layers stacked together to approximate a bi-frustum shaping of the mask pixels. (Item 37) 1. A method comprising: projecting a location of a first photon interaction detected by a gamma ray photon sensor onto a first reference plane to generate a first projected interaction point, the gamma ray photon sensor having energy and position resolution sensing capabilities, the gamma ray photon sensor providing the location of the photon interaction, a coded aperture mask positioned in front of the photon sensor, the mask comprising photon attenuating mask pixel elements shaped as bi-frustums, the physical space between the bi-frustum mask pixel elements having a common edge being partially or completely occupied by material, the mask generating an image field in front of the sensor; reading a photon attenuation coefficient stored in a memory for the first projected interaction point relative to a direction toward the image field of view; projecting a location of a second photon interaction detected by the gamma ray photon sensor onto a second reference plane to generate a second projected interaction point; reading a photon attenuation coefficient stored in the memory for the second projected interaction point relative to a direction toward the image field of view; reconstructing an image of a gamma ray source using the retrieved attenuation coefficients for the first and second photon interactions; and A method comprising: [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 shows a diagram of a portable SPECT imaging system operated by a jointed arm.

[0018] [Diagram 2]2A-2D show diagrams of two gamma camera panel systems in two configurations.

[0019] [Diagram 3] FIG. 3 shows a representation of a portable SPECT camera system that is co-aligned with an ultrasound probe placed between the panels by using a visual camera.

[0020] [Figure 4] FIG. 4 shows a depiction of a portable SPECT camera system that is co-aligned with an ultrasound probe mounted on the side of the panel by using a visual camera.

[0021] [Diagram 5] FIG. 5 shows a representation of a portable SPECT camera system that is co-aligned with an ultrasound probe placed between the panels by using an articulated mechanical arm.

[0022] [Figure 6] FIG. 6 shows a depiction of a portable SPECT camera system that is co-registered with an ultrasound probe placed between the panels by using an electromagnetic field tracker.

[0023] [Figure 7] FIG. 7 shows a depiction of a portable SPECT camera system co-registered with an ultrasound probe to guide a percutaneous medical intervention.

[0024] [Figure 8] FIG. 8 shows a depiction of a portable SPECT camera system that is co-registered with an ultrasound probe used to correct for tissue deformation in SPECT imaging.

[0025] [Figure 9] FIG. 9 shows a representation of a portable SPECT camera system set up in a configuration for scanning a specific body part of a patient.

[0026] [Figure 10] FIG. 10 shows a depiction of an embodiment in which a portable cart integrates a SPECT gamma camera and a medical ultrasound system mounted on an articulating arm.

[0027] [Figure 11] FIG. 11 shows a depiction of the operational connections between the components of a portable imaging system, including a SPECT gamma camera and a medical ultrasound system.

[0028] [Figure 12] FIG. 12 illustrates an embodiment in which arrays of ultrasound transducers are aligned with each other and with a gamma camera panel that are used in combination to scan or treat a patient.

[0029] [Figure 13] FIG. 13 shows a depiction of a portable SPECT system used in combination with a separate medical imaging system such as a CT scanner.

[0030] [Figure 14] FIG. 14 shows a depiction of the fiducials used to provide co-registration between a portable SPECT system and another medical imaging scanner.

[0031] [Figure 15] FIG. 15 shows a depiction of a form-fitting garment that may be used by the patient to assist the optical-based computer vision tracking and mapping process.

[0032] [Figure 16] FIG. 16 shows the processing workflow for importing other image datasets to achieve multi-modality image fusion and to refine the SPECT reconstruction.

[0033] [Figure 17]FIG. 17 shows a cross-sectional side view of a wide field coded aperture imaging system.

[0034] [Figure 18] FIG. 18 shows a front view of a wide field coded aperture mask.

[0035] [Figure 19] FIG. 19 shows a profile view of a mask element.

[0036] [Figure 20] FIG. 20 shows a cross-sectional side view of a wide field coded aperture imaging system showing a coded aperture mask that extends across multiple planes.

[0037] [Figure 21] FIG. 21 shows a cross-sectional side view of a wide-field coded aperture imaging system illustrating how detected events are backprojected into image space using a reference plane for image reconstruction.

[0038] [Figure 22] FIG. 22 shows a cross-sectional side view of a wide-field coded aperture imaging system illustrating how detected events are backprojected into image space for image reconstruction using a reference plane that coincides with the mask plane.

[0039] [Figure 23] FIG. 23 shows a cross-sectional side view of a wide-field coded aperture imaging system illustrating how detected events are backprojected into image space for image reconstruction using multiple reference planes.

[0040] [Figure 24] FIG. 24 shows a perspective view of a wide-field coded aperture mask showing one of its compositions, in this case made up of multiple layers stacked together.

[0041] [Diagram 25]FIG. 25 shows a cross-sectional side view of a wide-field coded aperture imaging system that employs sensors arranged in different planes to minimize the range of gamma ray photon incidence angles impinging on the sensor.

[0042] [Figure 26] FIG. 26 shows a top view of an array of four sensor panels positioned in different planes to minimize the range of gamma ray photon incidence angles impinging on the sensors.

[0043] [Figure 27] FIG. 27 shows a cross-sectional side view of a coded aperture imaging system, here with an adjustable three-layer mask in a wide field configuration.

[0044] [Figure 28] FIG. 28 shows a cross-sectional side view of a coded aperture imaging system, here with an adjustable three-layer mask in a collimated (foveated) viewing configuration.

[0045] [Figure 29] 29A and 29B show schematic top views of a coded aperture mask made from nine panels arranged in three layers in two configurations: wide field and collimated field.

[0046] [Diagram 30] 30A and 30B show schematic top views of a coded aperture mask made from 19 panels arranged in three layers in two configurations: wide field and collimated field.

[0047] [Diagram 31] FIG. 31 shows a top view depiction of an intermediate layer in an adjustable mask having a pseudo-random pattern with a 22% open fraction.

[0048] [Diagram 32] FIG. 32 shows a top view depiction of a coded aperture mask with curved slits of various curvatures.

[0049] [Diagram 33] FIG. 33 shows an embodiment of a handheld SPECT camera showing the location of the wide-field coded aperture mask and the sensor.

[0050] [Diagram 34] FIG. 34 shows a top view of an embodiment of a handheld SPECT camera showing the location of the wide-field coded aperture mask and the sensor.

[0051] [Diagram 35] FIG. 35 shows a drawing of an embodiment of a handheld SPECT camera with a wide-field coded aperture that scans a patient's body from a reduced range of angles around the patient while simultaneously providing sufficient angular sampling of the image space.

[0052] [Diagram 36] FIG. 36 shows a flow chart summarizing some of the systems and methods enabled by the portable molecular imaging system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0053] Single photon emission computed tomography (SPECT) is the main molecular imaging modality used for medical imaging, either alone or in combination with computed tomography (CT) (SPECT / CT). Most commonly, a SPECT imaging device comprises an array of gamma-ray sensors that either surrounds the patient's body or orbits around the patient. During the image scan, the patient most commonly lies on a table, and for some cardiac imaging systems may be seated on a custom-made chair. A parallel-hole collimator is commonly used in front of the detector array to constrain the direction that the gamma-ray photons can take before interacting with the position-sensitive sensor. This produces a parallel projection of the distribution of gamma-ray-emitting isotopes inside the patient. A computer program is used to reconstruct this distribution in 3D by using analytical or iterative image reconstruction algorithms.

[0054] The sensor and associated collimator may be placed at a relatively long distance from the patient so that patients of various sizes can be accommodated. Since SPECT imaging sensors are characterized by a finite angular resolution, this larger standoff distance between the sensor and the radioisotope-tagged molecules may translate to lower image resolution and sensitivity. The reduced standoff distance will lead to increased sensitivity and resolution. Some SPECT imaging systems include sensors that can be operated to reduce the standoff distance to the body. Such systems may use parallel-hole collimators that change direction and capture projections at various angles. Thus, the standoff distance between the patient's body and the sensor may be reduced, which may improve performance.

[0055] Computed tomography (CT) may be used in conjunction with the SPECT images to provide 3D morphological context to the molecular images provided by SPECT. Co-registration of the two 3D maps allows the radiologist to identify organs where increased X-ray contrast has increased uptake. Additionally, the CT map can achieve more accurate reconstruction of the SPECT map by providing a photon attenuation map that allows the SPECT image reconstruction algorithm to take into account the photon attenuation between the image voxels and the SPECT image sensor.

[0056] The embodiments comprise an improved SPECT imaging device that is portable and allows for co-registration with ultrasound. The SPECT imaging device described herein can be used to guide medical interventions such as biopsy and ablation therapy, and can also be used to guide surgical procedures. The improved SPECT imaging device also provides for co-registration with other medical imaging modalities such as X-ray, CT, and various optical imaging modalities.

[0057] Among other aspects, the SPECT imaging devices described herein provide portable, high-image-performance molecular imaging using SPECT radioisotope-labeled molecular agents. The SPECT imaging devices described herein provide a novel multi-modality imaging system and corresponding methods that combine SPECT images with ultrasound images and other medical imaging modalities to enable multiple medical uses, such as imaging diagnostics, biopsy guidance, ablation therapy guidance, and surgical guidance.

[0058] These and other advantages of one or more aspects will become apparent from a consideration of the ensuing description and accompanying drawings.

[0059] The portable SPECT imaging system and associated method can provide images of radiotracer distribution inside a patient's body with high resolution and sensitivity by employing at least a specifically designed gamma camera panel mounted on a jointed mechanical arm. In some embodiments, the gamma camera panel utilizes gamma photon imaging, which provides the distance from the gamma camera to the gamma radiotracer location even when the gamma camera scans the patient from an essentially static location. In some embodiments, the jointed mechanical arm is a 6-axis robotic arm. In some embodiments, the jointed mechanical arm is actuated by a user, either by applying a direct force or through the use of a computer. In some embodiments, the robotic arm can be actuated by a computer to perform automated specific examinations of the patient, such as whole body scans, head scans, neck scans, heart scans, torso scans, etc. For navigation purposes, a computer vision system can be used to provide information regarding the location and position of the patient, generate a 3D model of the patient's body, and identify specific body parts. A tracking system may be used to determine the positions of the SPECT system components relative to each other and to the patient. The 3D model of the patient can be combined with the tracking information to guide the SPECT scan. In some embodiments, the SPECT system may be mounted on motorized wheels, and these wheels may be actuated during the scan to extend the reach of the gamma camera panel around the patient.

[0060] The embodiments further include a multi-modality imaging system and method for co-registering the portable SPECT imaging system with other imaging modalities. In an embodiment, an ultrasound probe and corresponding ultrasound image are co-registered with the portable SPECT imaging system using a tracking system to enable molecular imaging combined with ultrasound image guidance of medical intervention, molecular imaging with correction for tissue deformation, and molecular image guided ultrasound examination. In an embodiment, the portable SPECT imaging system comprises two physically separated gamma camera panels that can be actuated to leave a space between them and to be oriented at various angles relative to each other. In yet another embodiment, another imaging probe, such as an ultrasound probe, is essentially placed between the two gamma camera panels, such that at least the image fields of view of the gamma camera panels and other medical imaging devices overlap. This embodiment allows a user to visualize the SPECT image co-registered with the ultrasound image in real time. Other medical instruments may be placed in the space between the two panels. Examples of such medical instruments are percutaneous biopsy devices and ablation therapy devices. The use of these instruments may be guided by SPECT and / or ultrasound images.

[0061] The presented portable multi-modality imaging SPECT and ultrasound system and method can offer several advantages over previous multi-modality SPECT / CT and PET / CT systems, among them elimination of radiation dose associated with CT by providing ultrasound-delivered co-registered anatomical information; portability afforded by a much smaller robotic SPECT and ultrasound system; increased molecular imaging resolution and sensitivity by performing image scanning with a gamma camera panel placed much closer to the patient's body; precise and easy guidance of medical interventions; in-operating room adoption to guide surgical procedures; and improved molecular imaging quality by using real-time ultrasound to guide molecular imaging corrections that account for tissue deformation and organ movement captured by the ultrasound transducer.

[0062] Among other aspects, embodiments of the portable SPECT imaging system and associated methods also enable scanning of a patient from a limited range of angles around the patient without loss of image performance. This advantage over previous systems is provided by embodiments in which the gamma camera panel provides image resolution in a direction normal to the face of the gamma camera panel from an essentially static location. This advantage is also provided by embodiments in which the gamma camera panel provides an image field of view that is greater than 15 degrees, and preferably close to 45 degrees. The image field of view is defined as the range of angles in at least one direction off the normal to the gamma camera panel from which gamma photons can be detected and imaged by the gamma sensor comprising the gamma camera panel. Scanning a patient from a limited range of angles, while usually not preferred, may be subject to various operational and specific image task constraints, such as a requirement for reduced scan time, limited physical access of the image sensor panel around the patient, or increased radiation attenuation or attenuation non-uniformity in some directions around the patient that may contribute to increased image artifacts.

[0063] Image co-registration between the portable SPECT system, other medical imaging modalities, and medical instruments can be achieved by using a position tracking system that captures the relative positions of other medical imaging sensors, such as gamma sensors, ultrasound transducers, and medical instruments, relative to each other and to the patient's body. A range of position tracking systems can be used alone or in combination. Such position tracking systems can use inertial measurement units (IMUs), RGB cameras, depth image sensors, infrared cameras, optical systems such as stereoscopic optical systems, electromechanical joint arms, electromagnetic field trackers, ultrasound trackers, servo motors, or any other device suitable for providing the position and orientation of a component of interest with suitable accuracy.

[0064] In some embodiments, the gamma camera panel comprises a gamma photon attenuation imaging component placed in front of the position sensitive gamma ray sensor. The photon attenuation imaging component can be selected from a group including a coded aperture mask, a linear and / or tilted parallel hole collimator, a pinhole collimator, or a multiple pinhole collimator. In a preferred implementation, the gamma ray sensor provides the location of gamma photon interactions with a resolution better than 2 millimeters (mm) in all three dimensions. In some embodiments, the photon attenuation imaging component is a coded aperture mask with a field of view greater than 20 degrees. Preferably, the pattern of the coded aperture mask minimizes side lobes in the autocorrelation function of the instrument (see, e.g., Fenimore, Edward E. and Thomas M. Cannon. See, "Coded aperture imaging with uniformly redundant arrays." Applied optics 17.3 (1978): 337-347. Repeating patterns at different translations without rotation or reflection are discouraged as they may produce peaks in the side lobes of the autocorrelation function for the magnification of interest, leading to reconstruction artifacts. Similarly, apertures that produce long straight slits that may be essentially as long as the mask width, or as long as a significant portion of the mask width, and especially apertures that produce multiple long straight slits that are parallel to one another, may produce image artifacts for this application. The coded aperture pattern may have an open fraction, defined as the ratio of empty mask pixels to the total number of pixels, ranging from near 0% to near 100%. In some embodiments, the open fraction may range from 0.1% to about 80%. In some embodiments, the open fraction may range from 5% to 30%. Such an open fraction may maximize the image signal-to-noise ratio for a given distribution of molecular SPECT agents in humans. In some embodiments, an adjustable mask may achieve a range of open percentages, such as 5% to 70%, by adjusting the mask elements. For example, a mask comprising overlapping layers with overlapping or partially overlapping openings (holes) may achieve such adjustability by moving the layers laterally relative to one another. Other assemblies of mask elements may produce the same effect.

[0065] In some SPECT imaging systems, the sensor and associated collimator may be placed at a relatively long distance from the patient so that patients of various sizes can be accommodated. Since SPECT imaging sensors are characterized by a finite angular resolution, this larger standoff distance between the sensor and the radioisotope-tagged molecules may translate into lower image resolution and sensitivity. The reduced standoff distance may lead to increased sensitivity and resolution. Some SPECT imaging systems include sensors that can be operated to reduce the standoff distance to the body. Such systems may also include parallel-hole collimators that change direction and capture projections at various angles. Thus, the standoff distance between the patient's body is reduced and sensor performance is improved.

[0066] Embodiments include alternative image and sensing modalities that allow the image sensor assembly to approach the patient's body in closer proximity while adequately sampling the image space to provide even greater image sensitivity and resolution in 3D. These techniques eliminate the need to capture sensor data uniformly around the patient by allowing efficient sampling of multiple projection angles from a reduced (tomographic) range of angles around the patient.

[0067] Among other aspects, embodiments enable the construction of a high-resolution three-dimensional map of radioisotope distribution inside a patient by scanning the patient from a reduced range of directions around the patient and with a radiation sensor placed in close proximity to the patient. This leads to increased image sensitivity and resolution. Thus, some advantages of one or more aspects are that they enable SPECT images with high resolution and efficiency through the employment of compact SPECT image sensors and systems that can be portable or even handheld.

[0068] Among other aspects, embodiments include imaging systems and associated methods that enable imaging of radiotracer distribution inside a patient's body with high resolution and sensitivity in three dimensions (3D) by using a sensor assembly that scans the patient from close proximity to the patient and from directions that may cover only a limited range of angles around the patient. Sensor proximity to the patient may be used because it may enable better image resolution and, in some cases, such as current approaches, better image sensitivity. Scanning a patient from a limited range of angles is usually not preferred, but may be subject to various operational constraints such as limited scan time, limited physical access of the image sensor around the patient, or increased radiation attenuation or attenuation non-uniformity in some directions around the patient that may contribute to increased image artifacts.

[0069] A major requirement of such a sensitive imaging system is a wide image field of view to overcome scanning the patient from a limited range of orientations. At the same time, it is important that the imaging system provide high image resolution and high sensitivity, which previously required a narrow field of view.

[0070] The embodiment enables SPECT imaging using an image sensor that provides both a wide image field of view and high image resolution and sensitivity. The image sensor includes a custom-designed wide-field radiation attenuation mask placed in front of a radiation position sensitive sensor, and a processing unit that enables efficient and effective image reconstruction techniques that result in reconstructed images of excellent image resolution, signal-to-noise ratio (SNR), and sensitivity. To image gamma ray photons, the attenuation mask may be made of a high-density, high atomic number Z material, such as tungsten, tungsten alloys, etc.

[0071] In some embodiments, the mask can be a coded aperture mask with coding elements shaped in such a way as to enable a wide field of view image. The coded aperture pattern contains a combination of radiation attenuating pixels and empty non-attenuating pixels. In this description, the attenuating pixels will be referred to as mask pixels and the non-attenuating pixels will be referred to as empty mask pixels. Examples of coded aperture patterns that can be used in the mask are a uniformly redundant array, a modified uniformly redundant array, a pseudo-random array, a random array, or any other pattern. In some embodiments, the coded aperture pattern minimizes side lobes in the response function of the instrument. The coded aperture pattern can have an open percentage, defined as the ratio of empty mask pixels to the total number of pixels, ranging from near 0% to near 100%. However, the most useful open percentage can range from a percentage of 1% to about 50%.

[0072] To adequately attenuate gamma ray photons between 120 keV and 170 keV, which correspond to the specific gamma rays of many SPECT isotopes, the attenuation mask may need to have a thickness that is within the same range, if not greater, than the size of the mask element. For example, fabricating a mask from a tungsten alloy, a mask thickness of about 2 mm may be used to attenuate 140 keV photons, which may be within the same range as the size of the mask pixel element.

[0073] Using mask pixel elements shaped as rectangular parallelepipeds, i.e., with straight side edges and single planar sides for every Cartesian direction in the mask plane, provides poor performance, and adequate encoding contrast can only be achieved for photons with low angles of incidence, up to 10-20 degrees from the normal on the mask plane, when assuming a single-sided mask and square mask pixels. Due to this limitation of rectangular parallelepiped mask pixels, embodiments provide for customized shaping of mask pixel elements.

[0074] In some embodiments, the sides of the pixel elements are designed using customized geometries that, in conventional masks, include non-linear side edges and multiple planar sides in each of the two Cartesian directions perpendicular to the mask surface. Curved sides can also be used in place of multiple planar sides.

[0075] The above described sides can be combined with various geometries of mask pixel elements, such as square, triangular, and hexagonal pixels, where the sides of the triangle or hexagonal prism can each contain multiple flat or curved sides. These geometries can provide image coding with high contrast for radiation photons within a wide range of angles of incidence relative to the normal to the mask surface. For example, such angles of incidence can reach at least 50 degrees from the normal to the mask surface. Thus, customized shaping of mask elements within a single mask may combine edges, ridges, and curvatures of various geometries and orientations.

[0076] In some specific embodiments, square or rectangular mask pixel elements in a mask array can be constructed, for example, from a bi-frustum (or bi-pyramid) with a square or rectangular base. The bi-frustum can be symmetric about the center of the rectangular base, symmetric about the rectangular base face, or may have no such symmetry. The shape of the bi-frustum mask element can vary across the mask. When two mask pixels have a common edge, the space between two adjacent bi-frustums can be filled with a damping material. Instead of planar sides, curved sides can be used to approximate the shape of the bi-frustum. Similarly, multiple thin layers of masks can be stacked together to approximate the shape of the bi-frustum mask element and the fill material between adjacent bi-frustums.

[0077] A bi-pyramid frustum with a triangular or hexagonal base can be used for mask pixel elements that are triangular or hexagonal, respectively. The bi-pyramid frustum can be symmetric about the center of the triangular or hexagonal base, symmetric about the triangular or hexagonal base face, or may have no such symmetry. The shape of the bi-pyramid mask element can vary across the mask. Filling attenuating material can again be used between mask elements that share a common edge.

[0078] The masks can be arranged in a plane or in multiple planes that may or may not be parallel to each other. A shield may be used to cover portions of the space around the sensor that are not covered by the coded aperture mask and to limit detection of photons from other angles that are not covered by the coded aperture.

[0079] To achieve better image resolution, the position sensitive sensor may capture all three coordinates of the radiation interaction inside the sensor with high resolution, such as with a resolution of better than 1 or 2 mm in all three dimensions. Furthermore, the sensor may provide radiation detection with high energy resolution, as this may allow for better selection of events that are not scattered before reaching the mask and sensor. Such sensors may be scintillators, semiconductors, or other types of radiation detectors capable of providing the location of the radiation interaction.

[0080] An image reconstruction analysis package is used to process the sensor data to reconstruct the 3D distribution of radioactive molecules. In a particular implementation, an attenuation map is pre-calculated to relate the sensor location to a distribution of incident gamma ray photon angles. This distribution is weighted by the probability of these photons having traversed the coding mask. The detailed description of the edges of each mask pixel increases the information bandwidth of the mask coding. Thus, the attenuation map can be complex and can be pre-computed by using high resolution ray tracing and / or Monte Carlo simulation. In a particular implementation, the attenuation map is pre-calculated at least across a plane called the reference plane. For each point in the reference plane, the attenuation map comprises attenuation coefficients, or other information that can be used to extract attenuation coefficients, such as path lengths through the mask material, for a given type of radiation and for various angles across the image field of view. The calculation of the attenuation map may use ray tracing methods, simulation methods such as Monte Carlo simulation methods, or a combination thereof to determine the radiation paths through the shielding and mask assembly from various origin angles across the field of view. For a particular type of radiation, the attenuation factor can then be calculated from the path values.

[0081] For each radiation detected by the radiation sensor, the 3D position of interaction is first projected onto a reference plane using various sampling techniques. The sampling is focused in a direction toward the coded mask field of view, and its projected position onto the reference plane is used to determine the attenuation coefficients through the mask along those directions. If the path of the photon along a direction can include a section through other materials that are not captured by the sensor or the pre-calculated attenuation coefficients, an additional attenuation calculation can be added to appropriately scale the intensity of the backprojection along that direction. The backprojection process can determine a probability density function (pdf) for the detected event or for the detected intensity within a bin. The pdf for at least two detected radiation events can be used in any suitable iterative or analytical image reconstruction algorithm known in the art. The image reconstruction analysis can be performed in list mode or in binned mode. If the analysis is performed in binned mode, the binning can be performed across multiple planes, and each detected photon can be added onto the closest plane in space.

[0082] In some implementations, the attenuation coefficients may also be calculated on-the-fly during image reconstruction analysis by using a fast path estimator such as a ray tracing method, however, pre-computation of these coefficients may provide the best image reconstruction processing speed performance.

[0083] FIG. 1 shows a depiction of a portable SPECT imaging system with a mechanical joint arm, such as a robotic arm. An instrument controller (100) is connected to at least a gamma camera panel through a mechanical joint arm (101). In the depicted embodiment, two gamma camera panels (102a) and (102b) are used. The panels (102a) and (102b) may be attached to each other and to the joint arm through a joint lever (103). This joint lever may allow the two panels to move relative to each other and to the joint arm, and may allow the panels to change their relative orientation, in particular their roll, but also their pit and yaw. Thus, the relative angle between the panels can be modified. A computer vision camera (106) may be attached to the joint arm (101), to the lever (103), or to another component connected to the portable SPECT system through a connector (107). The computer vision camera (106) may be used to monitor the overall area in which the gamma camera panel is scanning the patient. The computer vision camera may comprise an RGB camera, an optical camera, an infrared camera, a depth imaging optical camera, a structured light camera, a stereo optical camera, a time-of-flight optical camera, a terahertz emitter-sensor assembly, a lidar scanner, an ultrasound emitter-sensor assembly, another tracking and / or mapping sensor, or a combination thereof. The purpose of the camera is to determine the extent of the patient's body such that the movement of the gamma camera panel will not collide with the patient's body. In some embodiments, an ultrasound scanner such as a tetrahertz imaging scanner or an electromagnetic scanner may be used in addition to or instead of the optical camera to scan the contours of the patient's body.

[0084] Using such a portable SPECT imaging system, a patient scan can be performed by having instructions in a computer memory operatively coupled to actuated components such as the robotic arm (101), lever (103), and gamma sensor consisting of gamma cameras (102a) and (102b), and computer vision camera (106) to read and analyze the computer vision camera data, determine the contours of the patient's body and determine the relative position and orientation of the computer vision camera (106) with respect to the patient's body, articulate the arm (101) and lever (103) so that the gamma camera panels (102a) and (102b) are moved at relevant locations around the patient, obtain data from the sensor consisting of gamma cameras (102a) and (102b), spatially register the gamma sensor data with respect to the patient's body, and use the spatially registered sensor data to generate a 3D image of the radiotracer distribution inside the patient's body. The gamma sensor provides the position and energy of the gamma photon interaction with high resolution within the sensor. Preferably, the gamma sensor provides the position of the gamma event interaction in 3D with a resolution better than 2 mm, and ideally with a resolution better than 0.5 mm.

[0085] In some embodiments, the controller may comprise wheels (108a) and (108b). In such an embodiment, the memory may be operatively coupled to at least the motors that actuate the wheels (108a) and (108b) and move the controller (100). This may allow the system to scan a larger area over a longer distance than would otherwise be possible with the articulated arm (101). For navigation purposes and to avoid obstacles, other sensors (109) may also be installed at one or both ends of the controller to ensure that there are no obstacles in the path of the system. In such an embodiment, the memory would also be operatively coupled to the sensors (109) to guide the actuation of the wheels (108a) and (108b). The sensors (109) may be optical sensors, lidar scanners, depth imaging sensors, ultrasonic sensors, or any other sensors that may detect obstacles.

[0086] In some embodiments, the panels (102a) and (102b) may include proximity sensors (110a) and (110b), preferably installed on the panel surface facing the patient, to obtain information about the proximity between the panel and the patient's body or between the panel and other objects. The sensors (110a) and (110b) may be part of a computer vision system and may be capable of providing a 3D model of the patient's body directly under the sensor panel in real time. This map may be used by the computer to adjust the movement of the scanner across the patient's body and to provide an estimate of the attenuation map used in the image reconstruction process. These proximity sensors may be operatively coupled to a computer, which is also connected to an actuator that moves the panel relative to the patient. The computer may use a program to modify or stop the scanning process to keep the distance between the panel and the patient or other object within a desired range of distances. The proximity sensors may be capacitive, inductive, magnetic, ultrasonic, optical, terahertz, x-ray backscatter, or any other sensor capable of providing the distance to an object.

[0087] In some embodiments, the mechanical articulated arm (101) may be actuated by a user. In such cases, the purpose of the articulated arm is to support the weight of the gamma camera panels (102a) and (102b) and potentially determine the position and orientation of the panels relative to the body of the controller (100). Additionally, in some embodiments, the articulation of the lever (103) may be actuated by a user to manually position the gamma camera panel in a location and orientation desired by the user. In some embodiments, the user may actuate the arm (101) and lever (103) through the application of direct force. In some embodiments, the user may actuate the arm (101) and lever (103) through a computer operably coupled to a motorized actuator that is mechanically connected to the arm (101) and lever (103).

[0088] A computer, comprising at least one processor and a memory operatively coupled to a computer vision camera (106) and a gamma camera sensor consisting of gamma camera panels (102a) and (102b), reads gamma sensor data and computer vision camera data, respectively, reads a first gamma ray photon sensing event received from the gamma sensor, provides a first position and orientation of the gamma camera panel (102a) or (102b) that senses the first photon relative to the patient's body (104), and determines the first position. A position and orientation can be used to co-register a first gamma ray photon sensing event with the patient's body (104), read a second gamma ray photon sensing event received from the gamma sensor, provide a second position and orientation of the gamma camera panel relative to the patient's body (104), co-register the second gamma ray photon sensing event with the patient's body (104) using the second position and orientation, and use the first and second co-registered sensing events to reconstruct a 3D distribution of gamma ray emitting radioisotopes inside the patient.

[0089] Co-registration of the sensed events to the patient's body can be performed by analyzing computer vision camera frames using computer vision methods known in the art to determine a pose estimate of the computer vision camera (106) pose relative to the patient's body. Additionally, the relative position and orientation between the camera and the gamma camera panel (102a) or (102b) can be achieved either by direct observation of the panel by the camera (106) or by obtaining joint status information regarding the joint in the lever (103) and / or potentially the joint arm (101). A combination of the two relative poses (computer vision camera-patient and computer vision camera-sensing gamma camera panel) can be combined to obtain co-registration of the sensed events to the patient's body. Other tracking and co-registration systems and methods may also be used, some of which are described elsewhere in this description.

[0090] The computer vision camera frame data may be periodically analyzed during the SPECT scan by a computer vision program to generate an updated 3D model of the patient's body. By monitoring the changes from one frame to another, it may be possible to detect changes in body posture and body deformation. These detections may be used to improve the quality of the reconstructed SPECT image by taking into account such body changes, may be used to stop the scanning process and avoid collisions between any components of the SPECT imaging system with the patient or other users, or may inform the user of significant body changes that may require a reset of the SPECT scan.

[0091] Thus, a computer connected to the camera (106) may employ computer vision methods to generate 3D models of the patient's body at regular intervals and detect body changes and deformations that occur from one 3D model of the body to another. By analyzing the amplitude and type of body changes, the computer may inform the user of significant body changes that may require a reset of the SPECT scan, since co-registration between detected events occurring before and after body modification may not be certain.

[0092] Further, the computer may also be operatively coupled to the gamma camera sensor, and may determine and assign a first 3D patient body model to the first sensor detected events, determine and assign a second 3D patient body model to the second sensor detected events, generate a tissue deformation model from the first 3D patient body model to the second 3D patient body model, and perform a reconstruction of the 3D distribution of the gamma ray emitting radioisotope inside the patient by using the first and second sensed events and the tissue deformation model.

[0093] In one example of such reconstruction, the computer uses an image space remapping method to generate correspondence between image elements or image nodes before and after the body deformation, and uses a backprojection operator that considers the remapped image elements or nodes in relation to the two detected gamma events. Several remapping algorithms known in the art can be used. The computer operably connected to the computer vision camera can also be operably connected to actuators that power the mechanical arm (101), lever (103), and wheels (108a) and (108b). When the computer vision algorithm detects a significant change in the patient's body, even indicative of a collision, the computer can be programmed to stop the movement of the joint arm and the gamma camera panel to avoid a collision between any component of the SPECT imaging system with the patient's body. Furthermore, the computer vision subsystem can monitor the space in the projected path of the gamma camera panel during the scan and detect other objects or people. When such detection occurs, the computer can stop the movement of the joint arm and the gamma camera panel to avoid a collision between any component of the SPECT imaging system with other objects or people.

[0094] In some embodiments, articulated arm (101) may be attached to another physical object instead of being directly attached to the body of controller (100). Examples of such objects are a floor, a ceiling, a wall, another portable controller, a rail.

[0095] In some embodiments where a user actuates one or more of the articulated arm, lever arm, and gamma camera panel assemblies to perform static scanning (e.g., the gamma camera panel does not move relative to the patient) or dynamic scanning (e.g., the patient with the gamma camera panel moving relative to the patient), no other tracking modalities external to the camera (106) and / or articulated arm may be used. In such embodiments, co-registration of sensed events to the patient's body may be performed by reading and analyzing position data of the articulated lever (103), articulated arm (101), and any articulations or actuators involved in the movement of the gamma camera panel within the room, such as wheels (108a) and (108b). This co-registration modality assumes that the patient's body remains stationary relative to the room in which the gamma camera panel is moved to perform the scan. In some embodiments, the bed (105) or chair on which the patient lies or sits may be moved relative to the controller to enhance the coverage area of ​​the panel during the scan.

[0096] In some embodiments, the gamma camera panels (102a) and (102b) are equipped with gamma sensors that provide sensor data containing information for use by an image reconstruction program running on a computer to provide an image of the radiotracer distribution with a finite resolution along the direction in which either gamma camera is most sensitive, even when the gamma camera panels are in an essentially static location relative to the patient. In some embodiments, the finite resolution is less than 20 mm for a range of distances covering at least 50 mm.

[0097] In some embodiments, the gamma camera panels (102a) and (102b) each provide an image field of view that is greater than 15 degrees, preferably close to 45 degrees. In some embodiments, the field of view can be greater than 20, 25, 30, 35, 40, or 45 degrees. The image field of view is defined as the range of angles relative to the direction in which the gamma camera has maximum image sensitivity, from which gamma photons can be detected and imaged by a gamma sensor consisting of a gamma camera panel with a sensitivity greater than 1 / 100 of the maximum image sensitivity. Such an image field of view allows the gamma camera sensor to capture image information from multiple directions, which allows for better coverage of image projection angles, even from a static location or from a reduced range of locations around the patient.

[0098] In some embodiments, the gamma camera panels (102a) and (102b) each comprise a gamma photon attenuation imaging component placed in front of a position sensitive gamma ray sensor. The photon attenuation imaging component can be selected from a group including a coded aperture mask, a linear and / or tilted parallel hole collimator, a pinhole collimator, or a multiple pinhole collimator. In a preferred implementation, the gamma ray sensor provides the location of the gamma photon interaction with a resolution better than 2 mm in all three dimensions. In a preferred embodiment, the photon attenuation imaging component is a coded aperture mask with a field of view greater than 30 degrees. Preferably, the pattern of the coded aperture mask minimizes side lobes in the instrument response function. The coded aperture pattern can have an open fraction, defined as the ratio of empty mask pixels to the total number of pixels, ranging from near 0% to near 100%. In some embodiments, the open fraction can range from a fraction of 1% to about 50%.

[0099] In some embodiments, the gamma sensor consisting of the gamma camera panels (102a) and (102b) is selected to detect gamma photons of higher energy with higher sensitivity, such as energies above 200 kiloelectron volts (keV) or above 500 keV. In this case, no gamma photon attenuation imaging components may be used, and instead, a gamma ray Compton scattering mechanism may be used to provide image information. The gamma sensor may be selected to provide position resolution in 3D with a resolution better than 2 millimeters (mm) and an energy resolution better than 4%. In this case, a computer operably coupled to the gamma sensor will determine the scattering angles around the scattering direction of gamma rays that interact at least twice as much in the sensor system by solving the kinematics of gamma ray interactions in the sensor system. These scattering angles around the scattering direction will then be used to generate spatially aligned cones. A 3D map of the gamma ray source can then be constructed by statistically resolving the intersection of at least two spatially aligned cones. The imaging modality may be used to image positron-emitting radioisotopes.

[0100] In some embodiments, the jointed mechanical arm (101) is a 6-axis robotic arm. A computer operatively coupled to an actuator inside the robotic arm can operate the joints to perform a SPECT scan by moving the panel around the patient's body. A computer vision process may run on the computer and analyze image frames from the computer vision camera (106) or from a separate patient 3D scanner to build a 3D model of the patient and identify individual body parts. Instructions may be stored on the computer to use the output of the computer vision process to operate the robotic arm and position the panel in a manner to perform a specific type of image scan, such as a head scan, neck scan, full body scan, heart scan, torso scan, etc.

[0101] In other embodiments, the portable SPECT imager may include a single gamma camera panel connected to a jointed mechanical arm. In other embodiments, the portable SPECT imager may include three, four, or more gamma camera panels connected to a jointed mechanical arm, either directly or through a similar lever as in FIG.

[0102] In some embodiments, computer vision cameras can be mounted on the gamma camera panels (102a) and (102b), lever (103), or other objects. Like the computer vision camera (106), this camera can provide data to be analyzed by a computer to determine the contours of the patient's body and to determine the relative positions and orientations of various SPECT system components with respect to each other and with respect to the patient's body.

[0103] Following the SPECT scan that achieved the 3D distribution of the gamma-emitting radioisotope, a computer operatively connected to the ultrasound probe or ultrasound transducer, a tracking system employed to track the ultrasound probe relative to the patient's body or relative to a fixed reference point, a memory storing the 3D distribution of the gamma-emitting radioisotope co-registered with the patient, and a visualization device may be used to track the ultrasound system relative to the patient or relative to a fixed reference point. The computer may further use the ultrasound tracking information to determine the co-registration between the 3D distribution of the gamma-emitting radioisotope and the ultrasound scan, and deliver an image to the visualization device comprising an enhancement of the features of the 3D distribution of the gamma-emitting radioisotope onto the ultrasound scan. Furthermore, the computer may generate a tissue deformation model by analyzing the live ultrasound image and tracking specific features in the ultrasound image from the first ultrasound frame to the second ultrasound frame. This deformation model may be used to remap the original SPECT image stored in the memory and generate a modified SPECT image that integrates the modeled tissue deformation. This modified SPECT image may then be augmented onto the second ultrasound frame.

[0104] Other tissue deformation corrections may be used, such as frame-based registration approaches and gamma sensor data-driven approaches such as distribution center of gravity (e.g., detecting body movement based on changes in the center of gravity of a distribution trace over a time interval).

[0105] In some embodiments, the platform (105) may also include a lever arm (110) that may extend out from the body of the platform when the articulated arm is extended to address the potential for tipping of the instrument. Such an arm may include a wheel (111) at its distal end.

[0106] It should be understood that an ultrasonic probe is a structure that includes an ultrasonic transducer, and the terms "ultrasonic probe" and "ultrasonic transducer" may be used interchangeably in the description.

[0107] In some embodiments, instead of using gamma ray panels mounted on the robotic arm, other types of sensors may also be used. For example, one or both panels (102a) and (102b) may comprise a magnetic sensor or a magnetic sensor array. Such a magnetic sensor system may be capable of measuring a magnetic signal or a change in a magnetic signal that reflects the magnetic properties of the tissue. In some implementations, the magnetic properties of the tissue are defined by molecular agents that are injected into the patient's body. In some implementations, these molecular agents may be labeled with magnetic nanoparticles. The panel may move across the patient's body and capture magnetic signals from different locations. A three-dimensional (3D) map of the distribution of the agents may be reconstructed by an operably connected computer by using the spatially registered magnetic signal data from the magnetic sensor assembly. The magnetic signal data may be spatially registered by the computer by using tracking data from a computer vision system or from the robotic arm kinematics. Although examples include using gamma sensors for molecular imaging and molecular imaging guidance of intervention, it should be understood that the methods and systems described herein may be envisioned by using sensor panels comprising magnetic sensor assemblies for molecular imaging. These magnetic sensor systems may be used in combination with injecting molecular agents with specific magnetic properties inside the patient's body.

[0108] 2A-2D show multiple views of a SPECT camera head with gamma camera panels (200a) and (200b), respectively, connected through mechanical arms (202a) and (202b) to a mechanical lever (201). A computer vision camera (203) is seen connected to the end of the distal end of the mechanical joint arm (205) through a connector (204).

[0109] FIG. 2A shows two panels (200a) and (200b) in a close configuration.

[0110] Figure 2B shows two panels (200a) and (200b) in a separated configuration where the relative distance between the panels has been increased and the roll of the panels has been changed, which would provide a space between the two panels for other imaging probes or medical instruments to be introduced.

[0111] 2C and 2D show side views of the SPECT camera head in two configurations. An assembly made of lever (201) and arms (202a) and (202b) can allow the gamma camera panels (200a) and (200b) to move sideways (away from each other) and from front to back.

[0112] FIG. 2C depicts the gamma camera panel in the forward configuration.

[0113] Figure 2D depicts the gamma camera panel in a rearward configuration where the center of gravity of the panel is located closer to the connector between the lever (201) and the joint arm (205). The configuration of Figure 2C may allow other imaging probes to be located between the two gamma camera panels by providing empty space above and between the gamma camera panels, while the configuration of Figure 2D may allow other imaging probes to be located on the sides of the two panels.

[0114] Various configurations and relative placement of the gamma camera panels may be used not only to accommodate other medical instruments or imaging probes being introduced into the field of view of the gamma camera, but also to allow various levels of overlap of the image field of view between the two panels, as well as to enable patient scanning by closely following the patient anatomy.

[0115] FIG. 3 shows a depiction of a SPECT camera head used in combination with another medical imaging probe. Two gamma camera panels (300a) and (300b) are shown mechanically connected to the distal end of a mechanical joint arm (302) by a lever (301). The system is shown scanning a patient (303). In this embodiment, an ultrasound imaging probe (304) is introduced into the space between the two panels (300a) and (300b). The image field of view of the ultrasound imaging probe (304) may partially or completely overlap onto the image field of view of either or both panels (300a) and (300b).

[0116] A computer vision camera (305) is connected to the lever (301) and the articulated arm assembly (302) through a connector (306). The computer vision camera may have a field of view that covers the general area in which the ultrasound probe is used to scan the patient. The computer vision camera provides data required by a tracking program stored in the memory of a computer operatively coupled to the computer vision camera to provide the location and orientation of the ultrasound probe relative to the camera. A fiducial marker (307) may be attached to the ultrasound probe to assist in determining the location and orientation of the ultrasound probe relative to the camera (305). Similarly, the computer vision camera may provide data required by a tracking program stored in the memory of the computer to provide the location and orientation of each of the gamma camera panels (300a) and (300b) relative to the computer vision camera (305). A fiducial marker (not shown) may be attached to each of the two gamma camera panels to assist in determining the location and orientation of each of the panels relative to the camera. The relative locations of the panel and ultrasound probe with respect to the computer vision camera can then be combined to determine the relative positions of the gamma camera panel and ultrasound probe with respect to each other. This can allow for co-registration of images generated by the gamma camera and ultrasound probe. Additionally, the memory may have instructions for execution by the processor to use the computer vision camera data to determine the contours of the patient's body. This can be used to further determine the relative positions of the gamma camera panel and ultrasound probe with respect to the patient's body.

[0117] Alternatively, or in addition, a computer (e.g., controller (100) shown in FIG. 1) may be operatively coupled to a sensor connected to lever (301) to receive position information regarding the articulation of the mechanical components of lever (301) and infer the position and location of gamma camera panels (300a) and (300b) relative to computer vision camera (305).

[0118] FIG. 4 shows an illustration of the system shown in FIG. 3, but with the ultrasound probe placed outside the space between the two panels. This modality may be useful for scanning organs inside the thorax, such as the heart and liver. One of the gamma camera panels (400) is shown mechanically connected to the distal end of the mechanical joint arm (403) by a lever (401). The system is shown scanning a patient (404). In this embodiment, an ultrasound imaging probe (405) is used to scan the patient from a location adjacent to, but outside, the gamma camera panel. The image field of view of the ultrasound imaging probe (405) may or may not partially or completely overlap the image field of either or both panels. A computer vision camera (406) is connected to the distal end (402) through a connector (407). The computer vision camera and processing of the data for tracking and co-registration may occur in a similar manner as described in FIG. 3. A reference object (408) is shown attached to the ultrasound probe to aid in ultrasound probe tracking.

[0119] In another embodiment, a jointed mechanical arm can be used to track the position and orientation of the ultrasound sensor relative to the gamma camera panel. FIG. 5 depicts an ultrasound probe that is co-registered to a gamma camera panel through the use of a jointed arm with coordinate measurement capabilities. Two gamma camera panels (500a) and (500b) are shown scanning a patient (501). The panels are mechanically connected to the distal end of a mechanical jointed arm (503) by a lever (502). In this embodiment, an ultrasound imaging probe (504) is introduced into the space between the two panels (500a) and (500b). The image field of view of the ultrasound imaging probe (504) may partially or completely overlap the image field of view of either or both panels (500a) and (500b). A jointed arm (505) with coordinate measurement capabilities may be mounted on the jointed arm assembly (502)-(503). This arm (505) may have its distal end rigidly affixed to the ultrasound probe.

[0120] A tracking program stored in the memory of the computer, operatively coupled to a sensor connected to the auxiliary arm (505), may be configured to receive position information regarding the articulation of the mechanical components of the arm (505) and infer the position and location of the ultrasound probe relative to the lever (502). In addition, the computer may be coupled to the sensor connected to the lever (502) to receive position information regarding the articulation of the mechanical components of the lever (502) and infer the position and location of the gamma camera panel relative to the lever (502). The tracking program may combine the tracking information regarding the ultrasound and gamma camera panels and determine their relative positions.

[0121] The use of an articulated auxiliary arm (505) may be used in addition to the camera (106) of Figure 1. Thus, a tracking algorithm on the computer may combine various tracking modalities and determine the location of the gamma camera panel, ultrasound probe, and patient body relative to each other.

[0122] In yet another embodiment, other tracking sensors can be used to track the position and orientation of the ultrasound sensor relative to the gamma camera panel. An example can be an electromagnetic field tracker. FIG. 6 depicts an ultrasound probe that is co-registered to a gamma camera panel through the use of a magnetic tracking system. Two gamma camera panels (600a) and (600b) are shown scanning a patient (601). The panels are mechanically connected to the distal end of a mechanical joint arm (603) by a lever (602). In this embodiment, an ultrasound imaging probe (604) is introduced into the space between the two panels (600a) and (600b). The image field of view of the ultrasound imaging probe (604) can partially or completely overlap on the image field of view of either or both panels (600a) and (600b). An electromagnetic transmitter (605) may be mounted in close proximity to the ultrasound probe, in this case mounted on the joint arm assembly (602)-(603). An electromagnetic receiver (606) may be rigidly affixed onto the ultrasound probe. A tracking program stored in the memory of a computer operatively coupled to the transmitter (605) and receiver (606) may be used to infer the position and location of the ultrasound probe relative to the transmitter (605). In addition, other electromagnetic receivers (not shown) may be rigidly affixed onto the gamma camera panels (600a) and (600b) and operatively coupled to the computer. The tracking program may use data from the receivers and transmitters to determine the locations of the gamma camera panels and the ultrasound probe relative to each other.

[0123] In some embodiments, the transmitter and receiver may be interchanged. In other embodiments, the transmitter may be affixed to another object and component (605) may be the receiver. In yet other embodiments, an electromagnetic unit affixed to the instrument may act as both the transmitter and receiver.

[0124] As an alternative or in addition to affixing a receiver on the gamma camera panel, the computer may be operatively coupled to a sensor connected to lever (602) to receive position information regarding the articulation of the mechanical components of lever (602) and infer the position and location of gamma camera panels (600a) and (600b) relative to transmitter (605). A tracking program may combine the tracking information regarding the ultrasound and gamma camera panels and determine their relative positions.

[0125] A tracking system based on electromagnetic transmitters and receivers may be used in addition to the camera (106) of FIG. 1. Thus, a tracking algorithm on the computer may combine the various tracking modalities and determine the location of the gamma camera panel, ultrasound, and the patient body relative to each other. An example of a tracking system is one that uses an external infrared stereo tracker combined with infrared reflective spheres that are mounted in a unique pattern on the various components that require tracking. Any combination of the tracking and co-registration techniques presented here, as well as other tracking systems, may be used. For example, the tracking system may be an optical tracking system, an electromechanical tracking system, an electromagnetic tracking system, an ultrasound tracking system, a depth image tracking system, or a combination thereof.

[0126] FIG. 7 shows a depiction of an embodiment of portable SPECT gamma camera panels (700a) and (700b) with fields of view (701a) and (701b), respectively, used to observe a patient (702). An ultrasound probe (703) with a field of view (704) is co-registered with the panels (700a) and (700b) through the use of any combination of tracking systems. In this depiction, a tracking sensor (705), such as a magnetic field receiver, is shown attached to the ultrasound probe (703). As a result of the use of image reconstruction algorithms on the data received from the gamma camera, two SPECT image features (706) and (707) with radioactive uptake may be reconstructed inside the patient in 3D. SPECT image features (706) and (707) may be constructed in "real time", i.e., ideally at a finite frame rate of better than one frame over a few seconds, or they can be constructed by acquiring data over a longer period of time with gamma camera panels scanning the patient, either from a static position, or dynamically by moving them around the patient's body. In current cases, features (706) and (707) may not correspond to their actual location inside the patient's body in the presence of tissue deformation, patient motion, or organ movement.

[0127] The ultrasound probe (703), guided and co-registered by the SPECT image, may be brought into close proximity with the gamma camera panel so that its field of view intersects a portion of the SPECT feature (706). In a first step, a computer and visualization device operatively connected to the ultrasound probe may generate an ultrasound image (708) delivered by the ultrasound probe (703) and augment it with a SPECT image feature (709) that represents a section through the SPECT 3D map in the ultrasound field of view (704). In a second step, the same computer may identify and correct possible rigid body transformations between the SPECT and ultrasound images due to the patient, furniture, or other sources of motion. The rigid body transformations are calculated by snapping the SPECT image features onto the ultrasound image features. The snapping process consists of (1) automatically identifying visual features in both images, (2) matching the SPECT image features to the ultrasound features, and (3) calculating a rigid body transformation (projection) based on the matched features. In other words, the system may generate a model of the movement of features in the ultrasound image from a first ultrasound frame to a second ultrasound frame and generate a modified SPECT image based on the model of the movement of features in the ultrasound image. The resulting enhanced image allows a user to identify the patient's anatomy surrounding the SPECT feature (706), similar to what a CT scan would provide in a SPECT / CT imaging system.

[0128] Additionally, the ultrasound image (708) may be used to guide an interventional medical instrument, such as a percutaneous biopsy needle (710) or an ablation therapy device, towards a target highlighted by the SPECT image. In some embodiments, the medical instrument may also be tracked to allow for co-registration between the medical instrument, the ultrasound, and the SPECT image. In FIG. 7, a tracking sensor (711), such as a magnetic field receiver, is shown to be used for co-registration purposes. Alternatively, or in addition, a mechanical instrument guide may be used to define the movement of the medical instrument. Using either method, the projected trajectory of the instrument, in this depiction the projected trajectory of the needle (712), may be enhanced on the ultrasound image (708).

[0129] In some embodiments, a head mounted display can be used to visualize ultrasound images (708) and / or SPECT image features (706) and (707) augmented on the user's natural field of vision with the ability to co-align between the user's eyes to a co-registered coordinate system associated with the ultrasound and SPECT images by using a head mounted display system worn by the user.

[0130] In some embodiments, the entire medical intervention process may be automated, where the ultrasound (703) and the interventional device (710) are controlled by a computer through the use of a mechanical joint arm.

[0131] In some clinical applications, it may be relevant to generate molecular SPECT images of organs that move or deform easily. In such cases, ultrasound images may be used to guide corrections in the SPECT images based on tissue deformation and movement observed in the ultrasound images. FIG. 8 shows a depiction of a method by which such corrections may be implemented. In the depicted embodiment of portable SPECT, gamma camera panels (800a) and (800b), with fields of view (801a) and (801b), respectively, observe a patient (802). An ultrasound probe (803) with a field of view (804) is co-registered to the panels (800a) and (800b) through the use of any combination of tracking systems. An ultrasound image feature (805) that may appear in one ultrasound scan may be deformed and displaced in a subsequent ultrasound scan and may appear as an ultrasound image feature (806). If there is an area of ​​increased radiotracer uptake, or SPECT image feature (807), at the time of the first ultrasound scan, the SPECT image feature may be located in a different location at the time of the second ultrasound scan. Without tissue deformation correction, the SPECT image reconstruction algorithm would not be able to generate the SPECT feature with the correct range because the acquisition time would extend over a time consisting of the time of two ultrasound scans. In such a case, ultrasound can be used during the SPECT scan to monitor the movement of the features visible in the ultrasound image.

[0132] A sequence of ultrasound images can be automatically analyzed by image analysis algorithms to determine the fields of organ motion and deformation that occur from one ultrasound image scan to another. This can be accomplished by automatically identifying ultrasound image structures (such as organs), generating parameterizations of those structures, and tracking their motion and deformation in time. Ultrasound image structures can be defined by geometric primitives (from simplest to most complex structures): points, lines, rectangles, and circles. These geometric primitives may be parameterized using patches (points with radial extents) for point-like structures, and contours for all other primitives. The contour parameterization depends on the underlying ultrasound image structure, with lines represented as curves, circles as ovals, and rectangles as polygons. Image structures identified in successive ultrasound frames are matched. Each match pair is used to quantify the motion and deformation undergone by the organ (or structure). The resulting motion field and deformations are used to remap image space elements from one frame to the next, and the SPECT image reconstruction will use the remapped image space elements to construct a SPECT image that is corrected for tissue deformation. When the organ motion is cyclical, such as in the case of cardiac motion, multiple SPECT images can be generated for each sequence within a period of cyclic motion. In some embodiments, the ultrasound probe can provide 3D ultrasound images. This will generate a "wedge-shaped" 3D image volume that will better capture more complex organ motions and potentially lead to better organ deformation correction.

[0133] If a more complete SPECT image is available, such as from a frontal SPECT scan, such a SPECT image can be used as a premise to the reconstruction of the current real-time SPECT image. For example, an algorithm can run on an operably connected computer to update the SPECT image and generate a real-time SPECT image by comparing the latest gamma-ray detected event with an estimate of the event. The estimate of the event can be calculated by the computer by computationally projecting the previous SPECT 3D map forward into the sensor. The calculated projection can take into account the latest configuration of the gamma camera, including the sensor and mask. The computer can calculate the deviation between the detected event and the estimated event and determine the deformation in the previous SPECT 3D map that is consistent with the latest detected event. An example of an algorithm that can be used to update in the real-time molecular image is described in Lu, Y., et al. (2019). Data-driven voluntary body motion detection and non-rigid event-by-event correction for static and dynamic PET. Physics in Medicine & Biology, 64(6), 065002.

[0134] In some embodiments, the computation of real-time SPECT images may use tracked deformable ultrasound features from images taken with an ultrasound probe co-registered with the SPECT sensor panel, the parameterization of which is described above. Examples of feature extraction methods are described in Revell, J., et al. (2002) “Applied review of ultrasound image feature extraction methods” In The 6th Medical Image Understanding and Analysis Conference (pp. 173-176). BMVA Press; Aleman-Flores, M., et al. (2005, February) “Semiautomatic snake-based segmentation of solid breast nodules on ultrasonography” In International Conference on Computer Aided Systems Theory (pp. 467-472) Springer, Berlin, Heidelberg; Zhou, S., Shi, J., Zhu, J., Cai, Y.,& Wang, R. (2013) “Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image” Biomedical Signal Processing and Control, 8(6), 688-696.These parameterized ultrasound features can be tracked by the computer to obtain tracked deformable ultrasound features by using a method as described in Yeung, F., et al. (1998). Feature-adaptive motion tracking of ultrasound image sequences using a deformable mesh. IEEE transactions on medical imaging, 17(6), 945-956. The tracked deformable ultrasound features can be used by the computer in the calculation of an updated real-time SPECT image. This can be done, for example, by using the tracked parameterized ultrasound features to constrain the solution for the SPECT image deformation model. In some embodiments, the computer can generate this SPECT image deformation model and obtain an updated SPECT image by using a previously collected SPECT 3D map. In this case, the SPECT image elements are essentially pinned on the tracked deformable ultrasound features and moved with the ultrasound features. In some other embodiments, the computer can generate a SPECT image deformation model and obtain an updated SPECT image by using real-time gamma data in combination with a previously collected SPECT 3D map. In this case, the tracked deformable ultrasound features are used to constrain a deformation model that is calculated by comparing the most recent gamma ray detection events to an estimate of the events calculated by a computer from a previously collected SPECT 3D map, as described in the previous paragraph. The computer may use a deformable model data fusion filter to combine the ultrasound deformable model with a deformable model that compares the real-time gamma ray data to the previous SPECT map. This deformable model data fusion filter may use a Kalman filter (Welch, G., & Bishop, G. (1995) "An introduction to the Kalman filter").The parameters of the filter, which determine how the filter is applied to an application, may be modified by the computer or by the user and may take into account the quality of the ultrasound image, the quality of the extracted ultrasound features, the quality of the ultrasound deformable tracking model, the gamma count rate, an estimate of the gamma count rate extracted from a forward projection of a previously reconstructed SPECT 3D map, the signal-to-noise ratio of the SPECT image, etc. For example, a filter running on the computer may give more weight to tracked ultrasound deformable structures in the construction of an updated SPECT image when the confidence of the ultrasound tracked structures is high and the detected gamma count rate is too low, and vice versa.

[0135] These systems and methods can also be employed in extended volume areas inside the patient that may not be directly visible in the ultrasound scan by propagating the deformations tracked in the ultrasound scan away from the ultrasound image volume by using the specific mechanical and elastic properties of tissue deformations. These deformation propagation methods can also use a 3D model of the patient as tracked by a computer vision system. For example, this 3D model of the patient can provide boundary conditions for deformations away from the scanned ultrasound volume.

[0136] 9 shows a depiction of an embodiment of portable SPECT gamma camera panels (900a) and (900b) mounted to a jointed arm (902) through a jointed lever (901). In this depiction, the panels are positioned to scan a particular body part of interest, such as a human breast (903). In this case, the panels are approximately parallel to each other and the fields of view of the gamma cameras very largely overlap. In other embodiments, the gamma cameras may be mounted at other angles relative to each other, such as 90 degrees from each other.

[0137] In some embodiments, the gamma camera panels (900a) and (900b) are equipped with gamma sensors that provide sensor data containing information used by an image reconstruction program running on a computer to provide an image of the radiotracer distribution with a finite resolution along the direction to which either gamma camera is most sensitive when the gamma camera panels are in an essentially static location relative to the patient's body part (903). In some embodiments, the finite resolution is less than 20 mm for a range of distances covering at least 50 mm. In such an embodiment, lesions (904) inside the patient's body with increased radiotracer uptake may be imaged in 3D. Compared to a planar image setting, a 3D image of the radiotracer distribution will provide better lesion detectability and localization.

[0138] In some embodiments, the panels (900a) and (900b) may comprise tactile sensors (905a) and (905b), preferably located on the panel surface facing the patient's body part (903), to obtain information about the pressure applied by the panel on the patient's body or on other objects by the panel. These tactile sensors may be operatively coupled to a computer, which is also connected to an actuator that moves the panel relative to the patient. The computer may use a program to vary the position of the panel to keep the pressure applied by the panel on the patient or other object within a desired range of values. The tactile sensors may be capacitive, resistive, piezoelectric, or any other sensor capable of providing tactile pressure between objects.

[0139] While in this configuration, a co-registered ultrasound probe and percutaneous medical device may be used on the examined patient body part, similar to the embodiment illustrated in FIG.

[0140] FIG. 10 shows a depiction of an embodiment in which a portable cart (1000) integrates a SPECT gamma and medical ultrasound system. A six-axis robotic arm (1001) is affixed onto the portable platform (1000). A mating gamma camera sensor mounting assembly (1002) is affixed onto the distal end of the robotic arm (1001). Gamma camera panels (1003a) and (1003b) as described above are mounted onto the mounting assembly (1002). A computer vision camera (1004) as described above is mounted onto the mounting assembly (1002). Ultrasound probes (1005) and (1006) are operatively connected to the ultrasound electronics and controller through a connector panel (1007). In a preferred embodiment, a console (1008) is used by a user to control the mechanical system including the ultrasound system, the SPECT camera system, and the robotic arm. A monitor (1009) is depicted here as a visualization device. Other visualization devices, such as head-mounted displays co-registered with the patient, may also be used. The cart may enclose any of the following components: a master controller (computer) including a graphical processing unit (GPU), a controller for a mechanical subsystem including a 6-axis robotic arm, electronics for ultrasound pulse formation and readout, and electronics for gamma camera readout. SPECT and ultrasound co-registered images may be delivered to the visualization device by the master computer. Other medical sensors or other medical imaging devices may also be used instead or in addition to the ultrasound probe. Examples are a fluorescence imaging probe, an optical coherence tomography probe, a computer vision camera, an infrared camera, an impedance sensor, etc.

[0141] FIG. 11 shows a depiction of the operational connections between the components mentioned in the description of FIG. 10. The enclosure of the portable platform (1000) is represented by (1100). The enclosure comprises at least a central computer controller (1101) that is used to read data from the sensors and other subsystems. This computer integrates the data to determine tracking information about objects of interest, reconstructs SPECT images and potentially other medical images from other sensors, generates co-registered images, enhances images on top of each other, transmits visualization data to visualization devices, and controls other controllers, subsystems, and electronics. The enclosure may also comprise electronics (1102) for reading and controlling the gamma cameras (1103a) and (1103b). The enclosure may also include a mechanical controller (1104) that receives sensed mechanical information and controls the articulated arm (1001) and potentially other actuators attached to the wheels of the platform (1100), to the patient bed or seat, to the gamma camera mounting assembly (1002), or to other objects, medical devices, or sensor assemblies that may be used during a scan or intervention. The enclosure may also include electronics (1105) that provide pulse shaping for and read out signals from the ultrasound transducer (1106). When other imaging and sensing modalities may be used, their control and readout electronics may also be housed inside the platform enclosure. Various ports would be provided for all such imaging probes and sensors, for example on a connector panel (1007). The central computer controller may also control and read out tracking devices, computer vision systems, etc. As an example, the illustration shows a computer vision camera (1107) read out and controlled by a central computer controller (1101).The visualization output generated by the central computer can be transmitted to a visualization device such as a monitor (1108), or over a network to other computers, or to a head mounted display.

[0142] In some embodiments, an array of ultrasound transducers, aligned to each other and to other sensors such as gamma camera panels, may be used instead of a single ultrasound transducer probe. Such an array may extend a volumetric area that may be imaged by a co-registered ultrasound imaging system. FIG. 12 shows a depiction of such a system. Gamma camera panels (1200a) and (1200b) are shown scanning a patient (1201). A flexible band (1202), conforming to the contours of the patient's body, comprises at least ultrasound transducers, three of which in this depiction (1203a-c). These ultrasound transducers image the patient (1201) by contacting the patient's body with ultrasound waves. Their positions and orientations relative to each other and to the gamma camera panels (1200a) and (1200b) are tracked by using tracking sensors (1204a-c). Such tracking sensors may be electromagnetic sensors, optical fiducial markers identifiable by optical systems, ultrasonic fiducial markers or sensors, infrared reflecting markers, active optical emitting markers, or any other components that can be used to track the position and orientation of ultrasonic transducers relative to each other. These ultrasonic transducers can be used separately or in combination to insonify the patient's body and sense the reflected ultrasonic waves to obtain a significant ultrasonic 3D field of view. Such a wide 3D field of view may significantly overlap with the gamma camera field of view, which will allow for much more precise tissue deformation correction, as illustrated in FIG. 8.

[0143] Furthermore, during reconstruction of the SPECT image, the computer can analyze the ultrasound image associated with each gamma camera detection and calculate the attenuation probability through tissue of the detected gamma rays. This calculation requires knowledge of a detailed map of tissue attenuation coefficients. Because ultrasound images do not directly provide information about tissue type, automated ultrasound tissue characterization modeling, such as using machine learning modeling methods, can be applied on the ultrasound image data set to extract a map of gamma attenuation factors. Tissue types such as water, fat, muscle, bone, or air-filled areas can be extracted. Standard gamma attenuation coefficients associated with each of these components can be used to build a gamma attenuation map inside the patient's body. By using regularly updated ultrasound images, this attenuation map can be regularly remapped as tissue and organ deformations can occur. Because ultrasound signals do not propagate well in bone and air, a map of these components inside the patient can be extracted by tracing adjacent tissues that are well visible in the ultrasound image. Other anatomical assumptions can also be used to aid in the tissue characterization process and the mapping of gamma attenuation coefficients. An example of the previous map is a CT image that robustly characterizes gamma ray attenuation features that are both fully visible and poorly visible to ultrasound.

[0144] The embodiment described in Figure 12 can also be used for precise intervention guidance: a wide 3D ultrasound field of view combined with SPECT images, corrected for tissue deformation and for gamma attenuation coefficient variations inside the patient's body, can produce a highly accurate guidance system for percutaneous biopsies or other interventions such as ablation treatments.

[0145] In some embodiments, the ultrasound transducers (1203a-c) may be used to deliver high intensity focused ultrasound beams to a region of interest as highlighted by SPECT and / or ultrasound images. Such beams may be used for a variety of purposes, such as to ablate tumors, to allow better pharmaceutical drug penetration within the region of interest, and to create other physicochemical changes within the region of interest.

[0146] In some embodiments, the ultrasound transducers may not reside within the same flexible band (1202) and they may be affixed to the patient independently of each other, however, they may still be tracked relative to each other and other reference points and used in combination to generate a wide 3D ultrasound image within the area of ​​interest.

[0147] In some embodiments, structures comprising ultrasound transducers, such as flexible bands (1202), may have an attachment mechanism that keeps the transducers (1203a-c) fully connected to the patient's body without user intervention. For example, such an attachment mechanism may be a mechanical arm or adhesive tape.

[0148] For some applications, it may be beneficial to co-register the SPECT image with other imaging modalities such as MRI or CT. FIG. 13 shows a depiction of a portable SPECT system used in combination with a separate medical imaging system. In some implementations, the portable SPECT system (1301) may be moved in close proximity with another medical imaging instrument (1302), such as a CT, MRI, or magnetic imaging system. This would allow the portable SPECT system (1301) to scan the patient (1303) simultaneously with, or immediately before or after, imaging performed with the system (1302). The portable imaging system (1301), which may be SPECT or another imaging modality, may include an image sensor panel (1304) mounted at the distal end of an articulating arm (1305). In a preferred embodiment, the articulating arm is a 6 or 7 degree of freedom (DOF) robotic arm. The scanning assembly includes a computer vision camera system (1306). The computer vision camera system is co-registered to the sensor panel by using the computer vision system itself, by mechanical co-registration, or by using other trackers. The computer vision camera can be located in a single housing, or its components and sensors can be distributed among several housings and parts of the scanning system (1301). In some embodiments, the computer vision system components can be mounted inside the sensor panel (1304). In a preferred implementation, a portion of the computer vision camera system (1306) is mounted at the distal end of an articulating arm that has a field of view that covers a portion of the patient. In other implementations, a portion of the computer vision camera system may be mounted elsewhere and may only have a field of view that covers a portion of the patient.The computer vision system (1306) may be understood as any system capable of generating images, streams of images, ranging data, and streams of ranging data that are analyzed by an operatively connected computer, and may comprise an RGB camera, an infrared camera, a depth imaging optical camera, a stereo optical camera, a time-of-flight optical camera, a terahertz emitter-sensor assembly, an ultrasonic emitter-sensor assembly, another tracking and / or mapping sensor, or combinations thereof.

[0149] As part of the process, the patient (1303) is scanned by the scanner (1302) by lying on the scanning table (1306). The table may insert the patient into the scanner and the scanning will occur. The image reconstructed by the scanner (1302) will be read by a computer operatively coupled to the scanner (1301). As another part of the process, the scanner assembly (1301) scans the patient (1303) and generates another image data set. This scan may occur in parallel with, before, or after the scan taken by the scanner (1302). The computer operatively coupled to the scanner (1301) can analyze the reconstructed image from the scanner (1302) and identify structures that are used for co-registration between the two image data sets generated by the scanners (1301) and (1302).

[0150] In some implementations, tags (1308) and (1309) may be placed adjacent to the patient prior to scanning the patient, either by placing tag (1308) on a table or by placing tag (1309) on the patient's body. These tags may contain features identifiable by both scanner (1302) and system (1301). An embodiment for these tags is depicted in FIG. 14. Structures that are part of tags (1308) and (1309) and that are imaged by scanner (1302) will be matched to structures in computer memory to determine a coordinate system associated with the image data set delivered by system (1302). Structures that are part of tags (1308) and (1309) and that may be imaged by computer vision camera (1306) will be analyzed by a computer operatively connected to system (1301) and matched to structures in computer memory to determine a transformation between the tag coordinate system and the camera. Additionally, using the known transformation between the camera and sensor panel (1304), an operably coupled computer can determine a transformation between a coordinate system associated with an image data set delivered by system (1302) and a coordinate system associated with an image data set delivered by system (1301). Once the transformation is known, the data delivered by system (1302) can be used by system (1301) to calculate an attenuation map and better reconstruct the location of a target within the body using techniques known in the art (e.g., SPECT-CT).

[0151] A distinctly novel application of the transformation communicated herein arises through the calculation of features common to the system (1302) and the data delivered by the system (1301) when operatively coupled to an ultrasound transducer (802). In this embodiment, the system (1301) calculates a modified attenuation model to account for various tissue deformations as described above.

[0152] In some implementations, the patient may expose skin over the part of the body that is primarily scanned by the scanner (1302). In addition, in some implementations, the patient may wear clothing that fits tightly to the body, as depicted in FIG. 15. This would allow a computer vision system based on optical sensors or based on sensors that detect signals that do not penetrate the fabric to generate an accurate 3D model of the patient's body contours. In some other implementations, the patient may wear regular loose-fitting hospital clothing. In such implementations, the computer vision system may include a clothing through-scanning system, such as an ultrasound scanning system, a tetrahertz scanning system, or a low-dose soft x-ray backscatter system, to generate a 3D model of the patient's body contours.

[0153] When the data from the computer vision system (1306) can be used by a computer operably coupled to generate a 3D model of the patient's body, the scanner (1302) can preferably include distinguishing between the patient's body and air and deliver a map of the anatomical structures. In this case, the computer operably connected to the scanner (1301) can then analyze the structures in the image data set provided by the scanner (1302) and associated with the contours of the patient's body and match them to the 3D model of the patient generated by the computer vision camera (1306). This will enable the computer to generate a co-registration transformation between the two image data sets and to map the image data generated by the scanner (1302), including the anatomical image data, into a frame of reference associated with the image data generated by the scanner (1301) and generate a co-registered anatomical map. The co-registration transformation can be the result of an iterative algorithm that searches for the best transformation applied to one of the models that minimizes the distance error between the two models. It is assumed that when the distance between two 3D models is minimal, the overlap is maximal and the models are aligned. An example of such an algorithm is the Iterative Closest Point (ICP) or Generalized ICP. ICP is described in Chen, Y., & Medioni, GG (1992). Object modeling by registration of multiple range images. Image Vision Comput., 10(3), 145-155. Generalized-ICP is described in Segal, A., Haehnel, D., & Thrun, S. (2009, June). Generalized-icp. In Robotics: science and systems (Vol. 2, No. 4, p. 435).

[0154] In some cases, systematic deviations may exist between the two 3D models of the patient's body. Some of these deviations may occur due to some patient body movements or body deformations that may occur between the moment the 3D scan is performed with the computer vision system (1306) and the moment the image scan is performed with the scanner (1302). In such cases, non-rigid 3D matching algorithms can be used to remap the image dataset generated by the scanner (1302), including the anatomical image dataset, to the 3D model delivered by the computer vision system (1306) and generate a co-registered deformable anatomical map. The algorithms may calculate a co-registration transformation between the two 3D models and may map the 3D model deformations. This is achieved by allowing different parts of the 3D models to move non-rigidly while trying to minimize distance errors. The algorithms ensure smooth motion between the model parts and are as rigid as possible deformations. An example of such an algorithm is dynamic fusion as described in Newcombe, RA, Fox, D., & Seitz, SM (2015). Dynamic Fusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 343-352).

[0155] The co-registered rigid or deformable anatomical map can be used by an operably coupled computer in the process of reconstructing images generated by the scanner (1301). For example, the co-registered anatomical map as delivered by the CT scanner (1302) can be used to generate a gamma ray photon attenuation map that can be further used by the operably coupled computer to improve the quality of the reconstructed images generated by the SPECT or PET scanner (1301). For example, the attenuation map is used to quantify the propagation or absorption probability or transmission of photons from specific image elements, such as voxels or grid points, to sensors in the panel (1304). Such attenuation correction processes are currently implemented in integrated SPECT / CT systems. With respect to the co-registered deformable anatomical map, it is particularly useful to model the deformation of low density volumes, such as bones and lungs, for accurate photon attenuation correction in SPECT or PET image reconstruction.

[0156] In some implementations, the anatomical map may be performed in a separate scanning session. In such cases, the patient may have moved significantly between the anatomical scanning session, such as performed by a CT or magnetic resonance imaging (MRI) scanner (1302), and the molecular image scanning session, such as performed with the portable system (1301). In this case, the deviation between the 3D body model extracted from the anatomical data and the 3D body model delivered by the computer vision system (1306) may be too large to generate an accurate co-registered deformable anatomical map. In this case, an ultrasound probe co-registered with the computer vision system as described above can be used to scan the patient at one or more locations and transmit ultrasound images of the patient to an operably coupled computer. The structures extracted by the computer from these ultrasound scans can then be matched to structures extracted by the computer from the anatomical map delivered by the scanner (1302) to generate co-registration pinning points. These co-registration pinning points will constrain the solution for the deformable co-registration of the anatomical map onto the molecular map. The co-registered deformable anatomical map can be used for attenuation correction in the molecular map reconstruction process, as described above.

[0157] The co-registered deformable anatomical map can also be used by an operably coupled computer to transmit to a visualization device a rendering of a molecular map provided by the scanner (1301) that is combined with a rendering of a co-registered anatomical map provided by the scanner (1302). The computer may also transmit to a visualization device a rendering of an ultrasound image delivered by the co-registered ultrasound probe that is combined with a rendering of an anatomical map delivered by the scanner (1302). This process can be performed when the scanning sessions performed by the scanners (1301) and (1302) are performed jointly or separately.

[0158] In some implementations, it may be useful to navigate co-registered anatomical, ultrasound, molecular image models, or a combination thereof, that are augmented onto a live red-green-blue (RGB) image of the patient captured by a camera, or onto the user's field of view by using a head-mounted display. In that case, a handheld stylus or probe tracked by a computer vision system can be used by the user to select image planes of interest and show a rendering of those image datasets in the plane selected with the stylus or probe.

[0159] In some other implementations, the computer vision system (1306) may determine its position relative to the scanner (1302) by analyzing geometric features on the scanner (1302) or by analyzing a tag attached to the scanner (1302). This would allow a computer operatively connected to the computer vision camera (1306) to determine the location and orientation of the panel (1304) relative to the scanner (1302). This in turn would allow the computer to perform co-registration between the image data sets provided by the scanner (1301) and the scanner (1302).

[0160] In some implementations, the scanner (1301) may scan the patient as the table moves the patient through the scanner (1302), in which case the computer adjusts the scanning protocol in real time to account for patient and scanning table movements as provided by the computer vision system.

[0161] FIG. 14 shows a depiction of a fiducial used to provide co-registration between a portable imaging system, such as illustrated by (1301) in FIG. 13, and another medical imaging scanner, such as illustrated by (1302) in FIG. 13. In an embodiment, the fiducial, whose outline is represented by (1401), may comprise a structure (1402) that is identifiable in an image map delivered by the scanner (1302). For example, if the scanner is a CT scanner, the structure (1402) may be made of a material that attenuates X-rays, such as steel, tungsten, or another high-density material. The fiducial (1401) may also comprise features (not depicted) that can be identified by a computer vision system (1306), such as a binary black and white tag. In the illustration, a ring of known diameter may support several protruding geometries (substructures) (1403) of various dimensions. These substructures would break the symmetry of the fiducial allowing unambiguous identification of the fiducial position and orientation, and allow its use with instruments of various imaging capabilities, such as resolution and contrast. Thus, larger structures will be visible in the maps delivered by most scanners, while smaller structures will add the added benefit of better co-registration with maps delivered by scanners that offer better image performance such as resolution and contrast. The shape is of known size and location relative to the center of the ring. In the illustration, the shape is a sphere, but in other embodiments it may be changed to, for example, a pyramid. Upon analysis of the data from the target modality, the computer can calculate the location of the center of the ring by observing the location of the material as well as the orientation of the structure (1402). The fiducial (1401) is constructed using a known transformation between the sides that defines the location inferred by the targeted modality through the use of computer vision techniques and the asymmetric structure (1402).

[0162] The position and orientation of the endoscope and the camera guided inside them can be inferred using fluoroscopy and techniques established in the art. Using fiducials (1401) to co-register the SPECT system with the coordinate frame of the fluoroscopy system, the same computation unit can provide the operator with an augmented image of the internal area of ​​the body as viewed by the camera inserted by the endoscope. The augmentation can guide the operator to objects within the body identified by the SPECT imaging system (1301). To achieve this, the computation unit can be assured that it is informed of the position and orientation of the camera within (or attached to) the endoscope. The tracked position of the endoscope can be interpreted by the same computation unit using techniques known in the art. If the endoscope tracking system is also articulated to view the co-registered fiducials (1401), the computation unit can also calculate the position and orientation of the camera attached to or within the endoscope relative to the coordinate frame selected by the SPECT system (1301). This is achieved by calculating the relative transformation between the endoscope fiducial structure and the co-registered fiducials (1401). This sequence of transformations allows the computation unit to overlay a reconstructed SPECT image onto that acquired by the endoscopic camera. This information is particularly useful for guiding interventions such as lung biopsies, where ultrasound cannot provide real-time guidance.

[0163] The endoscopes described above may contain instruments and tools capable of ablation, drilling, cutting, puncturing, debridement, or accessing targets identified by the SPECT imaging system. These tools can be articulated by the operator and monitored by visualization provided by the endoscopic camera augmented by a computing unit informed of the SPECT targets as described above.

[0164] Data from a camera or other computer vision system inserted in the endoscope can also be used to observe and measure tissue deformations throughout the computation process. With the described co-registration, which joins the computer vision system in the endoscope with the location of the SPECT system, these tissue deformations can also inform the SPECT reconstruction by applying the inferred deformation transformation to the attenuation map. Updating the attenuation map is important when computing real or near real-time SPECT images. Similarly, tissue deformations inferred by the tracked sensors in the endoscope can be used to compute accurate updates to the visualization of SPECT targets previously reconstructed in the body as described above. These updates can be presented as overlays or augmentations to images captured by sensors on or in the endoscope. These updates can also be presented by the SPECT visualization monitor (1009) to guide interventions visualized through other co-registered sensors.

[0165] FIG. 15 shows a depiction of a form-fitting garment that may be used by a patient to assist the optical-based computer vision tracking and mapping process. The patient, who may be either female (1501a) or male (1501b), may wear garments (1502a) and (1502b) that loosely follow the contours of the body. For example, the garment may be made from a stretchy fabric material. The garment may be made from a material that, especially in combination with an ultrasound conducting gel, allows ultrasound waves to pass essentially undisturbed through the garment. The garment material may be chosen such that when combined with an ultrasound conducting gel, the garment will provide reduced acoustic impedance and reflection for typical medical ultrasound frequencies. The garment material may include through holes that allow the ultrasound gel to pass through them and reach the patient's skin after applying the ultrasound gel on the garment. This may allow for excellent transmission of ultrasound waves.

[0166] In some implementations, the garments may cover only the upper body, such as the torso. In other embodiments, they may cover only the legs, buttocks, and portions of the waist, similar to swim trunks. In some other implementations, the garments may cover both the buttocks, waist, and torso (as shown). The garments may or may not cover significant portions of the legs and arms. The garments may be put on and taken off, similar to a swimsuit. In some implementations, the garments may be closed by binders (1503a) and (1503b). These binders may be selected from a group containing zippers, buttons, adhesive bands, hook-and-loop fasteners, or combinations thereof. Adhesive tapes (1504a) and (1504b) may be used to ensure that the garment follows the patient's skin where it may be lifted from the skin. Such adhesive tapes may be part of the garment or may be applied separately. In some implementations, the adhesive tape has a double-sided adhesive. The double-sided adhesive tape may be mounted on the skin at a specific location before donning the garment and pressing the garment over the tape to ensure its proximity to the skin.

[0167] In some embodiments, these garments may include prints or patterns. Such geometric features may be recognized and tracked by a computer vision analysis program in a computer operatively coupled to the computer vision system to generate a 3D model of the patient by tracking the print or pattern features. For that purpose, structure from motion algorithms (as described in Fuhrmann, S., Langguth, F., & Goesele, M, Mve-a multi-view reconstruction environment, in GCH pp. 11-18 (2014, October); Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T., Demon: Depth and motion network for learning monocular stereo, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5038-5047 (2017)) may be used in conjunction with a monocular camera. Structure matching algorithms may be used in conjunction with a stereoscopic camera.Structural matching algorithms are described in Hirschmuller, H, Stereo processing by semiglobal matching and mutual information, IEEE Transactions on pattern analysis and machine intelligence, _30_(2), 328-341 (2007); Sun, J., Zheng, NN, & Shum, HY, Stereo matching using belief propagation, IEEE Transactions on pattern analysis and machine intelligence, _25_(7), 787-800 (2003); Geiger, A., Roser, M., & Urtasun, R., Efficient large-scale stereo matching, in Asian conference on computer vision (pp. 25-38). Springer, Berlin, Heidelberg (2010, November).

[0168] These garments can be used to allow a computer vision system to generate an accurate 3D map of the patient's contours without requiring the patient to expose coverings over large areas of the body, allowing the patient to remain warm and comfortable. Another benefit of these garments is that they keep areas of the patient's body compressed that might otherwise be loose. Another benefit of these garments is that they allow convenient ultrasound examinations with a co-registered ultrasound probe.

[0169] FIG. 16 shows a processing workflow for importing other image data sets to achieve multi-modality image fusion and to improve SPECT reconstruction. A portable imaging instrument (1600) may comprise a molecular imaging camera system (1601), a co-registered ultrasound system (1602), and a co-registered computer vision camera (1603). Anatomical image data is delivered by another medical imaging scanner (1604). The scanner (1604) may be a CT, MRI, or another imaging system capable of providing anatomical data in 3D. In a preferred implementation, the scanner (1604) is a CT scanner. A computer operably connected to the computer vision camera analyzes the data from the computer vision camera and generates a computer vision generated 3D model of the patient's body. The computer also analyzes the data from the scanner (1604) and extracts an anatomical image generated 3D model of the patient's body. In process (1605), the computer vision generated 3D model is compared to the anatomical image device generated 3D model to generate a co-registration mapping between the two image datasets. This co-registration mapping can perform rigid co-registration or deformable co-registration. This generates a pre-co-registered anatomical image dataset. In some cases, a second stage co-registration (1606) may occur, where a co-registered ultrasound probe connected to the ultrasound system (1602) performs a scan at one or more locations on the patient's body. The computer analyzes the ultrasound scan, identifies ultrasound generated anatomical features, and matches these features with internal anatomical features in the pre-co-registered anatomical image dataset. This generates anchor points, where specific internal anatomical features in the pre-co-registered anatomical image dataset are assigned 3D locations equal to their location in the ultrasound image. The computer will use these anchor points to constrain the deformable co-registration solution and iterate on the pre-co-registered anatomical image dataset.This will generate a co-registered anatomical image dataset. In process (1607), the computer loads the molecular image data from (1601) and the pre-co-registered anatomical image dataset or the co-registered anatomical image dataset, and reconstructs the molecular image using the attenuation map extracted from the anatomical dataset. The computer can send a rendering, or a subsequent combination of the molecular image, the co-registered anatomical image, and the rendering of the ultrasound scan to a visualization device.

[0170] In some embodiments, the computer vision system (or a portion thereof) may not be attached to the assembly (1301). In this case, the computer vision system may monitor not only the tags and / or the patient, but also fiducials, which may be mounted on components of the scanner (1301) or parts connected to the cart (1301), panel (1306), robotic arm (1305), or other components. This would allow for co-registration of the sensor panel with respect to the computer vision camera and with respect to the patient or fiducial tags (1308) and (1309). Alternatively, the computer vision camera may be tracked by a separate tracking system, allowing for co-registration between the computer vision camera and the molecular imaging system.

[0171] FIG. 17 shows a cross-sectional side view of an embodiment of a wide-field coded aperture imaging system. This image illustrates a technique for enabling wide-field imaging using a mask element with multi-planar sides. The position-sensitive sensor (1700) is shielded on some sides not covered by the coded aperture by shields (1701a) and (1701b). The wide-field coded aperture mask comprises elements (1702a)-(1702h). The plane of the mask is placed at a distance (1703) away from the plane of the position-sensitive sensor (1700). Each individual mask element pixel has multi-planar sides that allow encoding of photons from a range of directions. For example, in this embodiment, the left mask elements (1702a)-(1702d) allow photon flux from a direction (7104) at about -45° from the mask normal to be coded based on their angle and sensed by the sensor (1700). These elements also allow photon flux from directions (1705) close to the mask normal to be coded based on their angle and sensed by the sensor (1700). Similarly, the right mask elements (1702e)-(1702h) allow photon flux from directions (1706) at about 45° from the mask normal to be coded based on their angle and sensed by the sensor (1700). These elements also allow photon flux from directions (1705) close to the mask normal to be coded based on their angle and sensed by the sensor (1700). With this type of mask configuration, images with high sensitivity can occur for angles of about -45° to 45° relative to the normal onto the mask surface. Similar mask element shaping and coding can occur in other perpendicular directions (normal onto the plane of the illustrated cross section). For example, the mask elements can be made of a rigid, dense, high atomic mass material such as tungsten or a tungsten alloy.

[0172] Figure 18 presents a top view of a wide-field coded mask (1800). The various shades of grey indicate different depths along the direction normal to the mask surface. Holes or openings into the mask are represented by white areas such as (1801). Identical holes arranged in a regular repeating pattern are not recommended as they may produce reconstruction artifacts.

[0173] FIG. 19 shows various views and geometries of pixel elements in a mask with square or rectangular pixels. A bi-pyramid mask pixel element is shown in a side view (1900). A square or rectangular base (1901) effectively defines the extent of the mask pixel in the mask plane. In this particular embodiment, the plane of the bi-pyramid base is parallel to the mask plane. The sides of the bi-pyramid can be perpendicular to the base of the bi-pyramid or can form various angles (1902) and (1903) with respect to the plane of the base. In some embodiments, the angles (1902) and (1903) have values ​​greater than 10 degrees. Thus, a bi-pyramid mask pixel element has at least a side that makes an angle greater than 10 degrees with respect to the normal on the bi-pyramid base. In some embodiments, the angles (1902) and (1903) have values ​​less than 60 degrees.

[0174] The pixel elements may have similar profiles when viewed vertically from the side, or may have sides that make other angles with the normal to the base surface. (1904) represents a 3D view of a rectangular bi-frustum that is symmetric about the center of a rectangular base and about a straight edge. This is only a representative example, and other bi-frustum geometries may be used. In some embodiments, when two mask pixels have a common edge, the direct space between the two bi-frustums may be filled with attenuating material. Two adjacent bi-frustums (1905) may be seen to form gaps (1906a) and (1906b) defined by the area between the dashed lines, as shown in FIG. 19. In some embodiments, these gaps are filled with attenuating material. A mask pixel element bi-frustum may also have a profile (1907) whose sides make the same angles (1908) and (1909) with the normal on the base surface (1910).

[0175] In some embodiments, a mask pixel bi-frustum can have a side view similar to (1900), another view like (1900), or a side view like (1900) and another view like (1907), or a side view like (1907) and another view like (1907). This last case is represented by the (1911) 3D view, which is essentially a symmetric bi-frustum of a right rectangular pyramid. Two adjacent bi-frustums (1911) can be seen forming gaps (1913a) and (1913b) defined in the view by the area between the dashed lines. In some embodiments, these gaps are partially or completely filled with a damping material.

[0176] In some embodiments, instead of using pixel mask elements shaped as bi-frustums with planar sides, pixel elements can be provided with rounded faces that substantially capture the attenuation profile of a bi-frustum with planar sides. A pixel element with rounded faces with profile (1914) as shown in FIG. 19 may provide attenuation characteristics similar to a bi-frustum with profile (1900). In some embodiments, when two such pixel elements have common edge neighbors as seen at (1915), the gap (1916) between the two pixel elements is filled with attenuation material. Similarly, a pixel element with a rounded nub with profile as shown at (1917) provides attenuation characteristics substantially similar to a bi-frustum with profile (1911). In some embodiments, when two such pixel elements have common edge neighbors as seen at (1918), the gap (1919) between the two pixel elements is filled with attenuation material.

[0177] In other embodiments, the sides of the mask pixel elements and the filling between the mask elements may have a stepped appearance or may have other microstructures that substantially follow macroscopic shapes similar to shapes (1900), (1905), (1907), (1912), (1914), (1916), (1917), and (1918).

[0178] In one embodiment, the mask pixel elements and the filler attenuating material between the mask pixel elements are made from a material with a density greater than 10 g / cc.

[0179] In some embodiments, the mask pixel elements and the filler attenuating material between the mask pixel elements are made from the same material that is dense and has a high atomic number Z, such as tungsten or a tungsten alloy.

[0180] While mask pixel elements shaped as bi-pyramidal frustums with rectangular bases are depicted, bi-pyramidal frustums with triangular or hexagonal bases can be used as mask pixel elements in a mask array with triangular or hexagonal pixels, respectively. A single coded aperture mask can combine mask pixels with different geometric shapes, such as rectangular, square, triangular, or hexagonal, and may have various dimensions. Similar to rectangular pixels, triangular or hexagonal bi-pyramidal frustum mask pixel elements may be symmetric about the center of the triangular or hexagonal base, symmetric about the triangular or hexagonal base face, or may have no symmetry either. The shape of rectangular, triangular, or hexagonal bi-pyramidal frustum mask elements may vary across the mask. Attenuating materials may be used in any of these geometries to partially or completely fill the space between bi-pyramidal shaped mask pixel elements that share a common edge.

[0181] FIG. 20 shows an embodiment of a wide-field coded aperture similar to that shown in FIG. 17, but with the coded aperture extending on another plane. In this embodiment, the coded aperture extends on a plane perpendicular to the first mask plane. The position-sensitive sensor (2000) is shielded on some sides not covered by the coded aperture by shields (2001a) and (2001b). The coded aperture mask in the first plane comprises elements (2002a)-(2002h). The first plane of the mask is placed at a distance (2003) away from the plane of the position-sensitive sensor (2000). In some embodiments, the distance (2003) can be increased to allow for increased angular resolution, which can be translated into increased image resolution. The mask elements (2004a) and (2004b) can be positioned in other planes than the first mask plane to maintain a wide image field of view (2005). In some embodiments, a wide image field of view can be maintained at a larger mask-sensor distance (2003) by extending the physical outline of the mask in the mask plane beyond the physical outline of the sensor in the same plane.

[0182] The position sensitive sensors (2000) can be made up of multiple sensor units having faces that are approximately parallel to each other, substantially coplanar and parallel to the first mask surface (2006), or they can have sensor units that have their front faces in a plane that forms an angle between 0 degrees and 90 degrees with respect to the first mask surface (2006).

[0183] FIG. 21 shows an embodiment of a wide-field coded aperture similar to that shown in FIG. 17, but with other components depicted to support the description of the process by which detected events can be used in image reconstruction analysis with a wide-field coded aperture. The position-sensitive sensor (2100) is shielded on some sides not covered by the coded aperture by shields (2101a) and (2101b). The coded aperture mask in a first plane comprises elements (2102a)-(2102h). The first plane of the mask is placed at a distance (2103) away from the plane of the position-sensitive sensor (2100). The first radiation interaction (2104) and the second radiation interaction (2105) are detected by the sensor (2100) with position resolution in all three dimensions. The sensor (2100) can be a semiconductor detector such as CdZnTe, CdTe, a scintillator detector such as Na(I), LSO, or any other scintillator material capable of providing the location of the interaction in three dimensions. The location of the interaction can be achieved in two dimensions by segmenting the readout electrodes, by employing a position sensitive photodetector array, by employing signal analysis methods, by creating individual sensor rods with individual signal readouts, or by a combination of the above. The location of the interaction in the third dimension can be achieved by using a sensor with depth of interaction capability. Some systems can provide the location of the interaction in depth for both semiconductor and scintillator sensors.

[0184] Once radiation has been detected in 3D, a process may be undertaken to use that information to reconstruct an image of the radiotracer distribution with high image resolution and sensitivity.

[0185] One step towards reconstructing an image is to determine a backprojection operator, which uses a probability density function (pdf) for a detected radiation event originating somewhere from a volume outside the coded aperture and shielding. To calculate the pdf for a detected event, a radiation transmission map or attenuation coefficient map is determined through the coded aperture and shielding and at the location where the radiation was detected.

[0186] In an embodiment, the attenuation map is pre-calculated across at least a plane (2106), referred to as the instrument response reference plane or simply the reference plane. The reference plane can be parallel to the first mask plane or can be at any angle with the first mask plane. Also, the plane can be attached to any physical component, such as the face of the sensor, the first mask plane, or positioned anywhere else in space. For illustrative purposes, in this figure, the reference plane is parallel to the first mask plane and coincides with the face of the sensor. For each point or pixel in the reference plane, the attenuation map comprises radiation attenuation coefficients, or other information that can be used to extract attenuation coefficients, such as path lengths through the mask material, for a given type of radiation for various angles across the coded aperture image field.

[0187] For each detected ray, the 3D position of the interaction, such as (2104) and (2105), is projected onto the reference plane along a direction towards the coded mask field of view, such as direction (2107a)-(2107d) for interaction (2104) and direction (2108a)-(2108c) for interaction (2105). The resulting intersection with the reference plane (2106) produces points (2109a)-(2109d) for projecting the interaction (2104) along directions (2107a)-(2107d), respectively. The locations (2109a)-(2109d) are used to read out the attenuation coefficients through the mask along directions (2107a)-(2107d), respectively. In this example, the path of the ray along all these directions includes segments through the sensor from (2104) to (2109a)-(2109d). This path may not be captured by the pre-calculated attenuation coefficients in the reference plane (2106). In some implementations, these paths can be added into the pdf calculation to scale the intensity of the backprojection along each of their directions (2107a)-(2107d). This process for calculating the backprojection operator is particularly useful when list-mode image reconstruction algorithms are employed. The pdf calculated from the attenuation coefficients for the interactions at positions (2104) and (2105) can be employed in the reconstruction of the gamma source image by employing methods commonly used for image reconstruction, such as statistical iterative methods, algebraic iterative methods, analytical methods, or compressive sensing methods.

[0188] Although this embodiment depicts only a single reference plane, in other embodiments, multiple instrument response reference planes may be used to increase imaging performance.

[0189] In other embodiments, the reference plane can be located at other positions. Figure 22 illustrates such a case. The position sensitive sensor (2200) is shielded on some sides not covered by the coded aperture by shields (2201a) and (2201b). The coded aperture mask in a first plane comprises elements (2202a)-(2202h). The first plane of the mask is placed at a distance (2203) away from the plane of the position sensitive sensor (2200). In this case, the instrument response reference plane (2204) is chosen to be identical to the first mask plane. The 3D position of the radiation interaction (2205) is projected onto the reference plane along a relevant direction towards the coded mask field of view, such as directions (2206a)-(2206d). The resulting intersection with the reference plane (2204) produces points (2207a)-(2207d), respectively. The locations 2207a-2207d are used to read out the attenuation coefficients through the mask along the directions 2206a-2206d, respectively. In this example, the paths of radiation along the directions 2206a-2206d include segments through the sensor from 2205 to 2208a-2208d, respectively. These paths may not be captured by the pre-calculated attenuation coefficients in the reference plane 2204. In some implementations, these paths can be added into the pdf calculation to scale the intensity of the backprojection along each of those directions 2206a-2206d. This process for computing the backprojection operator is particularly useful when a list-mode image reconstruction algorithm is employed in the reconstruction of the radioactive source distribution.

[0190] When a binned image reconstruction of the radioactive source distribution is desired, some embodiments may involve the use of multiple instrument-responsive reference planes. Figure 23 shows an example when multiple reference planes are used. The position-sensitive sensor (2300) is shielded on some sides not covered by the coded aperture by shields (2301a) and (2301b). A coded aperture mask in a first plane comprises elements (2302a)-(2302h). The first plane of the mask is placed at a distance (2303) away from the face of the position-sensitive sensor (2300). The first radiation interaction (2304) and the second radiation interaction (2305) are detected by the sensor (2300) with position resolution in all three dimensions. In this embodiment, five instrument-responsive reference planes (2306a)-(2306e) may be employed in the analysis. In this embodiment, the reference planes sample the sensitive volume of the sensor. The backprojection directions towards the coded mask field of view are represented by (2307a)-(2307d) for interaction (2304) and by (2308a)-(2308c) for interaction (2305). In this case, the attenuation coefficients along the directions (2307a)-(2307d) and (2308a)-(2308c) are extracted from attenuation data calculated at the reference plane and location closest to the interaction point. Thus, the attenuation coefficients from interaction (2304) along the directions (2307a)-(2307d) can be extracted from attenuation data calculated in the reference plane (2306d) at the location or bin closest to location (2304), and the attenuation coefficients from interaction (2305) along the directions (2308a)-(2308c) can be extracted from attenuation data calculated in the reference plane (2306b) at the location or bin closest to location (2305). These extracted attenuation coefficients can then be used to construct a pdf for the incremented bins. This sampling and pdf computation scheme can also be used for list-mode images.

[0191] When binned image reconstruction is implemented, the binned counts in the bin in the plane (2304d) closest to (2304) can be incremented as a result of the interaction detected in (2304), and the binned counts in the bin in the plane (2304b) closest to (2305) can be incremented as a result of the interaction detected in (2305). PDFs can then be calculated along directions (2307a)-(2307d) and (2308a)-(2308c), respectively, for the resulting binned intensities.

[0192] The resulting pdf, whether in list mode or binned mode, can be used in any suitable iterative or analytical image reconstruction algorithm known in the art. Thus, the calculated pdf for the counts in the bins in the reference plane closest to the interaction locations (2304) and (2305) can be employed in the reconstruction of the gamma source image by employing methods commonly used for image reconstruction, such as statistical iterative methods, algebraic iterative methods, analytical methods, or compressive sensing methods.

[0193] FIG. 24 shows a perspective view of a wide-field coded aperture mask (2400) made from multiple thin mask layers stacked and secured together. The light gray layer (2401) represents the top layer, the darker gray layer (2402) represents the middle layer, and the light gray layer (2403) represents the bottom layer. In this embodiment, the mask pixel elements are freestanding. In embodiments where the mask elements are not freestanding, a frame with low radiation attenuation properties can be used to hold the mask elements in the desired location. This manufacturing modality can produce mask elements with a stepped appearance, but with sides that may substantially follow the bi-frustum shaping of the mask elements as described above.

[0194] In some embodiments, the individual layers in the coded aperture mask are not fixed to each other and may be moved laterally relative to each other by actuators to generate different image fields and coding profiles. The actuators may be controlled by a computer. The actuators may also move the individual layers away from each other to increase the collimation effect. Similarly, in some embodiments, each layer may be formed by multiple plates with patterned holes that may be moved relative to each other.

[0195] In some embodiments where the gamma sensor may not be equipped to deliver the location of the gamma ray interaction with resolution in depth, the sensors may be positioned at different angles relative to each other and to the wide field mask to minimize the range of off-normal angles of the detected photon incident direction. This minimization will reduce image errors associated with unknown depth interactions. Figure 25 shows a cross-sectional side view of a wide field coded aperture mask (2500) placed in front of two sensors (2501) and (2502). The sensors are arranged in different planes relative to each other and to the mask (2500) to minimize the range of angles (2503) made by the incident gamma ray photon directions (2504), (2505), (2506) with respect to the normal of the sensors (2507) and (2508). In this implementation, the sensors have their normal directions (2507) and (2508) converge toward a space essentially adjacent to the middle of the mask (2500). Other embodiments of sensors arranged in different planes with respect to each other and with respect to the mask can also be envisioned to minimize the range of angles made by the incident gamma ray photon direction with respect to the sensor normal for most sensors across the sensor group. The mask can be a wide field mask as described above, or an adjustable field mask as described below, a localized collimated mask, or a combination thereof. The angle made by the sensor normal can range from 0° to 90°. In some configurations, the angle between the two sensor normal directions is between 30° and 60°. In some configurations, the angle between the two sensor normal directions is between 40° and 50°.

[0196] In some embodiments, the mask itself can be made of mask segments that lie in multiple planes that are generally non-coplanar.

[0197] 26 shows a top view from a mask (not shown) of an array of four sensors or sensor panels positioned in different planes with their normal directions essentially converging toward a volume area adjacent to the mask. This arrangement minimizes the range of angles made by the incident gamma ray photon direction with respect to the sensor normal. In this depiction, the closer the sensor angles (2601a-d) are to the mask, the further away the sensor angle (2602) is from the mask.

[0198] In some applications, the gamma ray sensor panel described above may be required to both scan a patient and generate a 3D molecular image map, and provide a real-time image of a reduced volume inside the patient. These two imaging modes may require a gamma ray imaging architecture that provides a wide field of view in the scanning mode, and a collimated narrow focal field of view in the real-time imaging mode. Similarly, there may be an advantage to being able to scan a portion of a patient with both a wide and a narrower field of view, and generate a more accurate 3D map. A coded aperture mask with an adjustable field of view can accommodate these requirements.

[0199] FIG. 27 shows a cross-sectional side view of a coded aperture imaging system with a gamma ray mask with an adjustable field of view. An image gamma ray sensor array (2700) is placed behind a mask assembly that comprises multiple overlapping layers made of dense high atomic number material such as tungsten alloy. In this example, the mask comprises three layers. The middle layer of the mask (2701) is made of a plate with holes or openings that penetrate from one side of the plate to the other to create a patterned plate. Examples of patterns made by the openings are shown in FIGS. 31 and 32. The pattern can be a pseudo-random array as shown in FIG. 31, an index class aperture array, an assembly with curved slits of various curvatures as shown in FIG. 32, or another pattern with a nearly flat side lobe in the autocorrelation function across multiple magnifications. It is important to maintain a nearly flat side lobe across multiple magnifications because the imager will be exposed to sources from very close to intermediate distances and will generate projections on the detector array through the mask at multiple magnifications. In this illustration, the top layer is made of two patterned plates (2702a) and (2702b). The patterns on the patterned plates (2702a) and (2702b) may essentially spatially match the pattern of the middle layer (2701), although in some implementations there may be differences between the overlapping patterns. In this illustration, the bottom layer is made of two patterned plates (2703a) and (2703b). The patterns on the patterned plates (2703a) and (2703b) may essentially spatially match the pattern of the middle layer (2701), although in some implementations there may be differences between the overlapping patterns. A set of overlapping mask layers may be mounted on the side collimators (2704a) and (2704b). These side collimators may also be made of a gamma ray attenuating material, such as a tungsten alloy. The mask may be placed at a focal distance (2705), which may be alterable by an actuator controlled by an operably coupled computer. Openings in the mask layer in cross section are represented by grey areas (2706).The mask layers can be made of bi-frustum elements, as described above. Also, as described above, these bi-frustum elements can have straight or rounded edges, and their specific shape can vary from one layer to another, and from one part of a layer to another part of the same layer, or from one element to another. In some implementations, the pattern can be as described in FIG. 32. This arrangement of layers close to each other allows for a far-field image on the mask that allows projection of the normal (2707), as well as projections near +45° (2708), and projections at −45° (2708) (2708).

[0200] FIG. 28 shows a cross-sectional side view of a coded aperture imaging system with an adjustable three-layer mask, here in a collimated (foveated) viewing configuration. An imaging gamma ray sensor array (2800) is placed behind the mask assembly described in FIG. 27. In this implementation, the middle layer of the mask (2801) or (2701) of FIG. 7 is in the same relative location to the sensor of FIG. 7. However, the top layer made of patterned plates (2802a) and (2802b) has been moved away from the middle layer by an actuator operably coupled to the computer. In some implementations, plates (2802a) and (2802b) may also be laterally biased relative to the middle layer by an actuator operably coupled to the control computer. In this case, the two plates are moved laterally toward the center of the mask. The bottom layer made of patterned plates (2803a) and (2803b) has also been moved away from the middle layer by an actuator operably coupled to the computer. In some implementations, plates (2803a) and (2803b) may also be laterally displaced relative to the intermediate layer by actuators operably coupled to the control computer. In this case, the two plates are moved laterally towards the edges of the mask. The focal length of the mask is represented by (2805). The net effect of these translations is a reduction in the field of view towards the center of the mask, since the separation of the layers creates a collimating effect. The main projection directions remaining through the aperture, represented by the grey area (2806), are represented by the directions (2807) and (2808).

[0201] 29A and 29B show schematic top views of an adjustable coded aperture mask made of nine panels arranged in three layers in two configurations: four at the top, one in the middle, and four at the bottom. The mask in FIG. 29A is configured with a wide field of view, while the mask in FIG. 2B is configured with a narrow collimated field of view. For clarity, the elements and movements are not drawn to scale. The purpose of this figure is to represent an example of the lateral movement of the plates as they transition from a wide field of view to a narrow field of view. The outline of the middle layer is represented by a continuous line (2900) in the wide field configuration. The outlines of the four plates that form the bottom layer (towards the detector) are represented by dashed squares (2901a-d). The outlines of the four plates that form the top layer (away from the detector) are represented by dotted squares (2902a-d). This configuration may have layers close to each other, particularly generating wide field apertures in the left-right direction.

[0202] The outline of the middle layer in the narrow field of view configuration is represented by a continuous line (2903). The outline of the four plates forming the bottom layer (towards the detector) is represented by dashed squares (2904a-d). The outline of the four plates forming the top layer (away from the detector) is represented by dotted squares (2905a-d). This configuration may have layers spaced apart from one another to produce a narrow collimated field opening towards the mask midline. Other lateral movements are also possible. For example, to produce collimation away from the mask midline in any direction of interest, plates within the same layer may be moved laterally in concert in the same direction. For example, to shift the collimation of the mask towards the right of the figure, the top plates (2905a-d) would be moved to the right from the configuration of FIG. 29B and the bottom plates (2904a-d) would be moved to the left from the configuration of FIG. 29B. Depending on the exact alignment of the patterns between the layers, other plate movements may achieve similar effects. Similarly, depending on the exact alignment of the patterns between layers, the same plate movement can achieve very different collimation and field of view changes. For example, in some implementations, when the patterns across layers have a certain alignment, lateral movement of the plate across different layers can reduce or increase the effective open fraction of the mask.

[0203] 30A and 30B show schematic top views of an adjustable coded aperture mask made of 19 panels arranged in three layers in two configurations: wide field in 30A and narrow collimated field in 30B: 9 at the top, 1 in the middle and 9 at the bottom. For clarity, the elements and movements are not drawn to scale. The purpose of this figure is to represent an example of the lateral movement of an array of 3x3 plates when transitioning from wide field to narrow field. The outline of the middle layer is represented by a continuous line (3000) in the wide field configuration. The outline of the nine plates forming the bottom layer (towards the detector) is represented by dashed squares (3001a-i). The outline of the nine plates forming the top layer (away from the detector) is represented by dotted squares (3002a-i). This configuration may have layers close to each other and generate wide field apertures in particular in the left-right direction. The outline of the middle layer in the narrow field of view configuration is represented by a continuous line (3003). The outline of the nine plates forming the bottom layer (towards the detector) is represented by dashed squares (3004a-i). The outline of the nine plates forming the top layer (away from the detector) is represented by dotted squares (3005a-i). This configuration may have layers spaced apart from each other, creating a narrow collimated field of view opening towards the centerline of the mask. Other lateral shifts and effects may occur as well, as illustrated in Figures 29A-B.

[0204] FIG. 31 shows a top view depiction of layers in an adjustable multi-layer mask with a pseudo-random pattern with a 22% open fraction. The pixel elements may be bi-frustum as described above with straight or rounded edges. The pattern may be optimized to provide approximately flat side lobes in the autocorrelation function across multiple magnifications, such as for any combination of magnifications from 1× to 6×. The open fraction for this mask may be between 0.1% and 70%. In a preferred embodiment, the open fraction is between 5% and 30%.

[0205] FIG. 32 shows a top view depiction of a coded aperture mask with curved slits of various curvatures. The combination and arrangement of slits can be optimized to provide a nearly flat side lobe in the autocorrelation function across multiple magnifications, such as for any combination of magnifications from 1× to 6×. The open percentage for this mask may be 0.1% to 70%. In a preferred embodiment, the open percentage is 5% to 30%. The slits may have straight edges, rounded edges similar to the rounded bi-frustum described above, or edges of various other geometries, such as a v-shaped profile. The shape of the edges may vary from one slit to another or within the same slit. The rounded slits may or may not interact. The curvature of the slits may vary within and across the same slit. Such slit geometries of various curvatures may help achieve flat side lobes in the autocorrelation function across multiple magnifications.

[0206] FIG. 33 shows a diagram of a handheld SPECT camera device (3300). The handheld instrument (3300) contains a position-sensitive sensor (3301) placed behind a wide-field coded aperture mask (3302) and between the shields (3303a) and (3303b) placed in places not covered by the mask (3302). The imaging system is characterized by a wide image field of view (3304). This particular instrument also includes a video camera or scanner (3305) that is oriented to generate a 3D model of the patient and to collect contextual information that can be used to locate the handheld camera system relative to the patient. A handle (3306) can be used to easily move the device.

[0207] Figure 34 shows a drawing of a top view of the handheld instrument (3400) presented in Figure 33. The handheld instrument (3400) is shown containing a position sensitive sensor (3401) placed behind a wide field coded aperture mask (3402) and between obscurants (3403a) and (3403b) placed in locations not covered by the mask (3402). The imaging system is characterized by a wide image field of view (3404).

[0208] Using the handheld SPECT camera device described in Figures 33 and 34, high-resolution SPECT images can be reconstructed by taking advantage of the high resolution, wide field of view, and high sensitivity characteristics of the various imaging embodiments presented here. Figure 35 shows a depiction of a modality in which such an instrument can perform SPECT images from a limited number of directions around the patient, but still retain a significant range of projections for each voxel in the field of view. The handheld imaging device is moved at different locations (3500a)-(3500d) on one side of the patient (3501). The figure shows how voxels (3502) inside a molecularly tagged organ (3503) inside the body can be imaged by the device at all locations (3500a)-(3500d) thanks to the wide field of view of the imaging system. An example of projection lines 3504a-3504d toward the SPECT camera at locations 3500a-3500d shows a range of angles (such as between directions 3504a and 3504d) that can be even greater than 90 degrees, which is sufficient disparity to reconstruct an image with high resolution in all three coordinates.

[0209] In some embodiments, the SPECT camera may be moved around and along the patient by a robotic arm or other mechanical system.

[0210] Thus, embodiments of the imaging modalities described herein enable simpler, lighter and more economical SPECT imaging devices with improved imaging performance.

[0211] FIG. 36 shows a flow chart summarizing some systems and methods enabled by a portable molecular imaging system. In this case, the process is illustrated assuming a portable SPECT imaging system. The first step in the process (3601) is to inject a molecular agent, in this example a SPECT molecular agent, into the patient. In a second step (3602), a SPECT system as described herein is used to scan a portion of the patient and provide data that is used by a computer operably coupled to construct a 3D map of the molecular agent in step (3603). The scan object will be selected by a user from a group of scan objects stored in the computer's memory. In particular, a computer vision system as described in FIG. 1 and FIG. 13 may deliver data to the computer and generate a 3D model of the patient's body, as described above. The computer may use the 3D model of the patient and generate a scan protocol given the scan object. The scan protocol will comprise a set of actuations of the robotic arm and the sensor panel will be caused to scan the patient. The set of actuations may also include instructions to actuators that move the gamma mask elements and the focal length of the mask. The set of actuations may also include instructions to actuators that move the portable cart shown in FIGS. 1 and 13.

[0212] The SPECT system may use an adjustable mask in a wide field of view configuration (see FIG. 27). However, for some applications, the mask-detector assembly may be in other narrower field of view configurations, or may change at different points during the scan. The computer or user may control those changes during the scan. Similarly, for some applications, during the scan (3602), the focal length between the mask and detector may change at various points during the scan. The computer or user may control those changes during the scan.

[0213] To aid in the image reconstruction process, image scanning can be performed using other instruments (3604) to generate an image dataset, such as an anatomical image dataset, that can be used by a computer operatively coupled to the SPECT scanner. For example, a CT dataset can be used to provide an attenuation map to be used by the computer to generate a more accurate reconstruction of the molecular image.

[0214] Co-registration systems (3605) and methods such as those described in FIG. 13 can be used to merge the SPECT data with other image data. For improved co-registration, especially when the 3D model of the patient's body and the 3D model extracted from the external image dataset deviate significantly, a co-registered ultrasound scan can be performed to allow the computer to pin specific structures in the other imaging modalities to specific locations (3606), as described in FIG. 13. The other image datasets can be rendered and sent to a display for inspection by the user (3607). Similarly, the computer can send a rendering of the reconstructed 3D molecular map to a visualization device for inspection by the user. The visualization device can be a screen, a head-mounted display, an augmented reality device, or another visualization system. The computer may send a combination of other anatomical and molecular images to the visualization device. The computer may also store in memory the molecular image dataset, the ultrasound image dataset used to assist in the co-registration, and the other anatomical image dataset.

[0215] Following the 3D image reconstruction (3603), using the reconstructed data, the computer may use instructions stored in memory to control the molecular imaging system to perform a tracking scan (3608) and improve the quality of the reconstructed 3D map. For repeat scans, the computer may generate another scan protocol with a new operating data set. For example, the computer may send instructions to the robotic arm and panel to scan the patient, or send instructions to the mask actuator to change the field of view or change the focal length for tracking scans. For example, it may be beneficial to rescan a specific part of the body in foveated narrow field of view mode to obtain better molecular image contrast within a specific area of ​​the body where a feature of interest may be present. The scan protocol may be sent to the user for approval. Additionally, the user may initiate a repeat scan given the data presented to the user after the first scan. In some implementations, the user may be remote. The computer may use data from the molecular image scan, and / or from other co-registered anatomical image scans, and / or from the co-registered ultrasound scans, and perform neural network, or deep learning, analysis of the data, determine usefulness for follow-up scans, and perform classification of image features. Results of the analysis can be presented to a user. Among other things, the results can comprise a rendering of the co-registered image dataset and a rendering of a processed fused image that may contain classification values. Statistical, deep learning, and sensor fusion algorithms can be used by the computer for this purpose.

[0216] Following the visualization step (3607), the user may wish to obtain a real-time image of a molecular structure. Using a user interface, in step (3609), the user may select such structures of interest that are identified in the rendering of the co-registered data set. The selected features will be characterized by 3D coordinates.

[0217] In step (3610), using instructions stored in memory, the computer will actuate any of the robotic arm, sensor panel, and mask actuators to orient the sensor toward the selected feature. For example, the mask may be actuated to generate a narrow field of view collimated toward the feature of interest, maximizing image sensitivity and signal-to-noise in the area in which the feature of interest resides. In calculating the movement of the panel toward where data may be captured from the region of interest, the computer will take into account a 3D model of the patient's body so that scanner components, including the panel, will not collide with the patient.

[0218] At this point, in step (3611), the data collected from the molecular image sensor may be analyzed by a computer to generate an image in near real time. As described above, the computer may use a previously scanned molecular 3D dataset, such as the dataset resulting from step (3603) or (3608), in conjunction with the real-time molecular data delivered by the molecular sensor under step (3611) to improve the quality of the real-time molecular image rendering. Similarly, as described above, a co-registered ultrasound scan may be performed under step (3612) to provide anatomical context to the molecular image. The molecular image to be enhanced onto the ultrasound image may be delivered by step (3603) or (3608), or may be a real-time image resulting from step (3611).

[0219] Additionally, a user may use the ultrasound scans visualized in the user interface to select features of interest that are 3D co-registered under step (3609). The user may select such structures of interest that are identified in any rendering of the co-registered image dataset. In step (3613), the computer co-registers the ultrasound dataset with the live molecular image using tracking data such as provided by a computer vision system as described in FIG. 1 or 13.

[0220] In some implementations, the computer may use the real-time ultrasound scans delivered under step (3612) to generate a tissue deformation model that is used in constructing and rendering either the stored 3D molecular image dataset (from steps (3603) or (3608)) or the real-time molecular image (from step (3611)). Details of the deformation modeling process are presented above.

[0221] In step (3614), the computer transmits a rendering of the ultrasound scan and the co-registered molecular image scan to a visualization device for user review. In some implementations, in step (3615), an intervention guided by the molecular imaging system can be performed. For example, the intervention can use a rendering of the molecular image to highlight the target. In another example, the intervention can use a rendering of the molecular image augmented on live ultrasound or other real-time imaging modality to highlight the target. Such an intervention can comprise a biopsy, ablation, resection, radiation therapy, or other medical procedure. The intervention can be selected from a group including an intervention performed manually by a user using a needle, an ablation system, a surgical device, or other medical instrument, an intervention performed by a high intensity focused ultrasound system co-registered to treat an area, an intervention performed through a stereotactic system co-registered to guide biopsy and surgery, an intervention performed by a robotic medical system to guide resection and surgery, an intervention performed by a laparoscopic system to guide resection and surgery, an intervention performed by a radiation therapy device co-registered to treat a tumor. For example, molecular images enhanced on ultrasound images can guide a user to drive a needle, an ablation system, or another medical device toward a feature of interest. Co-registration with other imaging and treatment systems can be performed using built-in computer vision cameras, other co-registration and tracking devices. In some implementations where the user performing the intervention (3615) is robotic or automated, step (3614) may be skipped.

[0222] Molecular imaging systems can be used in conjunction with imaging modalities other than ultrasound for diagnosis and for intervention guidance. For example, co-registered optical medical systems such as endoscopes, bronchoscopes, laparoscopes, colonoscopes, microscopes, robotic endoscopes, and robotic laparoscopes can be used. In step 3616, such optical imaging systems are used to image the patient.

[0223] In step (3617), the computer co-aligns the optical instrument with the molecular imaging system. If the optical medical imaging device is rigid, an on-board computer vision system as described in Figures 1 and 13 can be used to identify the medical imaging device to the computer vision system and the molecular imaging system. Tags and labels can be affixed to the optical medical devices to aid in their location and tracking.

[0224] If the optical system is flexible or not easily within the field of view of the computer vision system, other modalities can be used to generate the co-registration. For example, if the optical medical device is an endoscopic camera or a flexible laparoscopic camera, in step (3618), a fluoroscope may be used to determine the position and orientation of the endoscope relative to the fluoroscopic reference system by having the computer load the fluoroscopic image, analyzing features associated with the structure of the endoscope, and inferring the location of the fluoroscopic camera relative to the fluoroscope's x-ray source and sensor. In some implementations, the optical medical device may already be co-registered with an x-ray system, such as a fluoroscopic system. In this case, it is not necessary for the computer to analyze the fluoroscopic image and infer the position of the optical medical device.

[0225] A fiducial tag comprising a feature identifiable in a fluoroscopic image can be positioned within the field of view of the fluoroscope, such as on or near the patient. The fiducial tag may also comprise a feature identifiable by an embedded computer vision system. A computer operatively connected to the computer vision system may use the computer vision data to determine the location of the computer vision camera relative to the fiducial tag. The computer may retrieve and analyze the fluoroscopic image to extract the location of the fiducial tag relative to the fluoroscope. The computer may then use the co-registration between the optical imaging camera and the fluoroscope, the position of the fluoroscope relative to the tag, and the position of the tag relative to the computer vision system to determine the position of the laparoscopic or endoscopic medical optical camera relative to the computer vision camera. This allows for co-registration between the image taken by the optical medical camera and the molecular image. In step (3619), the computer may send a rendering of the co-registered images taken by the optical medical device and the molecular imaging device to the visualization device. In some implementations, the molecular image will always be rendered by the computer in a perspective rendering to match the position, orientation, and focal length of the medical optical camera. This will generate rendered molecular image data suitable for augmentation onto the live image taken by the optical medical device. Renderings such as maximum intensity projection can be used to render the molecular image. Other tracking systems can also be used to co-register the X-ray system with the molecular imaging system. An intervention such as biopsy, ablation, resection, surgery, etc. can occur in step (3620), guided by the augmentation of the live optical image and molecular image rendering delivered by the optical medical device. Step (3619) can be skipped if the medical intervention (3620) is automated or robotic.

[0226] While the above description contains many specifics, these should not be construed as limitations on scope, but rather as exemplifications of one or several embodiments thereof. Many other variations are possible.

[0227] In some embodiments, a computer system may be used to implement any of the entities or components described above. The computer system includes a central processor for communicating with each subsystem and for controlling the execution of instructions from a system memory or fixed disk, as well as the exchange of information between other subsystems of the computer system. As used herein, a processor includes a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or networked. The system memory and / or the fixed disk may embody a computer-readable medium. The computer system may also include input / output (I / O) devices. The computer system may include a network interface that may be used to connect the computer system to a wide area network, such as the Internet.

[0228] Storage media and computer readable media for containing the code or portions of code can include any suitable media known or used in the art, including storage media such as, but not limited to, volatile and nonvolatile, removable and non-removable media and communication media implemented in any method or technology for storage and / or transmission of information, such as computer readable instructions, data structures, program modules, or other data, including RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical storage devices, magnetic cassettes, magnetic tapes, magnetic disk storage devices, or other magnetic storage devices, data signals, data transmissions, or any other media that can be used to store or transmit the desired information and that can be accessed by a computer.

[0229] Any of the software components or functions described herein may be implemented as software code to be executed by a processor using, for example, any suitable computer language, such as, for example, C, C++, C#, Objective-C, Swift, using conventional or object-oriented techniques, or a scripting language, such as Perl or Python. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media, such as hard disks or floppy disks, or optical media, such as compact disks (CDs) or DVDs (digital versatile disks), flash memory, and the like. The computer-readable medium may be any combination of such storage or transmission devices.

[0230] The descriptions "a," "an," or "the" are intended to mean "one or more," unless specifically stated to the contrary. The use of "or" is intended to mean "inclusive or" and not "exclusive or," unless specifically stated to the contrary.

[0231] Use of the terms "first," "second," "third," "fourth," etc. may be used to distinguish one element from another and does not necessarily imply an order or hierarchy between such elements unless otherwise indicated.

Claims

[Claim 1] The invention described in this specification.

Images