Methods and systems for high performance and multifunctional molecular imaging

Abstract

Improved imaging equipment and methods. Portable SPECT imaging devices can be co-registered with imaging modalities such as ultrasound. A gamma camera panel, including a gamma camera sensor, can be attached to a robotic arm. An coded aperture mask can be placed in front of the gamma-ray photon sensor and used to construct a high-resolution 3D map of the distribution of radioisotopes within a patient, generated by scanning the patient from a narrowed perspective and placing a radiation sensor near the patient. Enhanced imaging sensitivity and resolution are provided. SPECT imaging devices can be used to guide medical interventions such as biopsies and ablation treatments, as well as to guide surgery.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit and priority of U.S. Application No. 62 / 831504, filed April 9, 2019, entitled “Methods and Systems for High-Performance Spect Imaging”, and U.S. Application No. 62 / 836514, filed April 19, 2019, entitled “Methods and Systems for Portable Spect and Ultrasound Imaging”, the contents of which are incorporated herein by reference in their entirety for all purposes. Technical Field

[0003] This invention relates to the architecture of a gamma camera and its use with other co-registered medical imaging modalities such as ultrasound systems to achieve new high-performance and multifunctional imaging systems for diagnostic imaging, guidance of medical interventions such as percutaneous biopsy and ablation therapy, and guidance of surgical procedures. Background Technology

[0004] Single-photon emission computed tomography (SPECT) alone, or in combination with computed tomography (CT) (SPECT / CT), is a primary molecular imaging modality used for medical diagnostic imaging. Most commonly, SPECT imaging devices consist of an array of gamma-ray sensors orbiting or orbiting the patient's body. During the imaging scan, the patient typically lies on a table, and for some cardiac imaging systems, may sit in a custom-made chair. A parallel-aperture collimator is typically used in front of the detector array to limit the direction that gamma-ray photons can take before interacting with the position-sensitive sensors. This produces a parallel projection of the distribution of radioactive isotopes emitting gamma rays within the patient's body. Computer programs are used to reconstruct this 3D distribution using analytical or iterative image reconstruction algorithms.

[0005] The embodiments provide improved methods and systems for SPECT imaging. Summary of the Invention

[0006] The embodiments relate to systems and methods for single-photon emission computed tomography (SPECT) imaging.

[0007] Some embodiments provide a portable single-photon emission computed tomography (SPECT) imaging system for scanning a patient. The system includes a SPECT controller unit, which includes a computer. The system also includes a mechanical articulated arm connected to the controller unit. A user can position the articulated arm to a desired location by applying a direct force. The system also includes at least one gamma camera panel connected to the articulated arm. The gamma camera panel includes a gamma camera sensor with position and energy sensing resolution. The gamma camera panel can provide an imaging field of view greater than 15 degrees. The system also includes a camera mounted to observe the entire area of ​​the patient. The system further includes at least one processor and a memory operatively coupled to the at least one processor, the camera, and the gamma camera sensor. The memory has instructions executable by the at least one processor, causing the at least one processor to read a first gamma-ray photon sensing event received from the gamma camera sensor. The processor also provides a first position and orientation of the gamma camera panel relative to the patient's body. The processor also co-registers the first gamma-ray photon sensing event with the patient's body using the first position and orientation. The processor also reads a second gamma-ray photon sensing event received from the gamma sensor. The processor also provides a second position and orientation of the gamma camera panel relative to the patient's body. The processor further uses the second position and orientation to co-register a second gamma-ray photon sensing event with the patient's body. And the processor reconstructs the 3D distribution of radioisotopes emitting gamma rays within the patient's body using the first and second co-registered sensing events.

[0008] Some embodiments provide a real-time multimodal portable single-photon emission computed tomography (SPECT) imaging system for scanning patients. The system includes a SPECT controller unit, which includes a computer. The system also includes a mechanical articulated arm connected to the controller unit, wherein a user can position the articulated arm to a desired location by applying a direct force. The system also includes at least one gamma camera panel connected to the articulated arm. The gamma camera panel includes a gamma camera sensor with position and energy sensing resolution. The system also includes an ultrasound transducer, which can be positioned to have a field of view that at least partially overlaps with the field of view of the gamma camera. The system also includes a tracking system capable of providing tracking information about the relative position of the ultrasound transducer relative to the gamma camera. The system also includes a visualization device. The system also includes at least one processor and a memory operatively coupled to the gamma camera sensor, the ultrasound transducer, the tracking system, and the visualization device. The memory has instructions executable by the at least one processor, which cause the at least one processor to read a first gamma-ray photon sensing event received from the gamma sensor. The processor also executes instructions to read a second gamma-ray photon sensing event received from the gamma sensor. The processor also executes instructions to reconstruct the 3D distribution of gamma-emitting radioisotopes within the patient's body using first and second sensing events. The processor also executes instructions to determine co-registration between the ultrasound transducer and the gamma sensor using tracking information. The processor further executes instructions to determine co-registration between the 3D distribution of gamma-emitting radioisotopes and the ultrasound scan using the co-registration between the ultrasound transducer and the gamma sensor. The processor also executes instructions to deliver an image to a visualization device, the image including enhancement of the 3D distribution of gamma-emitting radioisotopes onto the ultrasound scan using the co-registration between the 3D distribution of gamma-emitting radioisotopes and the ultrasound scan.

[0009] Some embodiments provide a portable single-photon emission computed tomography (SPECT) imaging system for scanning body parts of a patient. The system includes a SPECT controller unit, which includes a computer. The system also includes a mechanical articulated arm connected to the controller unit. In some embodiments, the articulated arm may be mounted on other objects, such as a floor, ceiling, wall, track, and other fixed objects, rather than on the controller unit. The system also includes at least one gamma camera panel connected to the articulated arm, wherein the gamma camera panel includes a gamma camera sensor with position and energy sensing resolution. The gamma camera panel provides an imaging field of view greater than 15 degrees. The imaging field of view can be defined as an angular range deviating from the direction in which the gamma camera has maximum imaging sensitivity, and gamma photons can be detected and imaged from this angular range by the gamma sensor included in the gamma camera panel, the gamma sensor having a sensitivity greater than one percent of the maximum imaging sensitivity. The system also includes a tactile pressure sensor mounted on the panel. The tactile pressure sensor is operatively coupled to at least one processor and memory. Movement of the panel relative to the patient changes based on tactile pressure sensor data.

[0010] In some embodiments, the portable SPECT system uses data from an external computed tomography (CT) scanner or another medical imaging scanner (e.g., magnetic resonance imaging) to improve the quality of delivered molecular images by applying attenuation correction. In some embodiments, CT images may be co-registered with molecular images, and the combined rendering can be sent to a visualization device for user interpretation. In some embodiments, co-registration between CT and SPECT images can be accomplished by matching the patient's 3D contours. Co-registered ultrasound images can be used to aid co-registration. In some other embodiments, labels may be used for co-registration.

[0011] In some embodiments, the portable SPECT system can be co-registered with medical optical imaging equipment (e.g., endoscopes, laparoscopes) or X-ray equipment (e.g., fluorescein microscopes) to guide medical interventions such as biopsies, ablation, or surgery.

[0012] In an exemplary embodiment, a system includes a gamma-ray photon sensor with energy and position resolution sensing capabilities. The gamma-ray photon sensor can provide photon interaction locations. The system also includes an coded aperture mask placed in front of the photon sensor. The mask may include photon attenuation mask pixel elements shaped as biconical frustums, wherein the physical space between the biconical frustum mask pixel elements having a common edge is partially or completely occupied by material. The mask can generate an imaging field of view in front of the sensor. The system also includes at least one processor and a memory operatively coupled to the sensor and the processor. The memory may store instructions executable by the at least one processor, causing the processor to project a first photon interaction location onto a reference plane to create a first projected interaction point. For a direction toward the imaging field of view, the processor may also retrieve photon attenuation coefficients of the first projected interaction point stored in the memory. The processor may also project a second photon interaction location onto the reference plane to create a second projected interaction point. For a direction toward the imaging field of view, the processor may also retrieve photon attenuation coefficients of the second projected interaction point stored in the memory. The processor may also use the retrieved attenuation coefficients of the first and second photon interactions to reconstruct an image of the gamma-ray source.

[0013] In some embodiments, the sensor provides photon interaction locations in all three-dimensional spaces with a resolution better than 4 millimeters (mm). In some embodiments, the coded aperture mask is made of a material with a density greater than 10 g / cm³ (g / cc). In some embodiments, the mask pixel element is shaped as a biconical frustum, with at least one side of it forming an angle greater than 3 degrees relative to the normal on the biconical frustum substrate. In some embodiments, the mask pixel element is shaped as a biconical frustum, with at least one side of it forming an angle greater than 5 degrees relative to the normal on the biconical frustum substrate. In some embodiments, the density of the material between the biconical frustum mask pixel elements is greater than 10 g / cc. In some embodiments, the biconical frustum mask pixel element has a substrate selected from the group consisting of: a rectangular substrate, a triangular substrate, and a hexagonal substrate. In some embodiments, the shape of the biconical frustum mask pixel element is approximated by a mask pixel element having curved sides. In some embodiments, the coded aperture mask extends across multiple planes. In some embodiments, the system further includes photon attenuation shielding located in directions around the sensor not covered by the coded aperture mask. In some embodiments, the coded aperture mask has an opening fraction, defined as the proportion of the non-attenuating mask area to the total mask area, ranging from 0.1% to 70%. In some embodiments, the coded aperture mask is self-supporting. In some embodiments, the coded aperture mask is composed of multiple layers stacked together to approximate the frustum-shaped structure of the mask pixels.

[0014] In another exemplary embodiment, a method includes projecting a first photon interaction location detected by a gamma-ray photon sensor onto a first reference plane to create a first projected interaction point. In this embodiment, the gamma-ray photon sensor has energy and position resolution sensing capabilities. The gamma-ray photon sensor provides the photon interaction location. In this embodiment, an coded aperture mask is placed in front of the photon sensor. The mask includes photon attenuation mask pixel elements shaped as biconical frustums. In the mask, the physical space between the biconical frustum mask pixel elements having a common edge is partially or completely occupied by material. The mask generates an imaging field of view in front of the sensor. The method also includes retrieving photon attenuation coefficients of the first projected interaction point stored in memory for a direction toward the imaging field of view. The method further includes projecting a second photon interaction location detected by the gamma-ray photon sensor onto a second reference plane to create a second projected interaction point. The method also includes retrieving photon attenuation coefficients of the second projected interaction point stored in memory for a direction toward the imaging field of view. The method further includes reconstructing an image of the gamma-ray source using the retrieved attenuation coefficients of the first and second photon interactions.

[0015] In some embodiments, the mask used to encode incident gamma-ray photons in front of the sensor has adjustable geometry. Adjustment can be achieved by sending instructions from a computer to the actuator. Adjustment allows the mask to provide both large and narrow fields of view. This allows for large field-of-view scanning and narrowing of the structure of interest when needed. Furthermore, adjustment can change the distance between the mask and the detector. Additionally, adjustment can change the aperture fraction of the mask. In some embodiments, the mask may be made of overlapping parallel plates with partially or fully overlapping apertures. In one embodiment, the mask is made of three overlapping layers, but any number of layers is conceivable. In one embodiment, these layers are spaced apart to increase focusing capability or collimation. In some embodiments, the computer controls the arrangement of the mask elements based on the imaging task or user input.

[0016] In some embodiments, a portable SPECT system can be used both in scanning mode to create a more extensive SPECT image dataset and in real-time imaging mode to create operable images. Adjustable masks can be used to optimize these imaging modes, allowing for large field-of-view imaging (particularly suitable for scanning) and narrow field-of-view imaging (particularly suitable for real-time imaging of specific structures).

[0017] The nature and advantages of the embodiments can be better understood by referring to the following detailed description and accompanying drawings. Attached Figure Description

[0018] Figure 1 A view of a portable SPECT imaging system actuated by an articulated arm is shown.

[0019] Figures 2A-2D Views of two configurations of dual gamma camera panel systems are shown.

[0020] Figure 3 An illustration shows a portable SPECT camera system that uses a visual camera and an ultrasound probe placed between panels for co-registration.

[0021] Figure 4 An illustration shows a portable SPECT camera system that uses a visual camera co-registered with an ultrasound probe placed on the side of a panel.

[0022] Figure 5 An illustration shows a portable SPECT camera system that uses an articulated robotic arm to co-register with an ultrasound probe placed between panels.

[0023] Figure 6 An illustration shows a portable SPECT camera system that is co-registered with an ultrasonic probe placed between panels using an electromagnetic field tracker.

[0024] Figure 7 A diagram of a portable SPECT camera system co-registered with an ultrasound probe that guides percutaneous medical interventions is shown.

[0025] Figure 8 An illustration shows a portable SPECT camera system co-registered with an ultrasound probe used to correct tissue deformation during SPECT image formation.

[0026] Figure 9 An illustration shows a portable SPECT camera system placed in a configuration for scanning specific body parts of a patient.

[0027] Figure 10 An illustration shows an embodiment of a portable cart integrating a SPECT gamma camera and a medical ultrasound system mounted on an articulated arm.

[0028] Figure 11 A diagram illustrating the operational connections between components of a portable imaging system, including a SPECT gamma camera and a medical ultrasound system, is shown.

[0029] Figure 12 An embodiment is shown in which an ultrasound transducer array is registered to each other and to a gamma camera panel for combined scanning or treatment of a patient.

[0030] Figure 13 A portable SPECT system is shown for use in conjunction with a standalone medical imaging system, such as a CT scanner.

[0031] Figure 14An illustration shows a benchmark used to provide co-registration between a portable SPECT system and another medical imaging scanner.

[0032] Figure 15 The illustration shows body-fitting clothing that patients can use to assist in the optical-based computer vision tracking and mapping process.

[0033] Figure 16 The process workflow for importing other imaging datasets to achieve multimodal image fusion and improve SPECT reconstruction is shown.

[0034] Figure 17 A cross-sectional side view of a large field-of-view coded aperture imaging system is shown.

[0035] Figure 18 A front view of a large field-of-view coded aperture mask is shown.

[0036] Figure 19 A cross-sectional view of the mask element is shown.

[0037] Figure 20 A cross-sectional side view of a large field-of-view coded aperture imaging system is shown, illustrating the coded aperture mask extending across multiple planes.

[0038] Figure 21 A cross-sectional side view of a large field-of-view coded aperture imaging system is shown, illustrating how a reference plane can be used to back-project detected events into the image space for image reconstruction.

[0039] Figure 22 A cross-sectional side view of a large field-of-view coded aperture imaging system is shown, illustrating how detected events can be back-projected into image space for image reconstruction using a reference plane that coincides with the mask plane.

[0040] Figure 23 A cross-sectional side view of a large field-of-view coded aperture imaging system is shown, illustrating how multiple reference planes can be used to back-project detected events into the image space for image reconstruction.

[0041] Figure 24 A perspective view of a large field-of-view coded aperture mask is shown, illustrating one configuration in which it is made of multiple layers stacked together.

[0042] Figure 25 A cross-sectional side view of a large field-of-view coded aperture imaging system is shown, which employs sensors arranged in different planes to minimize the range of gamma-ray photon incident angles falling on the sensors.

[0043] Figure 26A top view shows an arrangement of four sensor panels on different planes to minimize the range of incident angles of gamma-ray photons falling on the sensors.

[0044] Figure 27 A cross-sectional side view of an coded aperture imaging system with adjustable 3-layer masks is shown, in a wide field-of-view configuration.

[0045] Figure 28 A cross-sectional side view of an coded aperture imaging system with adjustable 3-layer masks is shown, in collimated (central concave) field-of-view configuration.

[0046] Figure 29A and 29B A schematic top view of a coded aperture mask is shown, which is made of nine panels arranged in three layers in two configurations: a wide field of view and a collimated field of view.

[0047] Figure 30A and 30B A schematic top view of a coded aperture mask is shown, which is made of 19 panels arranged in three layers in two configurations: a wide field of view and a collimated field of view.

[0048] Figure 31 A top view of the middle layer in an adjustable mask with a pseudo-random pattern of 22% open fraction is shown.

[0049] Figure 32 A top view of an coded aperture mask including curved slits of various curvatures is shown.

[0050] Figure 33 An embodiment of a handheld SPECT camera is shown, illustrating the positions of a large field-of-view coded aperture mask and a sensor.

[0051] Figure 34 A top view of an embodiment of a handheld SPECT camera is shown, illustrating the location of the large field-of-view coded aperture mask and sensor.

[0052] Figure 35 A view of an embodiment of a handheld SPECT camera is shown, which has a large field-of-view coded aperture to scan the patient's body from a reduced angular range around the patient while providing sufficient angular sampling of the image space.

[0053] Figure 36 A flowchart summarizing some systems and methods implemented by portable molecular imaging systems is shown. Detailed Implementation

[0054] Single-photon emission computed tomography (SPECT) alone, or in combination with computed tomography (CT) (SPECT / CT), is a primary molecular imaging modality used for medical diagnostic imaging. Most commonly, SPECT imaging devices consist of an array of gamma-ray sensors orbiting or orbiting the patient's body. During the imaging scan, the patient typically lies on a table, and for some cardiac imaging systems, may sit in a custom-made chair. A parallel-aperture collimator is typically used in front of the detector array to limit the direction that gamma-ray photons can take before interacting with the position-sensitive sensors. This produces a parallel projection of the distribution of gamma-ray radioisotopes within the patient's body. Computer programs are used to reconstruct this 3D distribution using analytical or iterative image reconstruction algorithms.

[0055] The sensor and associated collimator can be placed at a relatively large distance from the patient, thus accommodating patients of various sizes. Because SPECT imaging sensors are characterized by limited angular resolution, a large gap between the sensor and the radiolabeled molecules can translate into lower imaging resolution and sensitivity. A reduced gap will result in improved sensitivity and resolution. Some SPECT imaging systems include sensors that can be driven to reduce the gap to the body. Such systems can use a parallel-aperture collimator with changing orientation to capture projections at different angles. This allows for a reduction in the gap between the patient's body and the sensor, which can improve performance.

[0056] Computed tomography (CT) scans can be combined with SPECT imaging to provide a 3D morphological background to the molecular images provided by SPECT. Through co-registration of the two 3D images, radiologists can identify organs with increased radiation uptake. Furthermore, by providing photon attenuation maps, CT images can enable more accurate SPECT image reconstruction, as photon attenuation maps allow SPECT image reconstruction algorithms to account for photon attenuation between image voxels of the SPECT imaging sensor.

[0057] Examples include an improved SPECT imaging device that is portable and allows for co-registration with ultrasound. The SPECT imaging device described herein can be used to guide medical interventions, such as biopsies and ablation treatments, and also to guide surgery. This improved SPECT imaging device also provides co-registration with other medical imaging modalities, such as X-ray, CT, and various optical imaging modalities.

[0058] In other respects, the SPECT imaging device described herein provides portable, high-performance molecular imaging using molecular reagents labeled with SPECT radioisotopes. The SPECT imaging device described herein offers a novel multimodal imaging system and corresponding methods that combine SPECT imaging with ultrasound imaging and other medical imaging modalities to achieve a variety of medical applications, such as diagnostic imaging, biopsy guidance, ablation therapy guidance, and surgical guidance.

[0059] These and other advantages in one or more aspects will become apparent upon consideration of the following description and accompanying figures.

[0060] Portable SPECT imaging systems and related methods can provide high-resolution and sensitive imaging of the distribution of radiotracers within a patient by employing at least one specially designed gamma camera panel mounted on an articulated robotic arm. In some embodiments, the gamma camera panel utilizes a gamma photon imaging modality that provides the distance from the gamma camera to the location of the gamma radiotracer, even when the gamma camera scans the patient from a substantially stationary position. In some embodiments, the articulated robotic arm is a 6-axis robotic arm. In some embodiments, the articulated robotic arm is actuated by a user by applying direct force or by using a computer. In some embodiments, the robotic arm can be computer-actuated to perform automated, patient-specific examinations, such as whole-body scans, head scans, neck scans, cardiac scans, trunk scans, etc. For navigational purposes, computer vision systems can be used to provide information about the patient's position and orientation, create a 3D model of the patient's body, and identify specific body parts. Tracking systems can be used to determine the position of SPECT system components relative to each other and relative to the patient. The 3D model of the patient can be combined with tracking information to guide the SPECT scan. In some embodiments, the SPECT system may be mounted on motorized wheels, which can be actuated during scanning to extend the coverage of the gamma camera panel around the patient.

[0061] The embodiments also include multimodal imaging systems and methods that co-register a portable SPECT imaging system with other imaging modalities. In one embodiment, a tracking system is used to co-register an ultrasound probe and corresponding ultrasound images with the portable SPECT imaging system to enable molecular imaging to be combined with ultrasound-guided medical interventions, molecular imaging with tissue deformation correction, and molecular image-guided ultrasound examinations. In one embodiment, the portable SPECT imaging system includes two physically separate gamma camera panels that can be actuated to leave space between them and are oriented at various angles relative to each other. In yet another embodiment, another imaging probe, such as an ultrasound probe, is substantially positioned between the two gamma camera panels, and at least the imaging fields of the gamma camera panels and another medical imager overlap. This embodiment allows the user to visualize the SPECT images co-registered with the ultrasound images in real time. Other medical devices may be placed in the space between the two panels. Examples of such medical devices are percutaneous biopsy devices and ablation therapy devices. The use of these instruments can be guided by SPECT and / or ultrasound images.

[0062] The proposed portable multimodal imaging SPECT and ultrasound system and method offer several advantages over existing multimodal SPECT / CT and PET / CT systems, including: elimination of CT-related radiation doses by providing co-registered anatomical information delivered by ultrasound; portability provided by a much smaller robotic SPECT and ultrasound system; increased molecular imaging resolution and sensitivity by performing imaging scans with a gamma camera panel placed closer to the patient's body; precise and direct guidance for medical interventions; use in the operating room to guide surgery; and improved molecular image quality by using real-time ultrasound to guide molecular image correction that takes into account tissue deformation and organ movement captured by the ultrasound transducer.

[0063] In other respects, embodiments of the portable SPECT imaging system and related methods also allow scanning of a patient from a limited angular range around the patient without sacrificing imaging performance. Embodiments providing imaging resolution from a substantially static position in a direction perpendicular to the surface of the gamma camera panel via a gamma camera panel offer this advantage over existing systems. This advantage is also provided by embodiments where the gamma camera panel provides an imaging field of view greater than 15 degrees, preferably close to 45 degrees. The imaging field of view is defined as the angular range from which gamma photons can be detected and imaged by the gamma sensor included in the gamma camera panel, at least in a direction deviating from the normal of the gamma camera panel. Scanning a patient from a limited angular range, while generally not preferred, can be limited by various operational and specific imaging tasks, such as the need to reduce scan time, restrict physical access to the imaging sensor panel around the patient, or increased attenuation or attenuation inhomogeneity of radiation in certain directions around the patient, which may lead to increased imaging artifacts.

[0064] Image co-registration between the portable SPECT system, other medical imaging modalities, and medical devices can be achieved using a position tracking system that captures the relative positions of the gamma sensor, other medical imaging sensors (e.g., ultrasound transducers), and medical devices relative to each other and relative to the patient's body. A range of position tracking systems can be used individually or in combination. Such systems can utilize inertial measurement units (IMUs), optical systems such as RGB cameras, depth imaging sensors, infrared cameras, stereo optics systems, electromechanical articulated arms, electromagnetic field trackers, ultrasonic trackers, servo motors, or any other device suitable for providing the position and orientation of the part of interest with appropriate accuracy.

[0065] In some embodiments, the gamma camera panel includes a gamma photon attenuation imaging element positioned in front of a position-sensitive gamma-ray sensor. The photon attenuation imaging element can be selected from the group consisting of a coded aperture mask, straight and / or tilted parallel-aperture collimators, pinhole collimators, or multi-pinhole collimators. In a preferred embodiment, the gamma-ray sensor provides the location of gamma photon interactions in all three dimensions with a resolution better than 2 millimeters (mm). In some embodiments, the photon attenuation imaging element is a coded aperture mask with a field of view greater than 20 degrees. Preferably, the pattern of the coded aperture mask minimizes sidelobes in the instrument's autocorrelation function (see, for example, Fenimore, Edward E., and Thomas M. Cannon. "Codedaperture imaging with uniformly redundant arrays." Applied optics 17.3 (1978): 337-347). Patterns that repeat under different translations without rotation or reflection are not recommended because they may produce peaks in the sidelobes of the autocorrelation function for the magnification of interest, resulting in reconstruction artifacts. Similarly, creating openings with straight, long slits, especially multiple straight, long slits parallel to each other, whose length can be substantially the same as or a considerable portion of the mask width, can produce image artifacts for this application. Encoded aperture patterns can have an aperture fraction, defined as the ratio of empty mask pixels to the total number of pixels, ranging from near 0% to near 100%. In some embodiments, the aperture fraction can range from 0.1% to about 80%. In some embodiments, the aperture fraction can range from 5% to 30%. This aperture fraction can maximize the image signal-to-noise ratio for certain distributions of molecular SPECT reagents in humans. In some embodiments, an adjustable mask can deliver a range of aperture fractions, for example, from 5% to 70%, by adjusting the mask elements. For example, a mask comprising overlapping layers with overlapping or partially overlapping openings (holes) can provide this adjustability by laterally moving these layers relative to each other. Other combinations of mask elements can produce the same effect.

[0066] In some SPECT imaging systems, the sensor and associated collimator can be placed at a relatively long distance from the patient, thus accommodating patients of various sizes. Because SPECT imaging sensors are characterized by limited angular resolution, a large gap between the sensor and the radiolabeled molecules can translate into lower imaging resolution and sensitivity. Reducing the gap can lead to improved sensitivity and resolution. Some SPECT imaging systems include sensors that can be actuated to reduce the gap to the body. Such systems may also include a parallel-aperture collimator that changes orientation to capture projections at different angles. This further reduces the gap between the patient and the body, and improves sensor performance.

[0067] The embodiments include optional imaging and sensing modalities that allow the imaging sensor assembly to be closer to the patient's body while appropriately sampling the image space to provide higher 3D image sensitivity and resolution. These techniques eliminate the need for uniformly acquiring sensor data around the patient by allowing efficient sampling of multiple projection angles from a narrowed (tomographic) range of angles around the patient.

[0068] In other aspects, the embodiments allow for the construction of a high-resolution 3D map of the distribution of radioisotopes within a patient's body by scanning the patient from a narrowed perspective around the patient and placing a radiation sensor near the patient. This results in improved imaging sensitivity and resolution. Therefore, several advantages of one or more aspects are that high-resolution and high-efficiency SPECT imaging is possible by employing compact SPECT imaging sensors and systems that can be made portable or even handheld.

[0069] In other aspects, embodiments include imaging systems and related methods capable of imaging the distribution of radiotracers within a patient in 3D with high resolution and sensitivity using a sensor assembly that scans the patient from a location close to the patient and from directions that may only cover a limited angular range around the patient. Using a sensor close to the patient allows for better imaging resolution and, in some cases, better imaging sensitivity, as in the current method. Scanning the patient from a limited angular range, while generally not preferred, can be subject to various operational limitations, such as limited scan time, limited physical access to the imaging sensor around the patient, or increased attenuation or attenuation inhomogeneity of radiation in certain directions around the patient, which may lead to increased imaging artifacts.

[0070] A key prerequisite for this sensing imaging system is a large imaging field of view in order to overcome the limitations of scanning patients from a limited range of directions. At the same time, providing high imaging resolution and high sensitivity is important for this imaging system, which previously required a narrow field of view.

[0071] This embodiment enables SPECT imaging using an imaging sensor that provides a large imaging field of view and high imaging resolution and sensitivity. The imaging sensor includes a custom-designed large-field-of-view radiation attenuation mask placed in front of a radiation position-sensitive sensor, and a processing unit that allows for efficient and effective image reconstruction techniques, resulting in reconstructed images with excellent image resolution, signal-to-noise ratio (SNR), and sensitivity. For imaging gamma-ray photons, the attenuation mask can be made of a high-density, high-atomic-number Z-material, such as tungsten or tungsten alloys.

[0072] In some embodiments, the mask may be a coded aperture mask, wherein the coded elements are shaped to allow for large field-of-view imaging. A coded aperture pattern comprises a combination of radiation-attenuated pixels and empty, non-attenuated pixels. In this specification, attenuated pixels will be referred to as mask pixels, and non-attenuated pixels will be referred to as empty mask pixels. Examples of coded aperture patterns that can be used in a mask are: uniformly redundant arrays, modified uniformly redundant arrays, pseudo-random arrays, random arrays, or any other pattern. In some embodiments, the coded aperture pattern minimizes sidelobes in the instrument response function. A coded aperture pattern may have an aperture fraction, defined as the ratio of empty mask pixels to the total number of pixels, ranging from near 0% to near 100%. However, the most useful aperture fractions are in the range of 1% to about 50%.

[0073] To properly attenuate gamma-ray photons in the 120-170 keV range, which corresponds to specific gamma-ray energies in many SPECT isotopes, the attenuation mask may need to have a thickness in the same range as (if not larger than) the size of the mask element. For example, a mask made of tungsten alloy with a thickness of about 2 mm can be used to attenuate 140 keV photons, which is within the same range as the size of the mask pixel element.

[0074] Using mask pixel elements shaped as rectangular parallelepipeds—that is, having straight sides and a single planar lateral surface for each Cartesian direction in the mask plane—offers poor performance; when assuming a single planar mask and square mask pixels, only adequate encoded contrast can be achieved for photons at low incident angles of up to 10-20 degrees to the normal on the mask plane. Due to this limitation of cuboid mask pixels, embodiments provide custom-shaped mask pixel elements.

[0075] In some embodiments, custom geometries are used to design the sides of the pixel elements. These custom geometries include non-straight sides and multiple planar sides for each of two Cartesian directions orthogonal to the mask plane in a conventional mask. Curved sides may also be used instead of multiple planar sides.

[0076] The aforementioned sides can be combined with mask pixel elements of different geometries, such as square, triangular, and hexagonal pixels, where each side of a triangular or hexagonal prism can contain multiple planar or curved sides. These geometries can provide high-contrast image encoding for radiating photons over a large range of incident angles relative to the mask plane normal. For example, such incident angles can reach 50 degrees from the normal to the mask plane, if not more. Therefore, the custom shaping of mask elements within a single mask can combine edges, ridges, and curvatures of various geometries and orientations.

[0077] In some specific embodiments, square or rectangular mask pixel elements in the mask array can be constructed from, for example, bifurcations (or double pyramids) having a square or rectangular base. The bifurcations may be centrally symmetrical with respect to the rectangular base, symmetrical with respect to the plane of the rectangular base, or may not have such symmetry. The shape of the bifurcation mask element can vary across the entire mask. When two mask pixels share a common edge, the space between two adjacent bifurcations can be filled with a decaying material. Curved sides can be used to approximate the shape of the bifurcation instead of planar sides. Similarly, multiple thin-layer masks can be stacked together to approximate the shape of the bifurcation mask element and the filling material between adjacent bifurcations.

[0078] Dual masks with triangular or hexagonal bases can be used to mask triangular or hexagonal pixel elements, respectively. The bifurcation can be centrally symmetric with respect to the triangular or hexagonal base, symmetric with respect to the plane of the triangular or hexagonal base, or may not have this symmetry. The shape of the dual mask elements can vary across the entire mask. In this case, a fill attenuation material can also be used between mask elements sharing a common edge.

[0079] Masks can be arranged in one or more planes, which can be parallel or non-parallel to each other. Shielding can be used to cover portions of the space around the sensor not covered by the coded aperture mask to limit the detection of photons from other angles not covered by the coded aperture.

[0080] To provide better imaging resolution, position-sensitive sensors can capture all three coordinates of radiative interactions within the sensor at high resolution, for example, better than 1 or 2 mm in all three dimensions. Furthermore, the sensor can provide radiation detection with high energy resolution, as this allows for better selection of events that have not been scattered before reaching the mask and sensor. Such sensors can be scintillators, semiconductors, or other types of radiation detectors capable of providing the location of radiative interactions.

[0081] To reconstruct the 3D distribution of radioactive molecules, an image reconstruction analysis package is used to process the sensor data. In a particular embodiment, an attenuation map is pre-computed to correlate the sensor location with the distribution of incident gamma-ray photon angles. This distribution is weighted by the probability that these photons will pass through the coded mask. The detail of the edges of each mask pixel described adds to the information bandwidth of the mask encoding. Therefore, the attenuation map can be complex and can be pre-computed using high-resolution ray tracing and / or Monte Carlo simulations. In a particular embodiment, the attenuation map is pre-computed at least on a plane referred to as the reference plane. For each point within the reference plane, the attenuation map includes an attenuation coefficient or other information that can be used to extract the attenuation coefficient, such as the path length through the mask material, the predetermined type of radiation, and various angles in the imaging field of view. The calculation of the attenuation map can use ray tracing methods, simulation methods (e.g., Monte Carlo simulations), or a combination thereof to determine the radiation path through the shielding and mask assembly from various origin angles in the field of view. For a specific type of radiation, the attenuation factor can be calculated from the path values.

[0082] For each radiation detected by the radiation sensor, the interacting 3D locations are first projected onto a reference plane using various sampling techniques. Sampling is focused on directions toward the coded mask's field of view, and this projected location on the reference plane is used to determine the attenuation coefficients across the mask along these directions. If the path of a photon along a certain direction may include fragments of other material captured by the sensor or not pre-calculated by the attenuation coefficient, additional attenuation calculations can be added to appropriately scale the intensity of the backprojection along that direction. The backprojection process can determine the probability density function (pdf) of the intensity in the detected event or detected bin. The pdfs of at least two detected radiation events can be used with any suitable iterative or analytical image reconstruction algorithm known in the art. Image reconstruction analysis can be performed in list mode or bin mode. If analysis is performed in bin mode, binning can be performed across multiple planes, and each detected photon can be added to the spatially nearest plane.

[0083] In some implementations, the attenuation coefficients can also be dynamically calculated during image reconstruction analysis using a fast path estimator (e.g., ray tracing methods). However, pre-computation of these coefficients can provide optimal image reconstruction processing speed performance.

[0084] Figure 1A diagram of a portable SPECT imaging system is shown, comprising a mechanical articulated arm, such as a robotic arm. An instrument controller (100) is connected to at least one gamma camera panel via the mechanical articulated arm (101). In the depicted embodiment, two gamma camera panels (102a) and (102b) are used. Panels (102a) and (102b) can be attached to each other and to the articulated arm via a joint rod (103). This joint rod allows the two panels to move relative to each other and relative to the articulated arm, and allows the panels to change their relative orientation, particularly their tilt, pitch, and yaw. Thus, the relative angle between the panels can be modified. A computer vision camera (106) can be attached to the articulated arm (101), the rod (103), or connected to another component of the portable SPECT system via a connector (107). This computer vision camera (106) can be used to monitor the gamma camera panels scanning the entire area of ​​the patient. The computer vision camera may include an RGB camera, an optical camera, an infrared camera, a depth imaging optical camera, a structured light camera, a stereo optical camera, a time-of-flight optical camera, a terahertz emitter-sensor assembly, a lidar scanner, an ultrasonic emitter-sensor assembly, another tracking and / or mapping sensor, or a combination thereof. The purpose of the camera is to determine the extent of the patient's body so that movement of the gamma camera panel does not collide with the patient's body. In some embodiments, an ultrasonic scanner or an electromagnetic scanner, such as a terahertz imaging scanner, may be used additionally, or in place of an optical camera, to scan the contours of the patient's body.

[0085] Using this portable SPECT imaging system, a patient scan can be performed by storing instructions in the memory of a computer operatively coupled to actuating components such as a robotic arm (101), a lever (103), and gamma sensors comprised of gamma cameras (102a) and (102b), and a computer vision camera (106), for: reading and analyzing computer vision camera data to determine the contours of the patient's body and to determine the relative position and orientation of the computer vision camera (106) relative to the patient's body; articulating the arm (101) and lever (103) such that the gamma camera panels (102a) and (102b) move to relevant positions around the patient; acquiring data from the sensors included in the gamma cameras (102a) and (102b); spatially registering the gamma sensor data relative to the patient's body, and using the spatially registered sensor data to create a 3D image of the distribution of radioactive tracers within the patient's body. The gamma sensor provides the position and energy of gamma photon interactions within the sensor at high resolution. Preferably, the gamma sensor provides the location of 3D gamma event interactions with a resolution better than 2 mm, ideally with a resolution better than 0.5 mm.

[0086] In some embodiments, the controller may include wheels (108a) and (108b). In such embodiments, the memory may be operatively coupled to at least one motor actuating the wheels (108a) and (108b) to move the controller (100). This may allow the system to scan a larger area over a distance greater than that allowed by the articulated arm (101). For navigation purposes and obstacle avoidance, other sensors (109) may be placed at one or both ends of the controller to ensure that there are no obstacles in the system's path. In such embodiments, the memory may also be operatively coupled to the sensors (109) to guide the actuation of the wheels (108a) and (108b). The sensors (109) may be optical sensors, lidar scanners, depth imaging sensors, ultrasonic sensors, or any other sensors capable of detecting obstacles.

[0087] In some embodiments, panels (102a) and (102b) may include proximity sensors (110a) and (110b), preferably placed on the panel surface facing the patient, to obtain information about the proximity between the panel and the patient's body or between the panel and other objects. Sensors (110a) and (110b) may be part of a computer vision system and are capable of providing a 3D model of the patient's body just below the sensor panels in real time. The computer can use this map to adjust the movement of the scanner over the patient's body and provide an estimate of the attenuation map used during image reconstruction. These proximity sensors may be operatively coupled to a computer that is also connected to an actuator that moves the panel relative to the patient. The computer can use a program to alter or stop the scanning process to maintain the distance between the panel and the patient or other objects within a desired range. The proximity sensors may be capacitive, inductive, magnetic, ultrasonic, optical, terahertz, X-ray backscattering, or any other sensor capable of providing distance to an object.

[0088] In some embodiments, the mechanical articulated arm (101) can be actuated by a user. In this case, the purpose of the articulated arm is to support the weight of the gamma camera panels (102a) and (102b) and potentially determine the position and orientation of the panels relative to the body of the controller (100). Furthermore, in some embodiments, the joints included by the rod (103) can be actuated by a user to manually position the gamma camera panels in a desired location and orientation. In some embodiments, the user can actuate the arm (101) and rod (103) by applying a direct force. In some embodiments, the user can actuate the arm (101) and rod (103) via a computer operatively coupled to an electric actuator mechanically connected to the arm (101) and rod (103).

[0089] A computer including at least one processor and a memory operably coupled to a computer vision camera (106) and a gamma camera sensor consisting of gamma camera panels (102a) and (102b) can be used to read gamma sensor data and computer vision camera data to: read a first gamma-ray photon sensing event received from the gamma sensor, provide a first position and orientation of the gamma camera panel (102a) or (102b) sensing the first photon relative to the patient body (104), co-register the first gamma-ray photon sensing event with the patient body (104) using the first position and orientation, read a second gamma-ray photon sensing event received from the gamma sensor, provide a second position and orientation of the gamma camera panel relative to the patient body (104), co-register the second gamma-ray photon sensing event with the patient body (104) using the second position and orientation, and reconstruct the 3D distribution of radioactive isotopes emitting gamma rays within the patient body by using the first and second co-registered sensing events.

[0090] Co-registration of the sensed event with the patient's body can be achieved by analyzing computer vision camera frames using computer vision methods known in the art to determine the pose estimate of the computer vision camera (106) relative to the patient's body. Furthermore, the relative position and orientation between the camera and the gamma camera panel (102a) or (102b) can be achieved through direct observation of the panel by the camera (106) or by obtaining joint state information of the joints in the rod (103) and / or the potential articulated arm (101). Combinations of the two relative poses (computer vision camera-patient and computer vision camera-sensing gamma camera panel) can be combined to obtain co-registration of the sensed event with the patient's body. Other tracking and co-registration systems and methods can be used, some of which are described elsewhere in this specification.

[0091] Computer vision programs can periodically analyze computer vision camera frame data during a SPECT scan to create an updated 3D model of the patient's body. By monitoring changes from frame to frame, it becomes possible to detect changes in body posture and deformities. These detections can be used to improve the quality of reconstructed SPECT images by taking these body changes into account, to stop the scanning process to avoid collisions between any parts of the SPECT imaging system and the patient or other users, or to notify the user of significant body changes that may require resetting the SPECT scan.

[0092] In this way, the computer connected to the camera (106) can use computer vision methods to create 3D models of the patient's body at regular intervals and detect changes and deformations in the body from one 3D model to another. By analyzing the magnitude and type of the changes in the body, the computer can notify the user of significant changes in the body that may require resetting the SPECT scan, because co-registration between detected events occurring before and after the body modification may be unreliable.

[0093] Furthermore, the computer can be operatively coupled to a gamma camera sensor and can determine a first 3D patient body model and assign it to a first sensor detection event, determine a second 3D patient body model and assign it to a second sensor detection event, create a tissue deformation model from the first to the second 3D patient body models, and perform reconstruction of the 3D distribution of radioactive isotopes emitting gamma rays within the patient body by using the first and second sensing events and the tissue deformation model.

[0094] In one example of this reconstruction, the computer uses an image space remapping method to create a correspondence between image elements or image nodes before and after the body deformation, and uses two detected gamma event backprojection operators to illustrate the remapped image elements or nodes. Various remapping algorithms known in the art can be used. The computer, operatively connected to the computer vision camera, can also be operatively connected to actuators that power the robotic arm (101), rod (103), and wheels (108a) and (108b). When the computer vision algorithm detects significant changes in the patient's body, or even potentially indicating a collision, the computer can be programmed to stop the movement of the articulated arm and gamma camera panel to avoid collisions between any part of the SPECT imaging system and the patient's body. Furthermore, the computer vision subsystem can monitor the space in the projection path of the gamma camera panel during scanning to detect other objects or people. When such detection occurs, the computer can stop the movement of the articulated arm and gamma camera panel to avoid collisions between any part of the SPECT imaging system and other objects or people.

[0095] In some embodiments, the articulated arm (101) may be attached to another physical object rather than directly to the body of the controller (100). Examples of such objects include: floors, ceilings, walls, other portable controllers, and railings.

[0096] In some embodiments where the user actuates one or more of the articulated arms, lever arms, and gamma camera panel assemblies to perform static scans (e.g., the gamma camera panel does not move relative to the patient) or dynamic scans (e.g., the gamma camera panel moves relative to the patient), a camera (106) and / or other tracking modalities external to the articulated arms may not be used. In such embodiments, co-registration of sensing events with the patient's body can be accomplished by reading and analyzing positional data of any joints or actuators involved in the movement of the gamma camera panel within the room, such as the articulated levers (103), articulated arms (101), and wheels (108a) and (108b). This co-registration modality assumes that the patient's body remains stationary relative to the room in which the gamma camera panel is moved to perform the scan. In some embodiments, a bed (105) or chair on which the patient lies or sits may be moved relative to the controller to increase the effective range of the panel during the scan.

[0097] In some embodiments, the gamma camera panels (102a) and (102b) include gamma sensors that provide sensor data containing information used by an image reconstruction program running on a computer to provide imaging of the distribution of the radiotracer at a limited resolution along the direction most sensitive to either gamma camera, even when the gamma camera panels are in a substantially static position relative to the patient. In some embodiments, the limited resolution is less than 20 mm for a distance range covering at least 50 mm.

[0098] In some embodiments, each of the gamma camera panels (102a) and (102b) provides an imaging field of view greater than 15 degrees, preferably close to 45 degrees. In some embodiments, the field of view may be greater than 20 degrees, 25 degrees, 30 degrees, 35 degrees, 40 degrees, or 45 degrees. The imaging field of view is defined as the angular range relative to the direction from which the gamma camera has maximum imaging sensitivity, from which gamma photons can be detected and imaged by a gamma sensor included in the gamma camera panel, the gamma sensor having a sensitivity greater than one percent of the maximum imaging sensitivity. This imaging field of view allows the gamma camera sensor to capture imaging information from multiple directions, which makes it possible to better cover the imaging projection angle, even from a static position or from a narrowed area around the patient.

[0099] In some embodiments, each of the gamma camera panels (102a) and (102b) includes a gamma photon attenuation imaging component positioned in front of a position-sensitive gamma-ray sensor. The photon attenuation imaging component can be selected from the group consisting of an coded aperture mask, straight and / or tilted parallel-aperture collimators, pinhole collimators, or multi-pinhole collimators. In a preferred embodiment, the gamma-ray sensor provides the location of gamma photon interactions with a resolution better than 2 mm in all three-dimensional spaces. In a preferred embodiment, the photon attenuation imaging component is an coded aperture mask with a field of view greater than 30 degrees. Preferably, the pattern of the coded aperture mask minimizes sidelobes in the instrument response function. The coded aperture pattern can have an aperture fraction, defined as the ratio of empty mask pixels to the total number of pixels, ranging from near 0% to near 100%. In some embodiments, the aperture fraction can range from 1% to about 50%.

[0100] In some embodiments, the gamma sensors included in the gamma camera panels (102a) and (102b) are selected to detect higher-energy gamma photons with higher sensitivity, such as those above 200 keV or 500 keV. In this case, instead of using gamma photon attenuation imaging components, the gamma-ray Compton scattering mechanism can be used to provide imaging information. The gamma sensor can be selected to provide 3D positional resolution better than 2 mm and energy resolution better than 4%. In this case, a computer operatively coupled to the gamma sensor will determine the scattering angles around the scattering directions of gamma rays interacting at least twice in the sensor system by resolving the kinematics of gamma-ray interactions within the sensor system. These scattering angles around the scattering directions will be used to create spatially registered cones. Then, by statistically resolving the intersections of at least two spatially registered cones, a 3D map of the gamma-ray source can be constructed. This imaging modality can be used for positron emission tomography (PET) radioisotope imaging.

[0101] In some embodiments, the articulated robotic arm (101) is a 6-axis robotic arm. A computer operatively coupled to actuators within the robotic arm actuates the joints to perform a SPECT scan by moving a panel around the patient's body. A computer vision process can run on the computer to analyze image frames from a computer vision camera (106) or from another patient 3D scanner to build a 3D model of the patient and identify individual body parts. Instructions can be stored on the computer to use the output of the computer vision process to actuate the robotic arm to position the panel, thereby performing specific types of imaging scans, such as head scans, neck scans, whole-body scans, cardiac scans, trunk scans, etc.

[0102] In other embodiments, the portable SPECT imager may include a single gamma camera panel attached to an articulated robotic arm. In other embodiments, the portable SPECT imager may include three, four, or more gamma camera panels, which are directly or indirectly connected to a multi-arm robotic arm. Figure 1 The rod is connected to the jointed robotic arm.

[0103] In some embodiments, a computer vision camera may be mounted on a gamma camera panel (102a) and (102b), a rod (103), or other object. Like the computer vision camera (106), this camera can provide data analyzed by a computer to determine the contours of the patient's body and to determine the relative positions and orientations of various SPECT system components relative to each other or relative to the patient's body.

[0104] Following a SPECT scan delivering a 3D distribution of gamma-emitting radioisotopes, a computer operatively connected to an ultrasound probe or transducer, a tracking system for tracking the ultrasound probe relative to the patient's body or relative to a fixed reference point, a memory storing the 3D distribution of gamma-emitting radioisotopes co-registered with the patient, and a visualization device are available for use with the ultrasound tracking system relative to the patient or relative to the fixed reference point. The computer can also use ultrasound tracking information to determine co-registration between the 3D distribution of gamma-emitting radioisotopes and the ultrasound scan, and deliver to the visualization device an image including features of the 3D distribution of gamma-emitting radioisotopes enhanced onto the ultrasound scan. Furthermore, the computer can analyze real-time ultrasound images to create a tissue deformation model by tracking specific features in the ultrasound images from a first ultrasound frame to a second ultrasound frame. This deformation model can be used to remap the original SPECT image stored in memory to create a modified SPECT image integrating the modeled tissue deformation. This modified SPECT image can then be enhanced onto a second ultrasound frame.

[0105] Other tissue deformation correction methods can be used, such as frame-based registration methods and gamma sensor data-driven methods, such as distributed centroids (e.g., detecting body motion based on changes in the centroid of the distributed trajectory over time intervals).

[0106] In some embodiments, the platform (105) may also include a lever arm (110) that extends beyond the body of the platform when the articulated arm is extended to mitigate the possibility of the instrument tipping over. Such an arm may include a wheel (111) at its distal end.

[0107] Although it should be understood that an ultrasonic probe is a structure that includes an ultrasonic transducer, the terms "ultrasonic probe" and "ultrasonic transducer" are used interchangeably in the instruction manual.

[0108] In some embodiments, other types of sensors may be used instead of a gamma-ray panel mounted on a robotic arm. For example, one or both of panels (102a) and (102b) may include magnetic sensors or arrays of magnetic sensors. Such magnetic sensor systems are capable of measuring magnetic signals or changes in magnetic signals that reflect the magnetism of tissue. In some embodiments, the magnetism of the tissue is defined by a molecular reagent injected into the patient. In some embodiments, these molecular reagents may be labeled with magnetic nanoparticles. The panel may be moved across the patient's body to acquire magnetic signals from different locations. A three-dimensional (3D) map of the drug distribution can be reconstructed by an operatively connected computer using magnetic signal data spatially registered from the magnetic sensor assembly. The magnetic signal data may be spatially registered by a computer using tracking data from a computer vision system or from robotic arm kinematics. While examples include molecular imaging guidance using gamma sensors for molecular imaging and intervention, it should be understood that the methods and systems described herein can be conceived using sensor panels that include magnetic sensor assemblies for molecular imaging. These magnetic sensor systems can be used in conjunction with the injection of molecular drugs with specific magnetic properties into the patient.

[0109] Figures 2A-2D Multiple views of the SPECT camera head are shown, including gamma camera panels (200a) and (200b) connected to a mechanical rod (201) via robotic arms (202a) and (202b), respectively. A computer vision camera (203) can be seen connected to the distal end of a mechanical articulated arm (205) via a connector (204).

[0110] Figure 2A Two panels (200a) and (200b) in a closed configuration are shown.

[0111] Figure 2B Two panels (200a) and (200b) in a separated configuration are shown, where the relative distance between the panels increases and the roll of the panels has been altered. This separated configuration provides space between the two panels for the introduction of other imaging probes or medical devices.

[0112] Figure 2C and 2D Side views of two configurations of the SPECT camera head are shown. The assembly, made of rod (201) and arms (202a) and (202b), allows the gamma camera panels (200a) and (200b) to move laterally (away from each other) and from front to back.

[0113] Figure 2C The gamma camera panel is depicted in a forward-facing configuration.

[0114] Figure 2DA gamma camera panel in a rearward configuration is depicted, with the panel's center of gravity closer to the connector placement between the rod (201) and the articulated arm (205). Despite Figure 2C The configuration allows for the placement of additional imaging probes between the two gamma camera panels by providing blank space above and between the gamma camera panels, but Figure 2D This configuration allows other imaging probes to be placed on the sides of both panels.

[0115] Various configurations and relative placements of the gamma camera panels can be used not only to accommodate other medical devices or imaging probes that will be introduced into the gamma camera's field of view, but also to allow for varying degrees of overlap in the imaging fields of view between the two panels, and to allow for patient scanning by closely tracking the patient's anatomy.

[0116] Figure 3 An illustration of a SPECT camera head used in conjunction with another medical imaging probe is shown. Two gamma camera panels (300a) and (300b) are shown mechanically connected to the distal end of a mechanical articulated arm (302) via a rod (301). The system is shown scanning a patient (303). In this embodiment, an ultrasound imaging probe (304) is introduced into the space between the two panels (300a) and (300b). The imaging field of view of the ultrasound imaging probe (304) may partially or completely overlap with the imaging field of view of one or both of the panels (300a) and (300b).

[0117] A computer vision camera (305) is connected to a rod (301) and an articulated arm assembly (302) via a connector (306). The computer vision camera may have an observation field covering the general area of ​​the patient scanned by the ultrasound probe. The computer vision camera provides data required by a tracking program stored in a computer memory operatively coupled to the computer vision camera to provide the position and orientation of the ultrasound probe relative to the camera. A reference marker (307) may be attached to the ultrasound probe to help determine the position and orientation of the ultrasound probe relative to the camera (305). Similarly, the computer vision camera may provide data required by a tracking program stored in the memory of the computer to provide the position and orientation of each of the gamma camera panels (300a) and (300b) relative to the computer vision camera (305). A reference marker (not shown) may be attached to each of the two gamma camera panels to help determine the position and orientation of each panel relative to the camera. The relative positions of the panels and the ultrasound probe relative to the computer vision camera can then be combined to determine the relative positions of the gamma camera panels and the ultrasound probe relative to each other. This enables the registration of images produced by the gamma camera and the ultrasound probe. Furthermore, the memory may contain instructions executed by the processor to use computer vision camera data to determine the contours of the patient's body. This can be used to further determine the relative positions of the gamma camera panel and the ultrasound probe relative to the patient's body.

[0118] Alternatively or additionally, computers (e.g.) Figure 1 The controller 100 shown can be operatively coupled to a sensor connected to the rod 301 to receive position information about the joints of the mechanical components of the rod 301, thereby inferring the position and orientation of the gamma camera panels 300a and 300b relative to the computer vision camera 305.

[0119] Figure 4 It shows Figure 3 The system is illustrated, but the ultrasound probe is positioned outside the space between the two panels. This modality may be helpful for scanning organs within the thoracic cavity, such as the heart and liver. One of the gamma camera panels (400) is shown as being mechanically connected to the distal end (402) of a mechanical articulated arm (403) via a rod (401). The system is shown scanning a patient (404). In this embodiment, an ultrasound imaging probe (405) is used to scan the patient from a location adjacent to but outside the gamma camera panel. The imaging field of view of the ultrasound imaging probe (405) may or may not partially or completely overlap with the imaging fields of view of any one or both panels. A computer vision camera (406) is connected to the distal end (402) via a connector (407). The computer vision camera and data processing for tracking and co-registration can... Figure 3 The procedure is performed in a similar manner to that described in the text. A reference object (408) is shown attached to the ultrasonic probe to assist in ultrasonic probe tracking.

[0120] In another embodiment, the articulated robotic arm can be used to track the position and orientation of the ultrasonic sensor relative to the gamma camera panel. Figure 5 An ultrasound probe co-registered with a gamma camera panel using an articulated arm with coordinate measurement capabilities is depicted. Two gamma camera panels (500a) and (500b) are shown scanning a patient (501). These panels are mechanically connected to the distal end of a mechanical articulated arm 503 via a rod (502). In this embodiment, an ultrasound imaging probe (504) is introduced into the space between the two panels (500a) and (500b). The imaging field of view of the ultrasound imaging probe (504) may partially or completely overlap with the imaging field of view of one or both of the panels (500a) and (500b). An articulated arm (505) with coordinate measurement capabilities may be mounted on the articulated arm assembly (502)-(503). The arm (505) can have its distal end rigidly fixed to the ultrasound probe.

[0121] A tracking program stored in the memory of a computer operatively coupled to sensors connected to the auxiliary arm (505) can be configured to receive positional information about joint movements of the mechanical components of the arm (505) to infer the position and orientation of the ultrasonic probe relative to the rod (502). Additionally, the computer can be coupled to sensors connected to the rod (502) to receive positional information about joint movements of the mechanical components of the rod (502), thereby inferring the position and orientation of the gamma camera panel relative to the rod (502). The tracking program can combine the tracking information from the ultrasonic probe and the gamma camera panel to determine their relative positions.

[0122] Apart from Figure 1 In addition to the camera (106), an articulated auxiliary arm (505) can also be used. In this way, the tracking algorithm on the computer can combine various tracking modalities to determine the position of the gamma camera panel, the ultrasound probe, and the patient's body relative to each other.

[0123] In another embodiment, other tracking sensors can be used to track the position and orientation of the ultrasonic sensor relative to the gamma camera panel. An electromagnetic field tracker is one example. Figure 6An ultrasound probe co-registered with a gamma camera panel using a magnetic tracking system is depicted. Two gamma camera panels (600a) and (600b) are shown scanning a patient (601). These panels are mechanically connected to the distal end of a mechanical articulated arm (603) via a rod (602). In this embodiment, an ultrasound imaging probe (604) is introduced into the space between the two panels (600a) and (600b). The imaging field of view of the ultrasound imaging probe (604) may partially or completely overlap with the imaging field of view of either or both of the panels (600a) and (600b). An electromagnetic transmitter (605) may be mounted near the ultrasound probe, in this case, it is mounted on the articulated arm assembly (602)-(603). An electromagnetic receiver (606) may be rigidly fixed to the ultrasound probe. A tracking program stored in the memory of a computer operatively coupled to the transmitter (605) and receiver (606) can be used to infer the position and orientation of the ultrasound probe relative to the transmitter (605). Additionally, other electromagnetic receivers (not shown) can be rigidly mounted on the gamma camera panels (600a) and (600b) and can be operatively coupled to a computer. The tracking program can use data from the receivers and transmitter to determine the position of the gamma camera panel and the ultrasound probe relative to each other.

[0124] In some embodiments, the transmitter and receiver can be interchanged. In other embodiments, the transmitter can be attached to another object, and the component (605) can be the receiver. In other embodiments, an electromagnetic unit attached to the instrument can serve as both a transmitter and a receiver.

[0125] As an alternative or supplement to fixing the receiver to the gamma camera panel, a computer can be operatively coupled to a sensor connected to the rod (602) to receive positional information about the joint movement of the mechanical components of the rod (602), thereby inferring the position and orientation of the gamma camera panels (600a) and (600b) relative to the transmitter (605). The tracking program can combine the tracking information from the ultrasonic waves and the gamma camera panels to determine their relative positions.

[0126] Apart from Figure 1 In addition to the camera (106) in the image, tracking systems based on electromagnetic transmitters and receivers can also be used. In this way, tracking algorithms on the computer can combine various tracking modalities to determine the positions of the gamma camera panel, ultrasound waves, and the patient's body relative to each other. One example of a tracking system is the use of an external infrared stereo tracker combined with infrared reflective spheres, which are attached in unique patterns to the various parts that need to be tracked. Any combination of the tracking and co-registration techniques described herein, as well as other tracking systems, can be used. For example, the tracking system can be an optical tracking system, an electromechanical tracking system, an electromagnetic tracking system, an ultrasound tracking system, a depth imaging tracking system, or a combination thereof.

[0127] Figure 7 Description of embodiments of portable SPECT gamma camera panels (700a) and (700b) is shown, each panel having fields of view (701a) and (701b) for observing a patient (702). An ultrasound probe (703) with a field of view (704) is co-registered with panels (700a) and (700b) using any combination of tracking systems. In this description, a tracking sensor (705), such as a magnetic field receiver, is shown attached to the ultrasound probe (703). As a result of using an image reconstruction algorithm on the data received from the gamma camera, two SPECT image features (706) and (707) with radioactive uptake can be reconstructed in 3D within the patient. The SPECT image features (706) and (707) can be constructed "in real time," i.e., with a limited frame rate, ideally better than one frame within 2-3 seconds, or they can be constructed by acquiring data over a longer period of time, either from a static position or dynamically by moving them around the patient's body. In the most recent cases, if there is tissue deformation, patient movement, or organ movement, features (706) and (707) may not correspond to their actual location within the patient's body.

[0128] Guided by the SPECT image, a co-registered ultrasound probe (703) can be brought near the gamma camera panel so that its field of view intersects a portion of the SPECT feature (706). In the first stage, a computer operatively connected to the ultrasound probe and visualization device can create an ultrasound image (708) delivered by the ultrasound probe (703) and enhance it with SPECT image features (709) representing a section through the SPECT 3D image in the ultrasound field of view (704). In the second stage, the same computer can identify and correct for possible rigidity transformations between the SPECT and ultrasound images caused by the patient, furniture, or other motion sources. The rigidity transformation is calculated by capturing SPECT image features on the ultrasound image features. The capture process includes: (1) automatically identifying visual features in the two images, (2) matching the SPECT image features with the ultrasound features, and (3) calculating the rigidity transformation (projection) based on the matched features. In other words, the system can create a motion model of features in the ultrasound image from a first ultrasound frame to a second ultrasound frame and create a modified SPECT image based on the motion model of features in the ultrasound image. The resulting enhanced images allow users to identify the patient’s anatomical structures around the SPECT feature (706), similar to what a CT scan would provide in a SPECT / CT imaging system.

[0129] Furthermore, ultrasound images (708) can be used to guide interventional medical devices, such as percutaneous biopsy needles (710) or ablation therapy devices, toward a target of interest highlighted by the SPECT image. In some embodiments, the medical device can also be tracked to allow co-registration between the medical device, ultrasound, and SPECT images. Figure 7 In this context, a tracking sensor (711), such as a magnetic field receiver, is shown for co-registration purposes. Alternatively or additionally, a mechanical instrument guide may be used to define the movement of the medical device. Using either method, the projected trajectory of the instrument (in this description, the projected trajectory of the needle (712)) can be enhanced onto the ultrasound image (708).

[0130] In some embodiments, the head-mounted display can be used to visualize ultrasound images (708) and / or SPECT image features (706) and (707) enhanced to the user's natural field of vision by using a head-mounted display system worn by the user, and has the ability to co-register with the user's eye in a co-registration coordinate system associated with the ultrasound and SPECT images.

[0131] In some embodiments, the entire medical intervention process can be automated, in which case the ultrasound (703) and interventional device (710) are controlled by a computer using a mechanical articulated arm.

[0132] In some clinical applications, creating molecular SPECT images of organs that are easily movable and deformable may be relevant. In such cases, ultrasound images can be used to guide corrections in SPECT images based on tissue deformation and movement observed in the ultrasound images. Figure 8 A description of a method for implementing this correction is shown. In the illustrated embodiment of the portable SPECT, each gamma camera panel (800a) and (800b) with a field of view (801a) and (801b) observes the patient (802). An ultrasound probe (803) with a field of view (804) is co-registered with the panels (800a) and (800b) using any combination of tracking systems. An ultrasound image feature (805) that may appear in a single ultrasound scan may be deformed and displaced in subsequent ultrasound scans and may appear as an ultrasound image feature (806). If an area of ​​increased radiotracer uptake or a SPECT image feature (807) is present during the first ultrasound scan, the SPECT image feature may be located in a different position during the second ultrasound scan. Without tissue deformation correction, if the acquisition time is extended over the time including the two ultrasound scans, the SPECT image reconstruction algorithm will not be able to create SPECT features with the correct range. In this case, ultrasound can be used during the SPECT scan to monitor the movement of visible features in the ultrasound image.

[0133] Ultrasound image sequences can be automatically analyzed using imaging analysis algorithms to determine the domain of organ movement and the deformations that occur from one ultrasound image scan to another. This is achieved by automatically identifying ultrasound image structures (e.g., organs), creating parameterizations of these structures, and tracking their movement and deformation in real time. Ultrasound image structures can be defined by geometric primitives (from simplest to most complex): points, lines, rectangles, and circles. These geometric primitives can be parameterized using patches of point structures (points with radii) and the contours of all other primitives. Contour parameterization depends on the ultrasound image structure: lines are represented as curves, circles as ellipses, and rectangles as polygons. Image structures identified in consecutive ultrasound frames are matched. Each matched pair is used to quantify the motion and deformation experienced by the organ (or structure). The resulting motion field and deformation are used to remap the imaging space elements from one frame to the next, and the reconstruction of the SPECT image will use the remapped imaging space elements to construct a SPECT image corrected for tissue deformation. When organ movement is periodic, such as in the case of heart movement, multiple SPECT images can be created for each sequence over a period of periodic movement. In some embodiments, the ultrasound probe is capable of providing 3D ultrasound images. This creates a “wedge-shaped” 3D imaging volume that captures better and more complex organ movements, resulting in potentially better organ deformation correction.

[0134] If a more complete SPECT image is available, such as an image from a previous SPECT scan, this SPECT image can be used as a priori for the reconstruction of the current real-time SPECT image. For example, an algorithm can be run on an operationally connected computer to update a SPECT image to create a real-time SPECT image by comparing the latest gamma-ray detected event with an estimate of the event. The event estimate can be calculated by the computer by computationally projecting a previous SPECT 3D map forward onto the sensor. The computational projection can indicate the latest configuration of the gamma-ray camera, including the sensor and mask. The computer can calculate the deviation between the detected event and the estimated event to determine the deformation in the previous SPECT 3D map consistent with the most recently detected event. An example of an algorithm that can be used for real-time molecular image updating is described in Lu, Y. et al. (2019) Data-driven voluntary body motion detection and non-rigid event-by-event correction for static and dynamic PET. Physics in Medicine & Biology, 64(6), 065002.

[0135] In some embodiments, the calculation of real-time SPECT images can use tracked deformable ultrasound features from images captured with an ultrasound probe co-registered with the SPECT sensor panel. The parameterization of the ultrasound features is as described above. Examples of feature extraction methods are described below: Revell, J. et al. (2002) "Applied review of ultrasound image feature extraction methods", 6th Medical Image Understanding and Analysis Conference (pp. 173-176). BMVA Press; Alemán-Flores, M. et al. (February 2005) "Semiautomatic snake-based segmentation of solid breast nodules on ultrasonography", International Conference on Computer Aided Systems Theory (pp. 467-472); Springer, Berlin, Heidelberg; Zhou, S., Shi, J., Zhu, J., Cai, Y., & Wang, R. (2013) "Shearlet-based texture feature extraction for classification of breast tumor in ultrasound image", Biomedical Signal Processing and Control, 8(6), 688-696. These parameterized ultrasound features can be tracked by a computer to obtain tracked deformable ultrasound features using methods such as those described in Yeung, F. et al. (1998), Feature-adaptive motion tracking of ultrasound image sequences using a deformable mesh. IEEE Transactions on Medical Imaging, 17(6), 945-956. The tracked deformable ultrasound features can be used by a computer to compute updated real-time SPECT images. For example, this can be achieved by using the tracked parameterized ultrasound features to constrain the solution of a SPECT image deformation model. In some embodiments, the computer can create this SPECT image deformation model to obtain updated SPECT images using previously collected SPECT 3D maps.In this scenario, SPECT image elements are essentially pinned to the tracked deformable ultrasound features and move along with them. In some other embodiments, the computer can create a SPECT image deformability model by using real-time gamma data combined with previously collected SPECT 3D maps to obtain an updated SPECT image. In this case, the tracked deformable ultrasound features are used to constrain the deformability model calculated by comparing the latest gamma-ray detection event with an event estimate calculated by the computer based on previously collected SPECT 3D maps, as described in the previous paragraph. The computer can use a deformable model data fusion filter to combine the ultrasound deformable model and the deformable model that compares real-time gamma-ray data with previous SPECT maps. Such a deformable model data fusion filter can use a Kalman filter (Welch, G., & Bishop, G. (1995) "An introduction to the Kalman filter"). The filter parameters, which determine how a filter is applied to a specific application, can be modified by a computer or the user, and can take into account factors such as the quality of the ultrasound image, the quality of the extracted ultrasound features, the quality of the ultrasound deformable tracking model, the gamma-ray count rate, an estimate of the gamma-ray count rate extracted from the forward projection of a previously reconstructed SPECT 3D image, and the signal-to-noise ratio of the SPECT image. For example, if the ultrasound-tracked structure is highly reliable and the detected gamma-ray count rate is too low, a filter running on a computer can emphasize the tracked ultrasound deformable structure more in the construction of an updated SPECT image, and vice versa.

[0136] These systems and methods can also be used to propagate deformation tracked in an ultrasound scan away from the ultrasound imaging volume by utilizing the specific mechanical and elastic properties of tissue deformation, thereby expanding volumetric regions within the patient that cannot be directly observed in an ultrasound scan. These deformation propagation methods can also utilize a 3D model of the patient tracked by a computer vision system. For example, this 3D model of the patient can provide boundary conditions for deformations away from the scanning ultrasound volume.

[0137] Figure 9 An illustration shows one embodiment of portable SPECT gamma camera panels (900a) and (900b), mounted to an articulated arm (902) via a joint rod (901). In this depiction, the panels are positioned to scan a specific body part of interest, such as a human breast (903). In this case, the panels are substantially parallel to each other, and the fields of view of the gamma cameras largely overlap. In other embodiments, the gamma cameras may be positioned at other angles relative to each other, such as 90 degrees to each other.

[0138] In some embodiments, gamma camera panels (900a) and (900b) include gamma sensors that, when the gamma camera panels are in a substantially static position relative to the patient's body part (903), provide sensor data containing information used by an image reconstruction program running on a computer to provide imaging of the radiotracer distribution at a limited resolution along the direction most sensitive to either gamma camera. In some embodiments, the limited resolution is less than 20 mm for a distance range covering at least 50 mm. In such embodiments, 3D imaging of lesions (904) with increased radiotracer uptake within the patient's body can be performed. Compared to a planar imaging setup, a 3D image of the radiotracer distribution will provide better lesion detection and localization.

[0139] In some embodiments, panels (900a) and (900b) may include tactile pressure sensors (905a) and (905b), preferably placed on the panel surface facing the patient's body part (903) to obtain information about the pressure applied by the panel to the patient's body or to other objects. These tactile pressure sensors may be operatively coupled to a computer, which is also connected to an actuator that moves the panel relative to the patient. The computer may use a program to change the position of the panel to maintain the pressure applied by the panel to the patient or other objects within a desired range. The tactile pressure sensors may be capacitive, resistive, piezoelectric, or any other sensor capable of providing tactile pressure between objects.

[0140] In this configuration, the co-registered ultrasound probe and percutaneous medical device can be used on the patient's body parts being examined, in conjunction with... Figure 7 The embodiments described herein are similar.

[0141] Figure 10A description of one embodiment is shown, in which a portable cart (1000) integrates a SPECT and medical ultrasound system. A 6-axis robotic arm (1001) is fixed to the portable platform (1000). An articulated gamma camera sensor mounting assembly (1002) is fixed to the distal end of the robotic arm (1001). As described above, gamma camera panels (1003a) and (1003b) are mounted on the mounting assembly (1002). As described above, a computer vision camera (1004) is mounted on the mounting assembly (1002). Ultrasound probes (1005) and (1006) are operatively connected to the ultrasound electronics and controller via a connector panel (1007). In a preferred embodiment, a user uses a console (1008) to control the ultrasound system, the SPECT camera system, and the mechanical system, including the robotic arm. A monitor (1009) is depicted herein as a visualization device. Other visualization devices may be used, such as a head-mounted display co-registered with the patient. The cart can contain any of the following components: a main controller (computer), including a graphics processing unit (GPU); a mechanical subsystem controller, including a 6-axis robotic arm; electronics for ultrasound pulse formation and readout; and electronics for gamma camera readout. SPECT and ultrasound co-registered images can be delivered from the main computer to a visualization device. Other medical sensors or other medical imaging devices can be used to replace or supplement the ultrasound probe. Examples include: fluorescence imaging probes, optical coherence tomography probes, computer vision cameras, infrared cameras, impedance sensors, etc.

[0142] Figure 11 It shows Figure 10Operational connections between the components mentioned in the description. The housing of the portable platform (1000) is represented by (1100). The housing includes at least one central computer controller (1101) for reading data from sensors and other subsystems. The computer integrates the data to determine tracking information for objects of interest, reconstructs SPECT images and potentially other medical images from other sensors, creates co-registered images, enhances images to each other, sends visualization data to visualization devices, and controls other controllers, subsystems, and electronic devices. The housing may also include electronics (1102) for reading and controlling gamma cameras (1103a) and (1103b). The housing may also include a mechanical controller (1104) that receives sensed mechanical information and controls the articulated arm (1001) and other actuators of wheels, patient beds or seats, gamma camera mounting components (1002), or other objects, medical devices, or sensor components potentially attached to the platform (1100). The housing may also include electronics (1105) for providing pulse shaping for and reading signals from the ultrasonic transducer (1106). When other imaging and sensing modalities can be used, their control and readout electronics may also be housed within the platform housing. Various ports will be provided for all such imaging probes and sensors, for example, on a connector panel (1007). The central computer controller may also control and read out tracking devices, computer vision systems, etc. As an example, in the illustration, a computer vision camera (1107) is shown as being read out and controlled by the central computer controller (1101). Visualization output created by the central computer can be sent via a network to visualization devices, such as a monitor (1108) or other computers, or to a head-mounted display.

[0143] In some embodiments, an array of ultrasonic transducers, registered to each other and to other sensors (e.g., a gamma camera panel), can be used instead of a single ultrasonic transducer probe. Such an array can expand the volumetric region that can be imaged by a co-registered ultrasonic imaging system. Figure 12An illustration of this system is shown. Gamma camera panels (1200a) and (1200b) are shown scanning a patient (1201). A flexible band (1202) conforming to the patient's body contours includes at least one ultrasound transducer, three of which are shown in the illustration (1203a-c). These ultrasound transducers image the patient (1201) by ultrasonically contacting the patient's body. Their position and orientation relative to each other and relative to the gamma camera panels (1200a) and (1200b) are tracked using tracking sensors (1204a-c). Such tracking sensors can be: electromagnetic sensors, optical reference markers recognizable by an optical system, ultrasound reference markers or sensors, infrared reflective markers, active light-emitting markers, or any other components that can be used to track the position and orientation of the ultrasound transducers relative to each other. These ultrasound transducers can be used individually or in combination to penetrate the patient's body and sense the reflected ultrasound waves, thereby obtaining a significant ultrasound 3D field of view. Such a large 3D field of view can significantly overlap with the gamma camera's field of view, which will allow for more precise tissue deformation correction, such as... Figure 8 As shown.

[0144] Furthermore, during the reconstruction of SPECT images, the computer can analyze ultrasound images associated with each gamma camera detection to calculate the attenuation probability of tissue passing through the detected gamma rays. This calculation requires a detailed map of the tissue attenuation coefficient. Because ultrasound images do not directly provide information about tissue type, automated ultrasound tissue characterization modeling, such as using machine learning modeling methods, can be applied to the ultrasound image dataset to extract a map of gamma attenuation factors. Tissue types, such as water, fat, muscle, bone, or air-filled regions, can be extracted. Standard gamma attenuation coefficients associated with each of these components can be used to build a gamma attenuation map within the patient. This attenuation map can be periodically remapped using regularly updated ultrasound images, as tissue and organ deformation may occur. Since ultrasound signals propagate poorly in bone and air, maps of these components within the patient can be extracted by tracing adjacent tissues that are clearly visible in the ultrasound images. Other anatomical priors can be used to aid in the tissue characterization process and the mapping of gamma attenuation coefficients. An example of a priori maps is CT images, which robustly characterize gamma attenuation features that are both well-visible and less visible to ultrasound.

[0145] Figure 12 The embodiments described can also be used for precise interventional guidance. A large 3D ultrasound field of view combined with SPECT images corrected for tissue deformation and changes in gamma attenuation coefficients within the patient can create a highly precise guidance system for percutaneous biopsies or other interventions, such as ablation therapy.

[0146] In some embodiments, the ultrasound transducer (1203a-c) can be used to deliver a high-intensity focused ultrasound beam to a region of interest, as highlighted by SPECT and / or ultrasound images. This beam can be used for a variety of purposes, such as ablating tumors, allowing drugs to better penetrate the tissue of interest, and inducing other physicochemical changes within the tissue of interest.

[0147] In some embodiments, the ultrasound transducers may not be located within the same flexible band (1202), and they may be attached to the patient independently of each other. However, they can still be tracked relative to each other and relative to other reference points, and can be used in combination to create large 3D ultrasound images in the region of interest.

[0148] In some embodiments, the structure including the ultrasound transducer, such as the flexible band (1202), may have a fixation mechanism that keeps the transducer (1203a-c) securely attached to the patient's body without user intervention. For example, such a fixation mechanism may be a robotic arm or tape.

[0149] For certain applications, registering SPECT images with other imaging modalities such as MRI or CT may be beneficial. Figure 13A description of a portable SPECT system for use in conjunction with a separate medical imaging system is shown. In some embodiments, the portable SPECT system (1301) may be moved to the vicinity of another medical imaging instrument (1302), such as a CT, MRI, or magnetic imaging system. This allows the portable SPECT system (1301) to scan the patient (1303) simultaneously with, before, or shortly after the system (1302) completes imaging. The portable imaging system (1301) may be SPECT or another imaging modality and may include an imaging sensor panel (1304) attached to the distal end of an articulated arm (1305). In a preferred embodiment, the articulated arm is a 6- or 7-DOF robotic arm. The scanning components include a computer vision camera system (1306). The computer vision camera system is co-registered with the sensor panel using the computer vision system itself, by mechanical co-registration, or by using other trackers. The computer vision camera may be located within a single housing, or its components and sensors may be distributed across multiple housings and sections of the scanning system (1301). In some embodiments, components of the computer vision system may be housed within the sensor panel (1304). In a preferred embodiment, a portion of the computer vision camera system (1306) is attached to the distal end of the articulated arm, having a field of view covering a portion of the patient. In other embodiments, a portion of the computer vision camera system may be placed elsewhere and may only have a field of view covering a portion of the patient. The computer vision system (1306) is understood to be any system capable of creating images, image streams, ranging data, and ranging data streams analyzed by an operatively connected computer, and may include an RGB camera, an infrared camera, a depth imaging optical camera, a stereo optical camera, a time-of-flight optical camera, a terahertz emitter-sensor assembly, an ultrasonic emitter-sensor assembly, another tracking and / or mapping sensor, or a combination thereof.

[0150] As part of the process, the patient (1303) is scanned by a scanner (1302) while lying on a scanning table (1306). The table allows the patient to be inserted into the scanner, and a scan will be performed. The image reconstructed by the scanner (1302) is read by a computer operatively coupled to the scanner (1301). As another part of the process, the scanner assembly (1301) scans the patient (1303) to create another imaging dataset. This scan can be performed simultaneously with, before, or after the scanner scan (1302). The computer operatively coupled to the scanner (1301) can analyze the reconstructed image from the scanner (1302) to identify the structure for co-registration between the two image datasets created by the scanners (1301) and (1302).

[0151] In some implementations, labels (1308) and (1309) may be placed near the patient before scanning, either by placing label (1308) on a table or by placing label (1309) on the patient's body. These labels may contain features that can be recognized by the scanner (1302) and the system (1301). Figure 14 Examples of these labels are described. Structures that are part of the labels (1308) and (1309) and imaged by the scanner (1302) are matched against structures in computer memory to determine the coordinate system associated with the image dataset transmitted by the system (1302). Structures that are part of the labels (1308) and (1309) and can be imaged by the computer vision camera (1306) are analyzed by a computer operatively connected to the system (1301) and matched against structures in computer memory to determine the transformation between the label coordinate system and the camera. Furthermore, using a known transformation (1304) between the camera and sensor panel, the operatively coupled computer can determine the transformation between the coordinate system associated with the image dataset delivered by the system (1302) and the coordinate system associated with the image dataset delivered by the system (1301). Once the transformation is known, the system (1301) can use the data transmitted by the system (1302) to calculate attenuation maps and better reconstruct the location of the in vivo target using techniques known in the art (e.g., SPECT-CT).

[0152] When operably coupled to the ultrasonic transducer (802), a novel use of the transformation transmitted via the common characteristics of the data transmitted by the computing system (1302) and the system (1301) emerges. In this embodiment, the system (1301) calculates a deformation attenuation model to address various tissue deformations as described above.

[0153] In some embodiments, the patient may expose skin to body parts primarily scanned by the scanner (1302). Additionally, in some embodiments, the patient may wear close-fitting clothing, such as… Figure 15 As shown. This will allow computer vision systems based on optical sensors or sensors that detect signals that do not penetrate fabric to create accurate 3D models of the patient's body contours. In some other implementations, the patient may wear a regular, loose-fitting hospital gown. In such implementations, the computer vision system may include fabric-penetrating scanning systems, such as ultrasound scanning systems, terahertz scanning systems, or low-dose soft X-ray backscattering systems, to create a 3D model of the patient's body contours.

[0154] When data from the computer vision system (1306) can be used by an operatively coupled computer to create a 3D model of a patient's body, the scanner (1302) is preferably capable of delivering maps of anatomical structures, including distinguishing between the patient's body and air. In this case, a computer operatively connected to the scanner (1301) can then analyze structures in the image dataset provided by the scanner (1302) that are related to the patient's body contours and match them with the 3D model of the patient created by the computer vision camera (1306). This allows the computer to create a co-registration transformation between the two imaging datasets and map the imaging data (including anatomical imaging data) created by the scanner (1302) to a reference system associated with the imaging data created by the scanner (1301) to create a co-registered anatomical map. The co-registration transformation can be the result of an iterative algorithm that searches for the optimal transformation to be applied to one of the models to minimize the distance error between the two models. When the distance between the two 3D models is minimized, it is assumed that the overlap is maximized and the models are aligned. Examples of such algorithms are Iterative Closest Point (ICP) or Generalized ICP. ICP is described in Chen, Y. and Medioni, GG (1992) Object modeling by registration of multiple range images. Image Vision Comput., 10(3), 145-155. Generalized-ICP is described in Segal, A., Haehnel, D. and Thrun, S. (2009 June). Generalized-icp. In Robotics: science and systems (Vol.2, No.4, p.435).

[0155] In some cases, systematic biases may exist between two 3D models of a patient's body. Some of these biases may occur because some patient body movement or deformation can happen between the moment a 3D scan is performed using a computer vision system (1306) and the moment an imaging scan is performed using a scanner (1302). In such cases, non-rigid 3D matching algorithms can be used to remap an image dataset (including an anatomical image dataset) created by the scanner (1302) to a 3D model delivered by the computer vision system (1306) to create a co-registered deformable anatomical map. These algorithms can compute the co-registration transformation between two 3D models and the deformations they can map in the 3D models. This is achieved by allowing non-rigid movement of different parts of the 3D model while attempting to minimize distance errors. The algorithm ensures smooth movement between model parts and deformations that are as rigid as possible. An example of this algorithm is dynamic fusion, as described in RA, Fox, D. and Seitz, SM (2015). Dynamic Fusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 343-352).

[0156] In the process of reconstructing images created by the scanner (1301), co-registered rigid or deformable anatomical maps can be used by an operationally coupled computer. For example, co-registered anatomical maps, such as those delivered by a CT scanner (1302), can be used to create gamma-ray photon attenuation maps, which can be further used by an operationally coupled computer to improve the quality of reconstructed images created by a SPECT or PET scanner (1301). For example, attenuation maps are used to quantify the probability of photons propagating, being absorbed, or being transmitted from a specific imaging element (e.g., a voxel or grid point) to a sensor in the panel (1304). This attenuation correction process is currently performed in integrated SPECT / CT systems. For co-registered deformable anatomical maps, deformation modeling of bones and low-density volumes (e.g., the lungs) is particularly important for accurate photon attenuation correction in SPECT or PET image reconstruction.

[0157] In some implementations, anatomical mapping can be completed in separate scanning sessions. In this case, the patient may have moved significantly between anatomical scanning sessions (e.g., performed by a CT or MRI scanner (1302)) and molecular imaging scanning sessions (e.g., performed by a portable system (1301)). In this case, the discrepancy between the 3D body model extracted from the anatomical data and the 3D body model delivered by the computer vision system (1306) may be too large to create an accurate co-registered deformable anatomical map. In this case, as described above, an ultrasound probe co-registered with the computer vision system can be used to scan the patient at one or more locations to transmit ultrasound images of the patient to an operatively coupled computer. Structures extracted by the computer from these ultrasound scans can then be matched with structures extracted by the computer from the anatomical map delivered by the scanner (1302) to create co-registration pins. These co-registration pins will constrain the solution for deformable co-registration from the anatomical map to the molecular map. As described above, the co-registered deformable anatomical map can be used for attenuation correction during the molecular map reconstruction process.

[0158] The co-registered deformable anatomical map can also be sent to a visualization device by an operatively coupled computer, combining the presentation of the molecular map provided by the scanner (1301) with the presentation of the co-registered anatomical map provided by the scanner (1302). The computer can also send to the visualization device a presentation of ultrasound images delivered by a co-registered ultrasound probe, combining the presentation of the anatomical map delivered by the scanner (1302). This process can be accomplished when scanning sessions performed by the scanners (1301) and (1302) are completed jointly or separately.

[0159] In some implementations, navigation co-registration of anatomical, ultrasound, molecular imaging models, or combinations thereof, may be useful, as they are augmented onto real-time red-green-blue (RGB) images of the patient captured by a camera or onto the user's field of view using a head-mounted display. In this case, the user can use a handheld stylus or probe tracked by a computer vision system to select the imaging plane of interest, displaying a representation of these imaging datasets in the plane selected with the stylus or probe.

[0160] In some other implementations, the computer vision system (1306) can determine its position relative to the scanner (1302) by analyzing geometric features on the scanner (1302) or by analyzing a label attached to the scanner (1302). This allows a computer operatively connected to the computer vision camera (1306) to determine the position and orientation of the panel (1304) relative to the scanner (1302). In return, this allows the computer to perform co-registration between image datasets provided by the scanner (1301) and the scanner (1302).

[0161] In some implementations, the scanner (1301) can scan the patient as the scanning table moves past the scanner (1302). In this case, the computer adjusts the scanning protocol in real time to take into account the movement of the patient and the scanning table, as provided by the computer vision system.

[0162] Figure 14 It shows the use of portable imaging systems (such as...) Figure 13 (as shown in (1301)) and another medical imaging scanner (such as Figure 13 The reference (1302) depicts a co-registration reference. In an embodiment, the reference, whose outline is represented by (1401), may include a structure (1402) recognizable in the image delivered by the scanner (1302). For example, if the scanner is a CT scanner, the structure (1402) may be made of a material that attenuates X-rays, such as steel, tungsten, or another high-density material. The reference (1401) may also include features (not shown) that can be recognized by a computer vision system (1306), such as a binary black-and-white label. In the illustration, a ring of known diameter may support multiple protruding geometries (substructures) of different sizes (1403). These substructures will break the symmetry of the reference, allowing for clear identification of the reference's location and orientation, and will enable it to be used with instruments of various imaging performances, such as resolution and contrast. Thus, larger structures will be visible in the images delivered by most scanners, while smaller structures will add the additional benefit of better co-registration of the images delivered by the scanner, which provides better imaging performance, such as resolution and contrast. The shapes have known dimensions and positions relative to the center of the ring. In the illustration, the shape is spherical, but it could be changed to a pyramid shape, for example, in other embodiments. Analyzing data from the target modality, the computer can calculate the center position of the ring by observing the position of the material and the orientation of the structure (1402). The reference (1401) is constructed using known transformations between aspects specifying the position inferred by computer vision techniques and the modality being targeted using the asymmetric structure (1402).

[0163] The position and orientation of the endoscope and the camera guided within it can be inferred using fluoroscopy and techniques established in the field. Using a reference (1401) to co-register the SPECT system with the coordinate frame of the fluoroscopy system, the same computational unit can provide the operator with an enhanced view of the internal body region observed by the camera inserted into the endoscope. This enhancement can guide the operator to objects within the body identified by the SPECT imaging system (1301). To achieve this, we can ensure that the computational unit is informed of the position and orientation of the camera within (or attached to) the endoscope. The tracking position of the endoscope can be interpreted by the same computational unit using techniques known in the art. If the endoscope tracking system is also articulated to view the co-registration reference (1401), the computational unit can also calculate the position and orientation of the camera attached to or within the endoscope relative to the coordinate frame selected by the SPECT system (1301). This is achieved by calculating the relative transformation between the endoscope reference structure and the co-registration reference (1401). This transformation sequence allows the computational unit to overlay the reconstructed SPECT image onto the image acquired by the endoscopic camera. This information is particularly useful for guiding interventional procedures, such as lung biopsies when ultrasound cannot provide real-time guidance.

[0164] The aforementioned endoscope may contain instruments and tools capable of ablating, drilling, cutting, puncturing, debridement, or approaching targets identified by a SPECT imaging system. These tools may be articulated by the operator and monitored by visualization provided by an endoscopic camera enhanced by a computing unit informed of the SPECT target as described above.

[0165] Data from cameras or other computer vision systems inserted within the endoscope can also be used to observe and measure tissue deformations through computational processes. Due to the co-registration described, which combines the positions of the computer vision system within the endoscope with the SPECT system, these tissue deformations can also inform SPECT reconstruction by applying the inferred deformation transformation to the attenuation map. Updating the attenuation map is crucial when computing real-time or near-real-time SPECT images. Similarly, tissue deformations inferred from sensors within the tracked endoscope can be used to calculate accurate updates to the visualization of previously reconstructed SPECT targets within the body, as described above. These updates can be presented as overlays or enhancements to images captured by sensors on or within the endoscope. These updates can also be presented by a SPECT visualization monitor (1009) to guide interventions visualized through other co-registered sensors.

[0166] Figure 15Illustrations show body-fitting garments that patients can use to assist in optical-based computer vision tracking and mapping processes. Patients, whether female (1501a) or male (1501b), can wear garments (1502a) and (1502b) that closely follow the body's contours. For example, the garments can be made of stretchable fabric. The garments can be made of materials that allow ultrasound waves to pass through the garment substantially unimpeded, particularly in combination with an ultrasound-conducting gel. The garment material can be selected such that, when combined with the ultrasound-conducting gel, the garment will provide reduced acoustic impedance and reflection for typical medical ultrasound frequencies. The garment material may include perforations that allow the ultrasound gel to pass through and reach the patient's skin after being applied to the garment. This enables superior transmission of ultrasound waves.

[0167] In some embodiments, the garment may cover only the upper body, such as the torso. In other embodiments, it may cover only the legs, hips, and part of the waist, similar to swimming trunks. In some other embodiments, the garment may cover the hips, waist, and torso (as shown). The garment may or may not cover most of the legs and arms. The garment can be put on and taken off like a swimsuit. In some embodiments, the garment may be closed by binders (1503a) and (1503b). These binders may be selected from a group including zippers, buttons, adhesive tape, hook and loop fasteners, or combinations thereof. Adhesive tapes (1504a) and (1504b) may be used to ensure that the garment conforms to the patient's skin where it may be raised from the skin. This tape may be part of the garment or used alone. In some embodiments, the tape has double-sided adhesive. This double-sided tape can be applied to specific locations on the skin before the garment is put on and presses the garment onto the tape to ensure it is close to the skin.

[0168] In some embodiments, these garments may include imprints or patterns. Such geometric features can be identified and tracked by a computer vision analysis program in a computer, operatively coupled to a computer vision system, to create a 3D model of the patient by tracking the imprint or pattern features. For this purpose, structures derived from motion algorithms (as described in Fuhrmann, S., Langguth, F. and Goesele, M., Mve-a multi-view reconstruction environment, in GCH pp. 11-18 (October 2014); Ummenhofer, B., Zhou, H., Uhrig, J., Mayer, N., Ilg, E., Dosovitskiy, A., & Brox, T., Demon: Depth and motion network for learning monocularstereo, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5038-5047 (2017)) can be used with a monocular camera. Structure matching algorithms can be used with a stereo camera. Structure matching algorithms are described in Hirschmuller, H, Stereo processing by semiglobal matching and mutual information, IEEE Transactions on pattern analysis and machine intelligence, 30(2), 328-341 (2007); Sun, J., Zheng, NN and Shum, HY, Stereo matching using belief propagation, IEEE Transactions on pattern analysis and machine intelligence, 25(7), 787-800 (2003); Geiger, A., Roser, M. and Urtasun, R., Efficient large-scale stereo matching, in Asian conference on computer vision (pp. 25-38). Springer, Berlin, Heidelberg (November 2010).

[0169] These garments allow computer vision systems to create precise 3D images of a patient's contours without exposing large areas of skin, thus keeping the patient warm and comfortable. Another benefit is that they preserve compressed areas of the patient's body that might otherwise be loose. Yet another advantage is that they allow for convenient ultrasound examinations using co-registered ultrasound probes.

[0170] Figure 16 A processing workflow for importing other imaging datasets to deliver multimodal image fusion and improve SPECT reconstruction is illustrated. A portable imaging instrument (1600) may include a molecular imaging camera system (1601), a co-registration ultrasound system (1602), and a co-registration computer vision camera (1603). Anatomical image data is delivered by another medical imaging scanner (1604). The scanner (1604) may be a CT, MRI, or another imaging system capable of providing 3D anatomical data. In a preferred embodiment, the scanner (1604) is a CT scanner. A computer operatively connected to the computer vision camera analyzes the data from the computer vision camera to create a computer-visualized 3D model of the patient's body. The computer also analyzes the data from the scanner (1604) to extract the 3D model of the patient's body generated from the anatomical images. In process (1605), the computer-visualized 3D model is compared with the 3D model generated by the anatomical imaging instrument to create a co-registration mapping between the two image datasets. This co-registration mapping can perform rigid co-registration or deformable co-registration. This creates a pre-registered anatomical image dataset. In some cases, a second-stage co-registration (1606) can be performed, where a co-registered ultrasound probe connected to an ultrasound system (1602) performs scans at one or more locations on the patient's body. The computer analyzes the ultrasound scans to identify anatomical features generated by the ultrasound and matches these features with internal anatomical features in a pre-registered anatomical image dataset. This creates anchor points where specific internal anatomical features in the pre-registered anatomical image dataset are assigned 3D locations equivalent to their positions in the ultrasound images. The computer uses these anchor points to constrain a deformable co-registration solution and iterates over the pre-registered anatomical image dataset. This creates a co-registered anatomical image dataset. In procedure (1607), the computer loads molecular imaging data from (1601) and the pre-registered or registered anatomical image dataset to reconstruct molecular images using attenuation maps extracted from the anatomical dataset. The computer can then send subsequent renderings or a combination of renderings of the molecular images, co-registered anatomical images, and ultrasound scans to a visualization device.

[0171] In some embodiments, the computer vision system (or a portion thereof) may not be attached to component (1301). In this case, the computer vision system can monitor not only the tag and / or patient, but also components of the scanner (1301), or a reference may be mounted on a portion attached to the cart (1301), panel (1306), robotic arm (1305), or other components. This would allow the sensor panel to be co-registered relative to the computer vision camera and relative to the patient or reference tags (1308) and (1309). Alternatively, the computer vision camera may be tracked by another tracking system that allows co-registration between the computer vision camera and the molecular imaging system.

[0172] Figure 17 A cross-sectional side view of an embodiment of a large field-of-view coded aperture imaging system is shown. The figure illustrates a technique for achieving large field-of-view imaging using mask elements with multi-planar sides. A position-sensitive sensor (1700) is shielded (1701a) and (1701b) on some sides not covered by the coded aperture. The large field-of-view coded aperture mask comprises elements (1702a)-(1702h). The planes of the mask are positioned at a distance (1703) from the surface of the position-sensitive sensor (1700). Each individual mask element pixel has multi-planar sides, allowing photons from a range of directions to be encoded. For example, in this embodiment, the mask elements on the left side (1702a)-(1702d) allow photon flux from a direction (7104) approximately -45° from the mask normal to be encoded based on their angle and sensed by the sensor (1700). These elements also allow photon flux from a direction (1705) close to the mask normal to be encoded based on their angle and sensed by the sensor (1700). Similarly, the mask elements on the right (1702e)-(1702h) allow photon flux from a direction (1706) approximately 45° from the mask normal to be encoded based on their angle and sensed by the sensor (1700). These elements also allow photon flux from a direction (1705) close to the mask normal to be encoded based on their angle and sensed by the sensor (1700). Using this type of mask configuration, high-sensitivity imaging can be performed at angles from approximately -45° to approximately 45° relative to the mask plane normal. Similar mask element shaping and encoding can occur in another perpendicular direction (orthogonal to the plane of this illustrated portion). For example, the mask elements can be made of rigid, high-density, high-atomic-mass materials, such as tungsten or tungsten alloys.

[0173] Figure 18A top view of the large field-of-view coded mask (1800) is presented. Various shades of gray indicate different depths along directions orthogonal to the mask plane. Holes or openings in the mask are represented by white areas, such as (1801). It is not recommended to use the same holes arranged in a regularly repeating pattern, as this may produce reconstruction artifacts.

[0174] Figure 19 Various views and geometries of pixel elements in a mask having square or rectangular pixels are shown. A biconical mask pixel element is shown in a side view (1900). A square or rectangular base (1901) effectively defines the extent of the mask pixels within the mask plane. In this particular embodiment, the plane of the biconical base is parallel to the mask plane. The sides of the biconical base may be perpendicular to the bottom surface of the biconical base, or may form various angles (1902) and (1903) relative to the normal to the bottom surface. In some embodiments, angles (1902) and (1903) have values ​​greater than 10 degrees. Thus, the biconical mask pixel element has at least one side that forms an angle greater than 10 degrees relative to the normal on the biconical base. In some embodiments, angles (1902) and (1903) have values ​​less than 60 degrees.

[0175] When viewed from a vertical side view, pixel elements can have similar profiles, or they can have sides at other angles to the normal to the base plane. (1904) represents a 3D view of a rectangular biconical frustum that is centrally symmetric with respect to a rectangular base and has straight edges. This is just a representative example, and other biconical geometries can also be used. In some embodiments, when two mask pixels have a common edge, the direct space between the two biconical frustums can be filled with a decaying material. It can be seen that two adjacent biconical frustums (1905) form gaps (1906a) and (1906b) defined by the area between the dashed lines, as shown. Figure 19 As shown. In some embodiments, these gaps are filled with a decaying material. Some mask pixel elements may also have a frustum profile (1907) in which the sides are at the same angles (1908) and (1909) relative to the normal on the base plane (1910).

[0176] In some embodiments, certain mask pixel frustums may have a side view similar to (1900) and another (1900), or a side view of (1900) and another (1907), or a side view of (1907) and another (1907). The last case is represented by the 3D view of (1911), which is essentially a symmetrical frustum of a right-angled pyramid. It can be seen that two adjacent frustums (1911) form gaps (1913a) and (1913b), defined in the figure by the area between the dashed lines. In some embodiments, these gaps are partially or completely filled with attenuating material.

[0177] In some embodiments, instead of using pixel mask elements shaped as biconical frustums with flat sides, pixel elements may include circular surfaces that substantially capture the attenuation profile of a biconical frustum with flat surfaces. Figure 19 As shown, a pixel element with a circular surface having a contour (1914) can provide attenuation characteristics similar to those of a biconical truncated pyramid having a contour (1900). In some embodiments, when two such pixel elements have adjacent common edges as shown in (1915), the gap (1916) between the two pixel elements is filled with an attenuating material. Similarly, a pixel element with a circular surface having a contour as shown in (1917) provides attenuation characteristics substantially similar to those of a biconical truncated pyramid having a contour (1911). In some embodiments, when two such pixel elements have adjacent common edges as shown in (1918), the gap (1919) between the two pixel elements is filled with an attenuating material.

[0178] In other embodiments, the sidewalls of the mask pixel elements and the fill between the mask elements may have a stepped appearance, or may have other microstructures that substantially follow a macroscopic profile similar to contours (1900), (1905), (1907), (1912), (1914), (1916), (1917), and (1918).

[0179] In one embodiment, the fill attenuation material between the mask pixel elements is made of a material with a density greater than 10 g / cc.

[0180] In some embodiments, the fill attenuation material between the mask pixel elements is made of the same high-density, high-atomic-number-Z material, such as tungsten or a tungsten alloy.

[0181] Although mask pixel elements oriented as biconical bases with rectangular bases have been depicted, biconical bases with triangular or hexagonal bases can also be used as mask pixel elements in mask arrays comprising triangular or hexagonal pixels, respectively. Individual coded aperture masks can combine mask pixels with different geometries, such as rectangular, square, triangular, or hexagonal, and can have various sizes. Similar to rectangular pixels, triangular or hexagonal biconical mask pixel elements can be centrally symmetric with respect to the triangular or hexagonal base, symmetric with respect to the plane of the triangular or hexagonal base, or may not have any symmetry. The shape of rectangular, triangular, or hexagonal biconical mask elements can vary across the entire mask. Attenuating material can be used for any of these geometries to partially or completely fill the space between biconical mask pixel elements sharing a common edge.

[0182] Figure 20 It shows something similar to Figure 17The embodiment shown is a large field-of-view coded aperture, but the coded aperture extends onto other planes. In this embodiment, the coded aperture extends onto a plane perpendicular to the first mask plane. The position-sensitive sensor (2000) is shielded (2001a) and (2001b) on some sides not covered by the coded aperture. The coded aperture mask in the first plane includes elements (2002a)-(2002h). The first plane of the mask is positioned at a distance (2003) from the surface of the position-sensitive sensor (2000). In some embodiments, the distance (2003) can be increased to achieve increased angular resolution, which can translate into increased imaging resolution. Mask elements (2004a) and (2004b) can be located in planes other than the first mask plane to maintain a large imaging field of view (2005). In some embodiments, a large imaging field of view can be maintained at a larger mask-sensor distance (2003) by extending the physical contour of the mask in the mask plane beyond the physical contour of the sensor in the same plane.

[0183] The position-sensitive sensor (2000) may be made of multiple sensor units that are substantially parallel to each other and have substantially the same plane and are parallel to the surface of the first mask plane (2006), or they may have sensor units whose front faces are in a plane at an angle between 0 and 90 degrees to the first mask plane (2006).

[0184] Figure 21 It shows something similar to Figure 17The embodiment shown is a large field-of-view coded aperture, but other components are depicted to support a description of a process that can be used for image reconstruction analysis involving a large field-of-view coded aperture. A position-sensitive sensor (2100) is shielded (2101a) and (2101b) on some sides not covered by the coded aperture. The coded aperture mask in a first plane includes elements (2102a)-(2102h). The first plane of the mask is positioned at a distance (2103) from the surface of the position-sensitive sensor (2100). A first radiative interaction (2104) and a second radiative interaction (2105) are detected by the sensor (2100) at position resolution in all three dimensions. The sensor (2100) can be a semiconductor detector, such as CdZnTe, a scintillator detector, such as Na(I), LSO, or any other scintillator material capable of providing 3D interaction positions. Interaction locations can be realized in 2D by segmenting readout electrodes, using position-sensitive optical detection arrays, using signal analysis methods, or by creating a single sensor rod with a single signal readout, or a combination of more than one sensor rod. Interaction locations in the third dimension can be realized by using sensors with depth interaction capabilities. Certain systems can provide depth interaction locations for semiconductor and scintillator sensors.

[0185] Once radiation is detected in 3D, the process of reconstructing an image of the radioactive tracer distribution with high imaging resolution and sensitivity can be implemented using this information.

[0186] One step in reconstructing the image is determining the backprojection operator. The backprojection operator uses a probability density function (pdf) to determine where detected radiation events originate from within the coded aperture and the outer volume of the shield. To calculate the pdf of the detected events, a radiation transmission map or attenuation coefficient map through the coded aperture and the shield is determined at the location where radiation was detected.

[0187] In one embodiment, an attenuation map is pre-calculated at least on a plane (2106) referred to as the instrument response reference plane, or only on the reference plane. The reference plane may be parallel to the first mask plane or may be at any angle to the first mask plane. Furthermore, the plane may be attached to any physical component, such as the surface of the sensor, the first mask plane, or it may be located at any other location in space. For illustrative purposes, in this figure, the reference plane is parallel to the first mask plane and coincides with the surface of the sensor. For each point or pixel within the reference plane, the attenuation map includes a radiation attenuation coefficient or other information that can be used to extract the attenuation coefficient, such as the path length through the mask material, to encode predetermined types of radiation at different angles in the aperture imaging field of view.

[0188] For each detected radiation, the 3D locations of the interaction, such as (2104) and (2105), are projected onto a reference plane along directions toward the coded mask field of view, e.g., directions (2107a)-(2107d) for interaction (2104) and directions (2108a)-(2108c) for interaction (2105). The result intersects with the reference plane (2106) to generate points (2109a)-(2109d), which are used to project the interaction (2104) along directions (2107a)-(2107d), respectively. The locations (2109a)-(2109d) are used to retrieve the attenuation coefficients through the mask along directions (2107a)-(2107d), respectively. In this example, the radiation path along all these directions includes a segment passing through the sensor from (2104) to (2109a)-(2109d). This path may not be captured by the pre-calculated attenuation coefficient at the reference plane (2106). In some implementations, these paths can be added to the pdf calculation to scale the intensity of the backprojection along each of those directions (2107a)-(2107d). The process of calculating the backprojection operator is particularly useful when employing list-mode image reconstruction algorithms. The pdf calculated from the attenuation coefficients of the interaction at locations (2104) and (2105) can be used for the reconstruction of gamma-ray source images using methods commonly used for image reconstruction, such as statistical iterative methods, algebraic iterative methods, analytical methods, or compressed sensing methods.

[0189] Although this embodiment depicts only a single reference plane, in other embodiments, multiple instruments can be used to respond to the reference plane to improve imaging performance.

[0190] In other embodiments, the reference plane may be located in other locations. Figure 22An example illustrates this situation. A position-sensitive sensor (2200) is shielded (2201a) and (2201b) on some sides not covered by the coded aperture. The coded aperture mask in a first plane comprises elements (2202a)-(2202h). The first plane of the mask is placed at a distance (2203) from the surface of the position-sensitive sensor (2200). In this case, the instrument response reference plane (2204) is selected to be the same as the first mask plane. The 3D position of the radiation interaction (2205) is projected onto the reference plane along relevant directions toward the coded mask field of view, such as directions (2206a)-(2206d). The resulting intersections with the reference plane (2204) create points (2207a)-(2207d), respectively. Positions (2207a)-(2207d) are used to retrieve attenuation coefficients through the mask along directions (2206a)-(2206d), respectively. In this example, the radiation path along the direction (2206a)-(2206d) includes segments passing through the sensor from (2205) to (2208a)-(2208d). This path may not be captured by the pre-calculated attenuation coefficient at the reference plane (2204). In some implementations, these paths can be added to the PDF calculation to scale the intensity of the backprojection along each of those directions (2206a)-(2206d). The process of calculating the backprojection operator is particularly useful when using list-mode image reconstruction algorithms in the reconstruction of radiation source distributions.

[0191] When it is necessary to reconstruct compartmentalized images of the distribution of radioactive sources, some embodiments may include using multiple instrument response reference planes. Figure 23An example using multiple reference planes is shown. The position-sensitive sensor (2300) is shielded (2301a) and (2301b) on some sides not covered by the coded aperture. The coded aperture mask in the first plane includes elements (2302a)-(2302h). The first plane of the mask is placed at a distance (2303) from the surface of the position-sensitive sensor (2300). A first radiative interaction (2304) and a second radiative interaction (2305) are detected by the sensor (2300) at position resolution in all three dimensions. In this embodiment, five instrument response planes (2306a)-(2306e) can be used in the analysis. In this embodiment, the reference planes sample the sensitive volume of the sensor. The back-projection directions toward the field of view of the coded mask are represented by (2307a)-(2307d) for the interaction (2304) and (2308a)-(2308c) for the coherence (2305). In this case, the attenuation coefficients along directions (2307a)-(2307d) and (2308a)-(2308c) are extracted from attenuation data calculated at the reference plane and the location closest to the interaction point. Similarly, the attenuation coefficient from the interaction (2304) along direction (2307a)-(2307d) can be extracted from attenuation data calculated at the location closest to (2304) or the reference plane (2306d) of the chamber, and the attenuation coefficient from the interaction (2305) along direction (2308a)-(2308c) can be extracted from attenuation data calculated at the location closest to (2305) or the reference plane (2306b) of the chamber. These extracted attenuation coefficients can then be used to construct a PDF for the added chamber. This sampling and PDF calculation scheme can also be used for list mode imaging.

[0192] When performing compartmentalized image reconstruction, the total compartmentalized count in the plane (2304d) closest to (2304) can be increased as a result of the interaction detected in (2304), and the total compartmentalized count in the plane (2304b) closest to (2305) can be increased as a result of the interaction detected in (2305). The pdf can then be calculated along the directions (2307a)-(2307d) and (2308a)-(2308c) respectively to obtain the compartmentalized intensity.

[0193] Whether in list mode or compartment mode, the generated PDF can be used with any suitable iterative or analytical image reconstruction algorithm known in the art. Thus, by employing methods commonly used for image reconstruction, such as statistical iterative methods, algebraic iterative methods, analytical methods, or compressed sensing methods, the PDF can be calculated using the counts at the compartments in the reference plane closest to the interaction positions (2304) and (2305) in the reconstruction of the gamma-ray source image.

[0194] Figure 24 A perspective view of a large field-of-view coded aperture mask (2400) fabricated from multiple thin mask layers stacked and secured together is shown. The light gray layer (2401) represents the top layer, the dark gray layer (2402) represents the middle layer, and the light gray layer (2403) represents the bottom layer. In this embodiment, the mask pixel elements are self-supporting. In embodiments where the mask elements are not self-supporting, a frame with low radiation attenuation characteristics can be used to hold the mask elements in the desired position. As described above, this fabrication method can produce mask elements with stepped sides, but can generally follow the biconical shape of the mask elements.

[0195] In some embodiments, the layers in the coded aperture mask are not fixed to each other, but can be laterally moved relative to each other by actuators to produce various imaging fields of view and coded profiles. The actuators can be computer-controlled. The actuators can also move the layers apart to increase collimation effects. Similarly, in some embodiments, each layer can be formed of multiple plates with patterned holes that can be moved relative to each other.

[0196] In some embodiments, where the gamma sensors may not be equipped to deliver the location of gamma-ray interactions at depth resolution, the sensors may be positioned at different angles relative to each other and relative to a large field-of-view mask to minimize the range of deviations of the detected photon incident direction from the normal angle. This minimization will reduce imaging errors associated with interactions at unknown depths. Figure 25A cross-sectional side view of a large field-of-view coded aperture mask (2500) placed in front of two sensors (2501) and (2502) is shown. These sensors are arranged in different planes relative to each other and relative to the mask (2500) to minimize the angular range (2503) formed by the incident gamma-ray photon directions (2504), (2505), and (2506) relative to the normals of the sensors (2507) and (2508). In this embodiment, the normal directions (2507) and (2508) of the sensors converge towards the space of substantially adjacent areas of the center of the mask (2500). For most sensors in a sensor group, other embodiments can be envisioned where the sensors are oriented in different planes relative to each other and relative to the mask to minimize the angular range formed by the incident gamma-ray photon directions relative to the sensor normals. The mask can be a large field-of-view mask as described above, or an adjustable field-of-view mask, a focusing collimating mask, or a combination thereof, as described below. The angular range formed by the sensor normals can be from 0° to 90°. In some configurations, the angle between the normal directions of the two sensors is between 30° and 60°. In other configurations, the angle between the normal directions of the two sensors is between 40° and 50°.

[0197] In some embodiments, the mask itself may be made of mask segments located in a plurality of substantially non-coplanar planes.

[0198] Figure 26 A top view shows an arrangement of four sensors or sensor panels located in different planes from a mask (not shown), whose normal directions converge substantially toward a volume region adjacent to the mask. This arrangement minimizes the angular range of the incident gamma-ray photon direction relative to the sensor normal. In this description, sensor corners (2601a-d) are closer to the mask, and sensor corner (2602) is farther from the mask.

[0199] In some applications, the aforementioned gamma-ray sensor panel may need to scan a patient to create a 3D molecular map and provide real-time images of reduced volume within the patient. Both imaging modalities may require a gamma-ray imaging architecture that provides a wide field of view in scanning mode and a collimated, narrow field of view in real-time imaging mode. Similarly, the ability to scan a portion of the patient with both wide and narrow fields of view to create a more accurate 3D map can be advantageous. Field-tunable coded aperture masks can meet these requirements.

[0200] Figure 27A cross-sectional side view of an coded aperture imaging system is shown, which includes a gamma-ray mask with an adjustable field of view. An imaging gamma-ray sensor array (2700) is positioned behind the mask assembly, which comprises multiple overlapping layers made of a high-density, high-atomic-number material, such as a tungsten alloy. In this example, the mask comprises three layers. The middle layer of the mask (2701) is made of a plate with holes or openings that penetrate from one side of the plate to the other to create a patterned plate. Figure 31 and 32 An example of a pattern formed by an opening is shown. The pattern could be as follows: Figure 31 The pseudo-random arrays and exponential aperture arrays shown are as follows: Figure 32 The illustration shows an assembly comprising curved slits of various curvatures, or another pattern with substantially flat sidelobes in the autocorrelation function at multiple magnifications. Maintaining substantially flat sidelobes at multiple magnifications is important because the imager will be exposed to sources at very close field distances to intermediate distances, thus generating projections onto the detector array through masks at multiple magnifications. In this illustration, the top layer is made of two patterned plates (2702a) and (2702b). The patterns on the patterned plates (2702a) and (2702b) can substantially spatially match the pattern of the intermediate layer (2701), although in some embodiments, differences may exist between the overlapping patterns. In this illustration, the bottom layer is made of two patterned plates (2703a) and (2703b). The patterns on the patterned plates (2703a) and (2703b) can substantially spatially match the pattern of the intermediate layer (2701), although in some embodiments, differences may exist between the overlapping patterns. The set of overlapping mask layers can be mounted on side collimators (2704a) and (2704b). These side collimators can also be made of gamma-ray attenuating materials, such as tungsten alloys. The mask can be positioned at a focal length (2705), which can be changed by a computer-controlled actuator that is operatively coupled. Openings in the mask layers in the cross-section are represented by gray areas (2706). As mentioned above, the mask layers can be made of frustum-shaped elements. Also as mentioned above, these frustum-shaped elements can have straight or rounded edges, and their specific shapes can vary from one layer to another, from a portion of one layer to another of the same layer, or from one element to another. In some embodiments, the pattern can be as follows: Figure 32 As shown. This arrangement of layers close to each other allows for far-field imaging, which allows for vertical projection onto the mask (2707), as well as projection at approximately +45° (2708) and at -45° (2709).

[0201] Figure 28A cross-sectional side view of an coded aperture imaging system with adjustable 3-layer masks is shown, in collimated (central concave) field-of-view configuration. The imaging gamma-ray sensor array (2800) is placed... Figure 27 Behind the mask assembly described in [the text]. In this embodiment, Figure 7 The intermediate layer of the mask (2801) or (2701) in the middle is positioned relative to the sensor. Figure 7 The same relative positions are maintained. However, the top layer made of patterned plates (2802a) and (2802b) has been removed from the intermediate layer by an actuator operably coupled to the computer. In some embodiments, plates (2802a) and (2802b) can also be laterally moved relative to the intermediate layer by an actuator operably coupled to the control computer. In this case, the two plates move laterally toward the center of the mask. The bottom layer made of patterned plates (2803a) and (2803b) has also been removed from the intermediate layer by an actuator operably coupled to the computer. In some embodiments, plates (2803a) and (2803b) can also be laterally moved relative to the intermediate layer by an actuator operably coupled to the control computer. In this case, the two plates move laterally toward the edge of the mask. The focal length of the mask is represented by (2805). The net effect of these translations is a reduction in the field of view toward the center of the mask because the separation of the layers produces a collimation effect. The remaining main projection directions of the opening represented by the gray area (2806) are represented by directions (2807) and (2808).

[0202] Figure 29A and 29B A schematic top view of an adjustable coded aperture mask in two configurations is shown. The mask consists of nine panels arranged in three layers, with four layers at the top, one layer in the middle, and four layers at the bottom. Figure 29A The mask is configured in a wide field of view, while Figure 2B The mask is configured in a narrow collimated field of view. For clarity, the elements and movements are not depicted to scale. The purpose of this figure is to illustrate an example of the lateral movement of the plate as it transitions from a wide field of view to a narrow field of view. In the wide field of view configuration, the outline of the intermediate layer is represented by a solid line (2900). The outlines of the four plates forming the bottom layer (facing the detector) are represented by dashed squares (2901a-d). The outlines of the four plates forming the top layer (away from the detector) are represented by dashed squares (2902a-d). This configuration brings the layers close to each other and can potentially create a large field of view opening, especially in the left-right direction.

[0203] The outline of the middle layer in the narrow field-of-view configuration is represented by a solid line (2903). The outlines of the four plates forming the bottom layer (facing the detector) are represented by dashed squares (2904a-d). The outlines of the four plates forming the top layer (away from the detector) are represented by dashed squares (2905a-d). This configuration keeps the layers far apart from each other and can produce a narrow collimated field of view toward the center of the mask. Other lateral movements may also occur. For example, to produce collimation away from the center of the mask in any direction of interest, plates within the same layer can be moved laterally together in the same direction. For example, to move the collimation of the mask to the right of the figure, the top plate (2905a-d) will be moved from... Figure 29B The configuration moves to the right, and the base plate (2904a-d) will be from... Figure 29B The configuration is shifted to the left. Other plate movements may achieve similar effects depending on the precise alignment of the patterns between layers. Similarly, the same plate movement can achieve very different collimation and field-of-view changes depending on the precise alignment of the patterns between layers. For example, in some implementations, lateral movement of the plate across different layers can reduce or increase the effective aperture fraction of the mask when the patterns across layers have certain alignments.

[0204] Figure 30A and 30B A schematic top view of an adjustable coded aperture mask in two configurations is shown. The mask consists of 19 panels arranged in three layers: nine at the top, one in the middle, and nine at the bottom. 30A is in the wide field of view, and 30B is in the narrow collimated field of view. For clarity, elements and movements are not depicted to scale. This figure aims to illustrate an example of lateral movement of the 3×3 panel array as it transitions from a wide field of view to a narrow field of view. In the wide field of view configuration, the outline of the middle layer is represented by a solid line (3000). The outlines of the nine panels forming the bottom layer (facing the detector) are represented by dashed squares (3001a-i). The outlines of the nine panels forming the top layer (away from the detector) are represented by dashed squares (3002a-i). This configuration brings the layers close together and can create a large field of view opening, particularly in the left-right direction. In the narrow field of view configuration, the outline of the middle layer is represented by a solid line (3003). The outlines of the nine plates forming the bottom layer (facing the detector) are represented by dashed squares (3004a-i). The outlines of the nine plates forming the top layer (away from the detector) are represented by dashed squares (3005a-i). This configuration keeps the layers far apart and produces a narrow collimated field of view towards the center of the mask. Figure 29A As shown in –29B, other lateral movements and effects may also occur.

[0205] Figure 31A top view of a layer in an adjustable multilayer mask with a pseudo-random pattern having an aperture fraction of 22% is shown. As described above, the pixel elements can be biconical frustums with straight or rounded edges. The pattern can be optimized to provide substantially flat sidelobes in the autocorrelation function across multiple magnifications, for example, for any combination of magnifications from 1x to 6x. The aperture fraction of the mask can range from 0.1% to 70%. In a preferred embodiment, the aperture fraction is 5% to 30%.

[0206] Figure 32 A top view of an coded aperture mask comprising curved slits of various curvatures is shown. The combination and arrangement of the slits can be optimized to provide substantially flat sidelobes in the autocorrelation function at multiple magnifications, for example, for any combination of magnifications from 1x to 6x. The aperture fraction of the mask can range from 0.1% to 70%. In a preferred embodiment, the aperture fraction is 5% to 30%. The slits can have straight edges, rounded edges, or various other contours, such as a V-shaped contour, similar to the circular biconical frustum described above. The shape of the edges can vary from one slit to another or within the same slit. Circular slits can intersect or not. The curvature of the slits can vary within the same slit and between slits. This slit geometry with varying curvature can contribute to achieving flat sidelobes in the autocorrelation function at multiple magnifications.

[0207] Figure 33 A diagram of a handheld SPECT camera device (3300) is shown. The handheld instrument (3300) includes a position-sensitive sensor (3301) positioned behind a large field-of-view coded aperture mask (3302) and between shields (3303a) and (3303b) placed in a location not covered by the mask (3302). The imaging system is characterized by a large imaging field of view (3304). This particular instrument also includes a camera or scanner (3305) oriented to collect contextual information that can be used to create a 3D model of the patient and to position the handheld camera system relative to the patient. A handle (3306) can be used to easily move the device.

[0208] Figure 34 It shows Figure 33 A top view of the handheld instrument (3400) is shown. The handheld instrument (3400) is shown to include a position-sensitive sensor (3401) placed behind a large field-of-view coded aperture mask (3402) and located between shields (3403a) and (3403b) placed in areas not covered by the mask (3402). The imaging system is characterized by a large imaging field of view (3404).

[0209] use Figure 33 and Figure 34The handheld SPECT camera device described herein can reconstruct high-resolution SPECT images by utilizing the high resolution, high field of view, and high sensitivity characteristics of the various imaging embodiments presented herein. Figure 35 This illustration shows a modality where the instrument can perform SPECT imaging from a limited range of directions around the patient, yet still retains a significant projection range for each voxel in the field of view. The handheld imaging device is moved at different positions (3500a)-(3500d) on one side of the patient (3501). The figure illustrates how voxels (3502) within molecularly labeled organs (3503) in the body are imaged by the device at all positions (3500a)-(3500d) due to the large field of view of the imaging system. An example of the projection lines (3504a)-(3504d) toward the SPECT camera at positions (3500a)-(3500d) shows an angular range that can even exceed 90 degrees (e.g., between directions (3504a) and (3504d), which is sufficient parallax for reconstructing high-resolution images in all three coordinates.

[0210] In some embodiments, a SPECT camera can be moved around and along the patient using a robotic arm or other mechanical system.

[0211] Therefore, the embodiments of the imaging modal described herein allow for simpler, lighter, and more economical SPECT imaging devices with improved imaging performance.

[0212] Figure 36 A flowchart summarizing some systems and methods implemented by a portable molecular imaging system is shown. In this case, we illustrate the process assuming a portable SPECT imaging system. The first step (3601) of the process is to inject a molecular reagent into the patient, in this example, a SPECT molecular reagent. In the second step (3602), the SPECT system described herein is used to scan a portion of the patient to provide data used in step (3603) by a computer operatively coupled to construct a 3D map of the molecular reagent. The scan target will be selected by the user from a set of scan targets stored in the computer memory. As described above, Figure 1 and Figure 13 The computer vision system described herein can transmit data to a computer to create a 3D model of a patient's body. Given a scanning target, the computer can use the 3D model of the patient to create a scanning protocol. The scanning protocol will include a set of actions taken by a robotic arm and sensor panels as they scan the patient. This set of actuations may also include instructions for actuators to move gamma-ray mask elements and the mask focal length. This set of actuations may also include instructions for movement... Figure 1 and 13 The instructions for the actuator of the portable trolley shown.

[0213] SPECT systems can use adjustable masks in wide field-of-view configurations (see...). Figure 27 However, for some applications, the mask-detector assembly can be in a different, narrower field-of-view configuration, or it can be changed at different points during scanning. These changes can be controlled by the computer or user during scanning. Similarly, for some applications, the focal length between the mask and detector can be changed at different points during scanning (3602). These changes can be controlled by the computer or user during scanning.

[0214] To support the image reconstruction process, imaging scans can be performed using other instruments (3604) to create imaging datasets, such as anatomical imaging datasets, which can be used by a computer operatively coupled to the SPECT scanner. For example, CT datasets can be used to provide attenuation maps for the computer to use in creating more accurate reconstructions of molecular images.

[0215] like Figure 13 The co-registration system (3605) and method described herein can be used to merge SPECT data with other imaging data. To improve co-registration, particularly when there is a significant discrepancy between the 3D model of the patient's body and a 3D model extracted from an external image dataset, a (3606) co-registration ultrasound scan can be performed to allow the computer to fix specific structures in other imaging modalities to specific locations, such as... Figure 13 As described above. This additional imaging dataset can be rendered and sent to a display for user (3607) examination. Similarly, the computer can send a rendered reconstructed 3D molecular map to a visualization device for user examination. The visualization device can be a screen, a head-mounted display, an augmented reality device, or another visualization system. The computer can send combinations of other anatomical and molecular images to the visualization device. The computer can also store the molecular image dataset, the ultrasound image dataset used for auxiliary co-registration, and other anatomical image datasets in memory.

[0216] Following 3D image reconstruction (3603), using the reconstructed data, the computer can use instructions stored in memory to control the molecular imaging system to perform subsequent scans (3608) to improve the quality of the reconstructed 3D image. For repeat scans, the computer can create another scan protocol that includes a new actuated dataset. For example, the computer can send instructions to the robotic arm and panel to scan the patient, or it can send instructions to the mask actuator to change the field of view or the focal length of subsequent scans. For example, rescanning a specific part of the body in a foveal, narrow field-of-view mode to obtain better molecular image contrast in a specific region of the body where features of interest may exist may be beneficial. The scan protocol can be sent to the user for approval. Furthermore, given the data presented to the user after the first scan, the user can initiate repeat scans. In some implementations, the user may be remote. The computer can use data from molecular imaging scans and / or from other co-registered anatomical imaging scans and / or from co-registered ultrasound scans to perform neural network or deep learning analysis of the data to determine the utility of subsequent scans and perform classification of imaging features. The results of the analysis can be presented to the user. The results may include rendering of the co-registered image dataset, as well as rendering of the processed fused image, which may contain classification values. Statistical, deep learning, and sensor fusion algorithms can be used by computers for this purpose.

[0217] Following the visualization step (3607), the user may want to obtain real-time images of certain molecular structures. Using the user interface, in step (3609), the user can select such structures of interest identified in the presentation of the co-registered dataset. The selected features will be characterized in 3D coordinates.

[0218] In step (3610), using instructions stored in memory, the computer actuates any one of the robotic arm, sensor panel, and mask actuator to orient the sensor toward the selected feature. For example, the mask can be activated to produce a narrow field of view collimated toward the feature of interest, thereby maximizing the imaging sensitivity and signal-to-noise ratio of the region of interest. As the computer calculates the movement of the panel toward its location from which it can acquire data from the region of interest, it considers a 3D model of the patient's body, ensuring that no part of the scanner, including the panel, collides with the patient.

[0219] At this point, in step (3611), the data collected from the molecular imaging sensor can be analyzed by a computer to create images in near real-time. As described above, the computer can use previously scanned molecular 3D datasets, such as the datasets generated in steps (3603) or (3608), combined with the real-time molecular data delivered by the molecular sensor in step (3611) to improve the quality of the real-time molecular image rendering. Similarly, as described above, co-registered ultrasound scans can be performed in step (3612) to provide an anatomical background to the molecular images. The molecular images to be enhanced onto the ultrasound images can be delivered via steps (3603) or (3608), or can be the real-time images generated in step (3611).

[0220] Furthermore, in step (3609), the user can select 3D co-registration features of interest using a visualized ultrasound scan in the user interface. The user can select such structures of interest identified in any presentation of the co-registered imaging dataset. In step (3613), the computer uses tracking data, such as that provided by a computer vision system, to co-register the ultrasound dataset with in vivo molecular images, as... Figure 1 Or as described in 13.

[0221] In some implementations, the computer may use a real-time ultrasound scan delivered in step (3612) to create a tissue deformation model, which is used in the construction and rendering of a stored 3D molecular image dataset (from step (3603) or (3608)) or real-time molecular images (from step (3611)). Details of the deformation modeling process are as described above.

[0222] In step (3614), the computer sends the ultrasound scan and the rendering of the co-registered molecular image scan to a visualization device for user examination. In some embodiments, in step (3615), an intervention guided by the molecular imaging system can be performed. For example, the intervention can use the rendering of the molecular image to highlight the target of interest. In another example, the intervention can use the rendering of the molecular image enhanced on real-time ultrasound or other real-time imaging modalities to highlight the target of interest. Such interventions can include biopsies, ablation, resection, radiotherapy, or other medical procedures. The intervention can be selected from the group consisting of: interventions performed manually by the user using a needle, ablation system, surgical device, or other medical device; interventions performed by a co-registered high-intensity focused ultrasound system for the treatment area; interventions for guided biopsies and surgeries via a co-registered stereotactic system; interventions for guided resections and surgeries by a robotic medical system; interventions for guided resections and surgeries by a laparoscopic system; and interventions for treating tumors by a co-registered radiotherapy device. For example, the molecular image enhanced onto the ultrasound image can guide the user toward the feature of interest to drive the needle, ablation system, or another medical device. Co-registration with other imaging and treatment systems can be accomplished using an onboard computer vision camera, which is another co-registration and tracking device. In some implementations where the user performing the intervention (3615) is a robot or an automated system, step (3614) can be skipped.

[0223] Molecular imaging systems can be used in conjunction with imaging modalities other than ultrasound for diagnostic and interventional guidance. For example, co-registered optical medical systems such as endoscopes, bronchoscopes, laparoscopes, colonoscopes, microscopes, robotic endoscopes, and robotic laparoscopes can be used. In step (3616), such an optical imaging system is used to image the patient.

[0224] In step (3617), the computer co-registers the optical instruments with the molecular imaging system. If the optical medical imaging equipment is rigid, Figure 1 and 13 The airborne computer vision system described herein can be used to locate medical imaging devices relative to computer vision systems and molecular imaging systems. Tags and markers can be affixed to these devices to aid in the positioning and tracking of optical medical devices.

[0225] If the optical system is flexible, or it is not easily within the field of view of the computer vision system, other modalities can be used to create co-registration. For example, if the optical medical device is an endoscope camera or a flexible laparoscopic camera, in step (3618), the fluoroscope can be used to infer the position of the fluoroscope camera relative to the fluoroscope's X-ray source and sensor by having the computer load the fluoroscope image and analyze features associated with the endoscope structure, thereby determining the position and orientation of the endoscope relative to the fluoroscope reference system. In some implementations, the optical medical device may already be co-registered with an X-ray system (e.g., a fluorescence system). In this case, it is not necessary for the computer to analyze the fluorescence image to infer the position of the optical medical device.

[0226] A reference label, including features identifiable in the fluorescein image, can be positioned within the fluorescein's field of view, for example, on or near the patient. The reference label may also include features identifiable by an onboard computer vision system. A computer operatively connected to the computer vision system can use computer vision data to determine the position of the computer vision camera relative to the reference label. The computer can read and analyze the fluorescein image to extract the position of the reference label relative to the fluorescein. The computer can then use co-registration between the optical imaging camera and the fluorescein, the position of the fluorescein relative to the label, and the position of the label relative to the computer vision system to determine the position of a laparoscopic or endoscopic medical optical camera relative to the computer vision camera. This allows for co-registration between images captured by the optical medical camera and molecular images. In step (3619), the computer can send a rendering of the co-registered image captured by the optical medical device and the molecular imaging device to a visualization device. In some embodiments, the molecular image will be rendered by the computer in a perspective projection rendering to always match the position, orientation, and focal length of the medical optical camera. This will produce rendered molecular imaging data suitable for enhancement onto real-time images captured by the optical medical device. Rendering such as maximum intensity projection can be used to render the molecular image. Other tracking systems can be used to co-register the X-ray system with the molecular imaging system. In step (3620), guided by enhanced rendering of real-time optical and molecular images transmitted from the optical medical device, interventions such as biopsies, ablation, resection, and surgery can be performed. If the medical intervention (3620) is automated or robotic, step (3619) can be skipped.

[0227] While the above description includes many features, these features should not be construed as limiting the scope, but rather as examples of one or more embodiments thereof. Many other variations are also possible.

[0228] In some embodiments, a computer system may be used to implement any of the entities or components described above. The computer system includes a central processing unit (CPU) for communicating with each subsystem and controlling the execution of instructions from system memory or a fixed disk, as well as information exchange between other subsystems of the computer system. As used herein, the processor includes a single-core processor, a multi-core processor on the same integrated chip, or multiple processing units on a single circuit board or network. System memory and / or a fixed disk may contain computer-readable media. The computer system may also include input / output (I / O) devices. The computer system may include a network interface that can be used to connect the computer system to a wide area network such as the Internet.

[0229] Storage media and computer-readable media used to contain code or code portions may include any suitable media known or used in the art, including storage media and communication media, such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology, for storing and / or transmitting information, such as computer-readable instructions, data structures, program modules or other data, including RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, cassette tape, magnetic tape, disk storage or other magnetic storage devices, data signals, data transmission, or any other medium that can be used to store or transmit the required information and is accessible by a computer.

[0230] Any software component or function described in this application may be implemented as software code executed by a processor using any suitable computer language, such as Java, C, C++, C#, Objective-C, Swift, or a scripting language using, for example, traditional or object-oriented techniques, such as Perl or Python. The software code may be stored as a series of instructions or commands on a computer-readable medium for storage and / or transmission. Suitable non-transitory computer-readable media may include random access memory (RAM), read-only memory (ROM), magnetic media such as hard disk drives or floppy disks, or optical media such as optical discs (CDs) or DVDs (Digital Versatile Discs), flash memory, etc. The computer-readable medium may be any combination of such storage or transmission devices.

[0231] The use of “one,” “an,” or “the” is intended to mean “one or more” unless otherwise specified. The use of “or” is intended to mean “inclusive or” rather than “exclusive or” unless otherwise specified.

[0232] Use terms such as "first," "second," "third," and "fourth." These can be used to distinguish individual elements and do not necessarily imply an order or hierarchy among these elements unless otherwise specified.

Claims

1. An imaging system comprising: a robotic arm configured to be positioned to a desired location by a user; at least one gamma camera panel connected to the arm, wherein the gamma camera panel comprises a gamma camera sensor having a position and energy sensing resolution, wherein the gamma camera panel provides an imaging field of view greater than 15 degrees; a camera mounted to view at least a portion of a patient; and at least one processor and a memory operably coupled with the at least one processor, the camera, and the gamma camera sensor, the memory having instructions executed by the at least one processor that cause the at least one processor to: read first gamma ray photon sensing events received from the gamma camera sensor; provide a first position and orientation of the gamma camera panel relative to the patient's body; co-register the first gamma ray photon sensing events with the patient's body using the first position and orientation; read second gamma ray photon sensing events received from the gamma camera sensor; provide a second position and orientation of the gamma camera panel relative to the patient's body, co-register the second gamma ray photon sensing events with the patient's body using the second position and orientation; and reconstruct a 3D distribution of a radioisotope emitting gamma rays within the patient's body by using the first and second co-registered sensing events. The robotic arm is a computer controllable 6-axis robotic arm. The memory further has instructions that cause the at least one processor to actuate the robotic arm to perform a SPECT scan.

2. The system of claim 1, wherein, The wheels are actuated to move the imaging system during a scan of a patient.

3. The system of claim 1, wherein, The proximity sensor is operably coupled to the at least one processor and memory, wherein movement of the panel relative to the patient is modified according to proximity sensor data obtained by the proximity sensor.

4. The system of claim 3, further comprising a wheel actuated by a motor coupled to the at least one processor and memory, wherein, 6. The system of claim 1, further comprising a movable scan table configured to move the patient relative to the at least one gamma camera panel.

5. The system of claim 3, further comprising a proximity sensor mounted on the panel, wherein, The first position and orientation of the gamma camera panel is different than the second position and orientation of the gamma camera panel. The memory further has instructions that cause the at least one processor to:

7. The system of claim 1, wherein, create 3D models of the patient's body at regular intervals using data from the camera; and 8. The system of claim 1, wherein, detect changes in the body from one 3D model of the body to another 3D model of the body. The memory further has instructions that cause the at least one processor to notify a user of a significant body change that can require a reset of a SPECT scan. The memory further has instructions that cause the at least one processor to:

9. The system of claim 3, wherein, determine a first 3D model of the patient's body and assign it to the first gamma ray photon sensing events, 10. The system of claim 8, wherein, determine a second 3D model of the patient's body and assign it to the second gamma ray photon sensing events, create a tissue deformation model from the first to the second 3D model of the patient's body, and reconstruct a 3D distribution of a radioisotope emitting gamma rays within the patient's body by using the first and second gamma ray photon sensing events and the tissue deformation model. ​ ​ 11. The system of claim 1, wherein, The memory also has instructions that cause the at least one processor to stop movement of the mechanical arm and the gamma camera panel to avoid a collision between a component of the imaging system and the patient body.

12. The system of claim 11, wherein, The memory also has instructions that cause the at least one processor to: monitor a space of other objects or people in a projection path of the gamma camera panel, and stop movement of the mechanical arm and the gamma camera panel to avoid a collision between any component of the imaging system and the other objects or people.

13. The system of claim 1, further comprising: an ultrasound probe; a tracking system, wherein the tracking system is used to determine a position and orientation of the ultrasound probe relative to the patient body; a visualization device; at least one ultrasound processor and an ultrasound memory, wherein the at least one ultrasound processor and the ultrasound memory are each operably coupled with the ultrasound probe, the tracking system, the memory storing a 3D distribution of a gamma ray-emitting radioisotope that is co-registered with the patient or a reference point, and the visualization device, the ultrasound memory having instructions executed by the at least one ultrasound processor that cause the at least one ultrasound processor to: track the ultrasound probe relative to the patient or relative to the reference point; determine a co-registration between the 3D distribution of the gamma ray-emitting radioisotope and an ultrasound scan using ultrasound probe tracking data obtained by the ultrasound probe; and deliver to the visualization device an image that includes a feature enhancement of the 3D distribution of the gamma ray-emitting radioisotope on the ultrasound scan.

14. The system of claim 13, wherein, The tracking system is selected from the group consisting of: an optical tracking system, an electromechanical tracking system, an electromagnetic tracking system, an ultrasound tracking system, a depth imaging tracking system, and combinations thereof.

15. The system of claim 13, wherein, The ultrasound memory also has instructions that cause the at least one ultrasound processor to: read a molecular image data set from the memory, create a model of movement of a feature in the image from a first ultrasound frame to a second ultrasound frame, create a changed molecular image based on the model of movement of the feature in the image, and enhance the changed molecular image onto the second ultrasound frame.

16. An imaging system, comprising: at least one gamma camera panel connected to a moveable arm, wherein the gamma camera panel includes a gamma camera sensor having a position and energy sensing resolution; an ultrasound transducer positionable to have a field of view that at least partially overlaps a field of view of the gamma camera panel; a tracking system capable of providing tracking information regarding a relative position of the ultrasound transducer relative to the gamma camera panel; and at least one processor and a memory operably coupled with the gamma camera sensor, the ultrasound transducer, and the tracking system, the memory having instructions executed by the at least one processor that cause the at least one processor to: read first gamma ray photon sensing events received from the gamma camera sensor; read second gamma ray photon sensing events received from the gamma camera sensor; reconstruct a 3D distribution of a gamma ray-emitting radioisotope within the patient by using the first and second gamma ray photon sensing events; determine a co-registration between the ultrasound transducer and the gamma camera sensor using the tracking information; and deliver to the visualization device an image that includes a feature enhancement of the 3D distribution of the gamma ray-emitting radioisotope on the ultrasound scan. determining co-registration between a 3D distribution of a gamma ray emitting radioisotope and an ultrasound scan using co-registration between an ultrasound transducer and a gamma camera sensor; and delivering an image to a visualization device including augmenting the 3D distribution of the gamma ray emitting radioisotope onto the ultrasound scan using co-registration between the 3D distribution of the gamma ray emitting radioisotope and the ultrasound scan.

17. The system of claim 16, wherein, co-registering between the 3D distribution of the gamma ray emitting radioisotope and the ultrasound scan includes pinning one or more features of the 3D distribution of the gamma ray emitting isotope to one or more features of the ultrasound scan.

18. The system of claim 16, wherein, the gamma camera panel provides an imaging field of view greater than 15 degrees, the imaging field of view defined as an angular range relative to a direction having maximum imaging sensitivity of the gamma camera panel from which gamma photons can be detected and imaged by a gamma camera sensor included in the gamma camera panel having a sensitivity greater than one percent of the maximum imaging sensitivity.

19. The system of claim 16, wherein, the reconstructed 3D distribution of the gamma ray emitting radioisotope is characterized by a limited depth of imaging resolution along a most sensitive direction of the gamma camera panel of less than 20 mm over a range of distances covering at least 50 millimeters (mm) from a substantially static position of the gamma camera panel relative to a patient.

20. The system of claim 16, wherein, the ultrasound transducer is fixed to a patient's body without requiring a user to hold the ultrasound transducer.

21. The system of claim 16, wherein, the ultrasound transducer is mounted to a movable arm.

22. The system of claim 16, wherein, the ultrasound transducer is mounted to a mechanical arm configured to automatically move relative to a patient.

23. The system of claim 16, wherein, the memory also has instructions executed by the at least one processor that cause the at least one processor to: read gamma ray photon sensing events received from the gamma camera sensor, thereby detecting gamma ray photons; correlate the ultrasound image map with the gamma ray photon sensing events; provide a tissue model from the ultrasound image map; create a gamma photon attenuation model from the tissue model; and use the gamma photon attenuation model to determine a probability of attenuation of the detected gamma ray photons within the patient.

24. The system of claim 16, comprising first and second gamma camera panels mounted at a distal end of one or more movable arms, wherein, the panels are actuatable such that a distance between the panels is modifiable, wherein the panels are actuatable such that a relative angle between the panels is modifiable, and / or wherein the panels are separable to allow positioning of another medical instrument between the panels.

25. The system of claim 24, wherein, the medical instrument is an ultrasound probe.

26. The system of claim 24, wherein, the medical instrument is a biopsy needle.

27. The system of claim 24, wherein, the medical instrument is an ablation therapy device.

28. An imaging system comprising: a mechanical arm configured to be positioned to a desired position by a user; at least one gamma camera panel connected to the arm, wherein the gamma camera panel includes a gamma camera sensor having a position and energy sensing resolution; a camera mounted to view at least a portion of a patient; and at least one processor and a memory operably coupled with the at least one processor, the camera, and the gamma camera sensor, the memory having instructions executed by the at least one processor that cause the at least one processor to: read first gamma ray photon sensing events received from the gamma camera sensor; reading first gamma ray photon sensing events received from a gamma camera sensor; reconstructing a 3D distribution of a gamma ray emitting radioisotope in a patient's body using the first and second gamma ray photon sensing events; co-registering the 3D distribution of the gamma ray emitting radioisotope with the patient's body using data from the camera to create a first image data set; obtaining a second image data set comprising anatomical scan data obtained via a medical imaging system; and co-registering the first image data set with the second image data set.

29. The imaging system of claim 28, wherein, The instructions further cause the at least one processor to deliver an image to a visualization device, the image comprising a feature of the 3D distribution of the gamma ray emitting radioisotope enhanced onto the second image data set.

30. The imaging system of claim 28, wherein, The anatomical scan data comprises one or more of: CT scan data, MRI scan data, or ultrasound scan data.

31. The imaging system of claim 28, wherein, Co-registering the first image data set and the second image data set comprises comparing a feature from the second image data set with a feature from the data from the camera.

32. The imaging system of claim 28, wherein, Co-registering the first image data set and the second image data set comprises using a fiducial identifiable in the data from the camera and also identifiable in the anatomical scan data obtained via the medical imaging system.

33. The imaging system of claim 28, wherein, Obtaining the second image data set comprising anatomical scan data obtained via a medical imaging system comprises obtaining a previously reconstructed 3D distribution of a gamma ray emitting radioisotope in a patient's body via the imaging system.

34. An imaging system comprising: at least one gamma camera panel; an ultrasound probe; a tracking system, wherein the tracking system is used to determine a relative position and orientation of the ultrasound probe; and at least one processor and a memory operatively coupled with the at least one processor, the gamma camera panel, and the ultrasound probe, the memory having instructions executed by the at least one processor that cause the at least one processor to: obtain a gamma ray image data set based on data from the at least one gamma camera panel; obtain an ultrasound image data set based on data from the ultrasound probe; obtain an anatomical scan image data set; and co-register the gamma ray image data set and the ultrasound image data set based at least in part on the anatomical scan image data set.

35. An imaging system comprising: at least one gamma camera panel comprising a gamma camera sensor having position and energy sensing resolution, wherein the gamma camera panel provides an imaging field of view greater than 15 degrees; a camera mounted to view at least a portion of a patient; and at least one processor and a memory operatively coupled with the at least one processor, the camera, and the gamma camera sensor, the memory having instructions executed by the at least one processor that cause the at least one processor to: read first gamma ray photon sensing events received from a gamma camera sensor; provide a first position and orientation of the gamma camera panel relative to a patient's body; co-register the first gamma ray photon sensing events with the patient's body using the first position and orientation; read second gamma ray photon sensing events received from the gamma camera sensor; reconstruct a 3D distribution of a gamma ray emitting radioisotope in a patient's body using the first and second gamma ray photon sensing events; providing a second position and orientation of the gamma camera panel relative to the patient's body, co-registering second gamma ray photon sensing events with the patient's body using the second position and orientation; and reconstructing a 3D distribution of the radioisotope emitting gamma rays within the patient's body using the first and second co-registered sensing events.

Images