Systems and methods for multi-scan image processing and computer readable media

Abstract

A framework for multi-scan image processing. A single real anatomical image of a region of interest is first acquired. One or more emission images of the region of interest are also acquired. One or more synthetic anatomical images can be generated based on the one or more emission images. One or more deformable registrations of the real anatomical image to the one or more synthetic anatomical images are performed to generate one or more registered anatomical images. Then, attenuation correction can be performed on the one or more emission images using the one or more registered anatomical images to generate one or more attenuation-corrected emission images.

Description

Technical Field

[0001] This disclosure relates generally to image data processing, and more particularly to a framework for multi-scan image processing. Background Technology

[0002] Since X-rays were first used to identify anatomical abnormalities, the field of medical imaging has seen significant advancements. Medical imaging hardware has progressed in the form of newer machines, such as medical magnetic resonance imaging (MRI) scanners and computed axial computed tomography (CAT) scanners. Digital medical images are constructed using the raw image data obtained from these scanners. Digital medical images are typically either two-dimensional (“2D”) images composed of pixel elements or three-dimensional (“3D”) images composed of volumetric elements (“voxels”). Due to the vast amount of image data generated in any given scan, there has always been, and continues to be, a need for image processing techniques that can automate some or all of the processes to determine the presence of anatomical abnormalities in scanned medical images.

[0003] Multimodal imaging plays a crucial role in accurately identifying diseased and normal tissues. It provides combined benefits by fusing images acquired from different modalities. For example, the complementarity between anatomical (e.g., computed tomography (CT), magnetic resonance (MR)) and molecular (e.g., positron emission tomography (PET), single-photon emission computed tomography (SPECT)) imaging modalities has led to the widespread use of PET / CT and SPECT / CT imaging.

[0004] Serial PET / CT (or SPECT / CT) involves a sequence of multiple scans (or multiple scans) to assess the state of a region of interest over a period of time. Serial PET / CT can be performed in various settings, including dosimetry assessment and multi-tracer studies. However, repeated scans may expose patients to increased radiation risks. Summary of the Invention

[0005] This paper describes a framework for multi-scan image processing. First, a single ground-based anatomical image of the region of interest (ROI) is acquired. One or more emission images of the ROI are also acquired. One or more synthetic anatomical images can be generated based on the one or more emission images. One or more deformable registrations are performed from the ground-based anatomical image to the one or more synthetic anatomical images to generate one or more registered anatomical images. Then, attenuation correction can be performed on the one or more emission images using the one or more registered anatomical images to generate one or more attenuated-corrected emission images. Attached Figure Description

[0006] A more complete understanding of this disclosure and its many accompanying aspects will be readily obtained when considered in conjunction with the accompanying drawings, by referring to the following detailed description, as they become easier to understand.

[0007] Figure 1 A block diagram illustrating an exemplary system is shown;

[0008] Figure 2 An exemplary method for image processing is shown;

[0009] Figure 3a Exemplary front and rear views of a non-attenuation-corrected (NAC) PET image and its corresponding synthetic CT image are shown;

[0010] Figure 3b An exemplary comparison of real CT images and synthetic CT images is shown;

[0011] Figure 4a The front and rear views of exemplary images obtained by overlaying NAC PET images onto real CT images are shown.

[0012] Figure 4b The front and rear views of exemplary images obtained by overlaying NAC PET images onto registered real CT images are shown.

[0013] Figure 5a The illustration shows a front view and a rear view of an exemplary image obtained by overlaying a real CT image onto an inverted synthetic CT image; and

[0014] Figure 5b The front and rear views of an exemplary image obtained by overlaying a registered real CT image onto a reversed synthetic CT image are shown. Detailed Implementation

[0015] In the following description, numerous specific details, such as examples of specific components, devices, methods, etc., are set forth to provide a comprehensive understanding of how this framework is implemented. However, it will be apparent to those skilled in the art that these specific details need not be adopted to practice the implementation of this framework. In other instances, well-known materials or methods have not been described in detail to avoid unnecessarily obscuring the implementation of this framework. While this framework allows for various modifications and alternatives, specific embodiments thereof are shown by way of example in the accompanying drawings and will be described in detail herein. However, it should be understood that the invention is not intended to be limited to the specific forms disclosed, but rather, it is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Furthermore, for ease of understanding, certain method steps are depicted as separate steps; however, these separately depicted steps should not be construed as necessarily depending on the order in which they are performed.

[0016] Unless otherwise stated, as will be apparent from the following discussion, terms such as “segmentation,” “generation,” “registration,” “determination,” “alignment,” “positioning,” “processing,” “computation,” “selection,” “estimation,” “detection,” and “tracking” can refer to the actions and processes of a computer system or similar electronic computing device that manipulates data represented as physical (e.g., electronic) quantities within computer system registers and memories, and transforms that data into other data similarly represented as physical quantities within computer system memories or registers or other such information storage, transmission, or display devices. Embodiments of the methods described herein can be implemented using computer software. Sequences of instructions designed to implement these methods can be compiled for execution on various hardware platforms and for interfacing with various operating systems if written in a programming language conforming to recognized standards. Furthermore, no specific programming language is referenced in describing the implementation of this framework. It will be understood that various programming languages ​​can be used.

[0017] For brevity, an image corresponding to an object (e.g., tissue, organ, tumor, etc. of a subject (e.g., a patient, etc.)) or a portion thereof (e.g., a region of interest (ROI) in the image) can be referred to as the object or the object itself, or an image that includes the object or the object itself or a portion thereof (e.g., an ROI). For example, an ROI corresponding to an image of the lungs or the heart can be described as the ROI including the lungs or the heart. As another example, an image of the chest or an image including the chest can be referred to as a chest image, or simply as the chest. For brevity, the processing (e.g., extraction, segmentation, etc.) of a portion of an image corresponding to an object can be described as the object being processed. For example, the extraction of a portion of an image corresponding to the lungs from the rest of the image can be described as the lungs being extracted.

[0018] Sometimes CT scans are required solely for attenuation correction, and these scans may offer little diagnostic advantage. Registering initial CT images to subsequent molecular imaging scans can avoid the cumulative dose resulting from successive CT scans. However, due to the orthogonality between image pairs (i.e., lack of mutual information), reliable registration between functional PET / SPECT images and structural CT images using conventional methods is not feasible.

[0019] Molecular and CT scans are typically acquired sequentially, so patient movement between scans can lead to spatial mismatches between molecular and CT images. These spatial mismatches are largely unresolved in conventional systems. In some cases (e.g., cardiac), CT images are rigidly translated to provide a better match, but this only provides regional correction and rigid translation is not a realistic representation. Current efforts focus on estimating attenuation directly from emission data. This may be practical in limited scenarios where the underlying anatomy is relatively consistent (e.g., cardiac or brain studies), but it is not suitable for systemic oncology scenarios where solid masses may appear in unusual locations. For example, consider large solid tumors identifiable on CT images, but only a portion of these tumors exhibit significant radiotracer uptake. CT images will not be able to predict attenuating masses without radiotracer uptake.

[0020] This paper proposes a framework for multimodal image registration. According to one aspect, emission images (e.g., SPECT or PET) are registered to anatomical images (e.g., CT) by generating synthetic anatomical images from non-attenuation-corrected (NAC) emission images and using these synthetic anatomical images as targets for anatomical-to-anatomical image registration with true anatomical images. Deforming the true anatomical image to the synthetic anatomical image preserves all attenuation quality, and the true anatomical image can be made usable for comparison in terms of quality assurance. In regions without mutual information (e.g., anomalously solid masses without radiotracer uptake), the deformation field is guided through the surrounding structure and remains effective.

[0021] This framework can be used to provide robust PET / CT, SPECT / CT, PET / MR registration, or other hybrid modal registration. Advantageously, it can be used to reduce dose accumulation and / or total scan time, thereby reducing associated risks. In sensitive populations (e.g., pediatrics), reducing radiation dose accumulation due to consecutive CT scans is particularly important. These and other exemplary advantages and features will be described in more detail in the following description.

[0022] Figure 1 This is a block diagram illustrating an exemplary system 100. System 100 includes a computer system 101 for implementing the framework described herein. In some implementations, computer system 101 operates as a standalone device. In other implementations, computer system 101 may be connected (e.g., using a network) to other machines, such as multimodal medical scanner 130 and workstation 134. In a networked deployment, computer system 101 may operate as a server (e.g., in a server-client user network environment), as a client user machine in a server-client user network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.

[0023] In one implementation, computer system 101 includes a processor device or central processing unit (CPU) 104 coupled via an input-output interface 121 to one or more non-transitory computer-readable media 106 (e.g., computer storage or memory devices), a display device 108 (e.g., a monitor), and various input devices 109 (e.g., a mouse, touchpad, or keyboard). Computer system 101 may further include supporting circuitry such as caches, power supplies, clock circuitry, and communication buses. Various other peripheral devices, such as additional data storage devices and printing devices, may also be connected to computer system 101.

[0024] This technology can be implemented in various forms of hardware, software, firmware, dedicated processors, or combinations thereof, either as part of microinstruction code, or as part of an application or software product, or a combination thereof, which is executed via an operating system. In some implementations, the technology described herein is implemented as computer-readable program code tangibly embodied in one or more non-transitory computer-readable media 106. In particular, this technology can be implemented by an image processing module 117. The non-transitory computer-readable medium 106 may include random access memory (RAM), read-only memory (ROM), floppy disk, flash memory, and other types of memory, or combinations thereof. The computer-readable program code is executed by CPU 104 to process data provided by, for example, database 119 and / or multimodal medical scanner 130. Thus, computer system 101 is a general-purpose computer system that becomes a dedicated computer system when executing the computer-readable program code. The computer-readable program code is not intended to be limited to any particular programming language or its implementation. It will be appreciated that various programming languages ​​and their encodings can be used to implement the teachings of the disclosure contained herein. The same or different computer-readable media 106 may be used to store the database 119, including but not limited to image datasets, knowledge bases, individual subject data, medical records, subject diagnostic reports (or documents), or combinations thereof.

[0025] A multimodal medical scanner 130 acquires image data 132. This image data 132 can be processed and stored in a database 119. The multimodal medical scanner 130 can be a radiological scanner (e.g., a nuclear medicine scanner) and / or suitable peripheral devices (e.g., a keyboard and display device) for acquiring, collecting, and / or storing this image data 132. The multimodal medical scanner 130 can be a hybrid modality designed to acquire image data using at least one anatomical imaging modality (e.g., CT, MR) and at least one molecular imaging modality (e.g., SPECT, PET). The anatomical imaging modality focuses on extracting structural information, while the molecular imaging modality focuses on extracting functional information from molecules of interest. The multimodal medical scanner 130 can be, for example, a PET / CT, SPECT / CT, or PET / MR scanner.

[0026] Workstation 134 may include a computer and suitable peripherals, such as a keyboard and display device, and may be operated in conjunction with the entire system 100. For example, workstation 134 may communicate with multimodal medical scanner 130, such that medical image data 132 from multimodal medical scanner 130 may be presented or displayed at workstation 134. Workstation 134 may communicate directly with computer system 101 to display processed data and / or output results 144. Workstation 134 may include a graphical user interface to receive user input via input devices (e.g., keyboard, mouse, touchscreen, voice or video recognition interface, etc.) to manipulate the visualization and / or processing of data.

[0027] It should be further understood that, because some of the system components and method steps depicted in the accompanying drawings can be implemented in software, the actual connections between system components (or process steps) may vary depending on how this framework is programmed. Given the teachings provided herein, those skilled in the art will be able to consider these and similar implementations or configurations of this framework.

[0028] Figure 2 An exemplary method 200 for image processing is illustrated. It should be understood that the steps of method 200 may be performed in the order shown or a different order. Additional, different, or fewer steps may also be provided. Furthermore, method 200 may utilize… Figure 1 The system can be implemented through 100 different systems or combinations thereof.

[0029] At position 202, the multimodal medical scanner 130 acquires a true anatomical image of the region of interest. More specifically, the anatomical imaging modality of the multimodal medical scanner 130 can directly acquire a single true anatomical image of the region of interest of the subject or patient. The true anatomical image can be, for example, a true CT or MR image. Other types of anatomical imaging modalities are also useful.

[0030] At position 204, the multimodal medical scanner 130 acquires an emission image of the region of interest. More specifically, the molecular imaging modality of the multimodal medical scanner 130 can directly acquire an emission image of the region of interest. The emission image is non-attenuated corrected (NAC). To generate the emission image, the molecular imaging modality can detect the emission generated by a radioactive isotope injected into the host bloodstream. In some implementations, the emission image is a PET or SPECT image. Other types of molecular imaging modalities are also useful.

[0031] At position 206, image processing module 117 generates a synthetic (or pseudo) anatomical image of the region of interest based on the emitted image. Artificial intelligence (or machine learning) techniques can be used to generate the synthetic anatomical image. For example, deep learning algorithms based on generative adversarial networks (GANs), U-net, or other image-to-image networks can be implemented. Deep learning networks can be trained to generate synthetic anatomical images from NAC emitted images. Other types of artificial intelligence techniques are also useful.

[0032] Figure 3a Exemplary front and rear views 302a-b of NAC PET images are shown, and exemplary front and rear views 304a-b of these NAC PET images are used to derive synthetic CT images. Figure 3b An exemplary comparison of a real (or authentic) CT image and a synthetic CT image is shown. More specifically, histogram 312 of the real CT image, along with front and rear views 314a-b, is shown side by side with histogram 315 of the synthetic CT image, along with front and rear views 316a-b.

[0033] return Figure 2 At 208, image processing module 117 performs deformable (or non-rigid) registration from the real anatomical image to the synthetic anatomical image to generate a registered anatomical image for improved attenuation correction. Deformable registration can be performed by finding a transformation that maps or aligns the source image to the target image by solving for an objective function (e.g., a similarity metric). In other implementations, deformable registration is performed using deep learning algorithms (e.g., neural networks) or other machine learning algorithms. In some implementations, a single neural network is used to implement the cross-modal registration process (i.e., steps 206 and 208). Deformable registration is performed to register or spatially align the real anatomical image (source image) with the synthetic anatomical image (target image) to generate a registered real anatomical image.

[0034] Figure 4a Front and rear views 402a-b of exemplary images obtained by overlaying NAC PET images onto real CT images are shown. Spatial mismatch can be observed in region 404. Figure 4b Front and rear views 412a-b of exemplary resulting images obtained by overlaying NAC PET images onto registered real CT images are shown. No spatial mismatch was observed in the resulting images.

[0035] Figure 5a The front and rear views 502a-b of exemplary quality control images obtained by overlaying a real CT image onto a reversed synthetic CT image are shown. Spatial mismatch can be observed in regions 504a-b. Figure 5bFront and rear views 512a-b of exemplary resulting images obtained by overlaying a registered real CT image onto a reversed synthetic CT image are shown. No spatial mismatch was observed in the resulting images.

[0036] return Figure 2 At 210, image processing module 117 performs attenuation correction on the emission image using the registered anatomical image to generate an attenuation-corrected emission image. More specifically, based on attenuation coefficient data obtained using the registered anatomical image (e.g., CT image), a correction factor can be determined and used to correct the emission image (e.g., SPECT, PET) for attenuation, thereby producing an attenuation-corrected emission image.

[0037] In some cases, it may be advantageous to move the emission image into the space of the real anatomical image. For example, if the real anatomical image is in a clinically more desirable spatial location or anatomical pose (e.g., deep inspiration breath-hold CT is a radiological convention for lung imaging, but the emission image may be obtained in a different breathing position), then moving the emission image into the anatomical space of the real anatomical image is desirable. Post-processing steps may optionally be performed after the registration of the real to synthetic anatomical image and subsequent attenuation correction of the emission image. This post-processing includes warping the attenuated-corrected emission image into the desired anatomical space of the real anatomical image using the inverse deformation field from the real-to-synthetic anatomical image registration in step 208. It should be understood that attenuation correction of the emission image may also be performed after warping the unattenuated emission image into the desired anatomical space.

[0038] At 212, the selection of the next emission image is determined. If no next emission image is selected, method 200 proceeds to 214. If a next emission image is selected, method 200 returns to 204. Steps 204 to 212 can be repeated multiple times over time (e.g., 1, 2...N) to generate a set of attenuation-corrected emission images using the single real anatomical image acquired in step 202. Advantageously, acquiring a single real anatomical image (rather than multiple anatomical images) reduces scan time and / or radiation dose, thereby making the scanning process more efficient and less risky due to the cumulative dose from, for example, repeated CT image acquisition.

[0039] According to this framework, multiple emission images can be acquired at different time points in various scenarios. For example, when multiple radiotracers are used in a single scan, different emission images can be acquired after different radiotracers are injected into the subject. As another example, a sequence of multiple emission images can be acquired over time within a single scan to perform temporal gating (e.g., for different respiratory or cardiac locations). Matched CT images can be created for each gating. Different spatial anatomical representations can be provided. As yet another example, multiple emission images can be acquired in a time series of scans following the injection of a long-lived radioisotope used in therapy into the subject. Examples of such therapies include alpha or beta emitter radionuclide therapy or Zr-based immunotherapy. As yet another example, multiple emission images can be acquired to monitor therapy response, particularly in cases with rapid responses.

[0040] At position 214, image processing module 117 outputs a sequence of one or more attenuation-corrected emission images. Spatial mismatches between functional and anatomical images have been removed, enabling more accurate and precise attenuation correction. One or more attenuation-corrected emission images can be displayed, for example, at workstation 134.

[0041] While the framework has been described in detail with reference to exemplary embodiments, those skilled in the art will appreciate that various modifications and substitutions can be made to the framework without departing from the spirit and scope of the invention as set forth in the appended claims. For example, within the scope of this disclosure and the appended claims, elements and / or features of different exemplary embodiments may be combined with and / or substituted for each other.

Claims

1. A system for multi-scan image processing, comprising: Non-transitory memory devices used to store computer-readable program code; as well as A processor device that communicates with the memory device, the processor device being operable using the computer-readable program code to perform steps, the steps including: (i) Acquire a single real anatomical image of the region of interest, wherein the real anatomical image includes a real computed tomography (CT) or magnetic resonance (MR) image. (ii) Acquire emission images of the region of interest, wherein the emission images include positron emission tomography (PET) or single-photon emission computed tomography (SPECT) images. (iii) Generate a synthetic anatomical image based on the emission image. (iv) Perform deformable registration of the real anatomical image to the synthetic anatomical image to generate a registered anatomical image, and (v) Attenuation correction is performed on the emission image using the registered anatomical image to generate an attenuated emission image. Steps (ii), (iii), (iv), and (v) are repeated to generate a set of attenuation-corrected emission images based on the single real anatomical image.

2. The system of claim 1, wherein the processor device is operable using the computer-readable program code to generate the synthetic anatomical image using a single neural network and to perform deformable registration of the real anatomical image to the synthetic anatomical image.

3. The system of claim 1, wherein the processor device is operable using the computer-readable program code to distort an attenuated emission image into the anatomical space of the real anatomical image by using an inverse deformation field of deformable registration from the real anatomical image to the synthetic anatomical image.

4. The system of claim 1, wherein the processor device is operable using the computer-readable program code to repeat step (ii) to acquire different emission images after different radiotracers are injected into the body.

5. The system of claim 1, wherein the processor device is operable using the computer-readable program code to repeat step (ii) to acquire a sequence of multiple emission images over time within a single scan phase to perform time gating.

6. The system of claim 1, wherein the processor device is operable using the computer-readable program code to repeat step (ii) to acquire multiple emission images in a time series following the injection of a radioisotope used in the therapy into the body.

7. A method for multi-scan image processing, comprising: (i) Acquire a single real anatomical image of the region of interest, wherein the real anatomical image includes a real computed tomography (CT) or magnetic resonance (MR) image; (ii) Acquire one or more emission images of the region of interest, wherein the emission images include positron emission tomography (PET) or single-photon emission computed tomography (SPECT) images; (iii) Generate one or more synthetic anatomical images based on the one or more emission images; (iv) Perform one or more deformable registrations of the real anatomical images to the one or more synthetic anatomical images to generate one or more registered anatomical images; as well as (v) Attenuation correction is performed on the one or more emission images using the one or more registered anatomical images to generate one or more attenuation-corrected emission images.

8. The method of claim 7, wherein acquiring the one or more emission images of the region of interest comprises: Different emission images were acquired after different radioactive tracers were injected into the subject.

9. The method of claim 7, wherein acquiring the one or more emission images of the region of interest comprises: A sequence of multiple emission images is acquired over time within a single scan cycle to perform time gating.

10. The method of claim 7, wherein acquiring the one or more emission images of the region of interest comprises: Multiple emission images were acquired in a time-series scanning sequence following the injection of the radioisotope used in the therapy into the body.

11. The method of claim 7, wherein generating the one or more synthetic anatomical images based on the one or more emission images comprises applying a deep learning algorithm.

12. The method of claim 7, further comprising: Using an inverse deformation field from the one or more deformable registrations, the one or more attenuation-corrected emission images are distorted into the anatomical space of the real anatomical image.

13. The method of claim 7, further comprising: Before performing the attenuation correction, the one or more emission images are distorted into the anatomical space of the real anatomical image using an inverse deformation field from the one or more deformable registrations.

14. The method of claim 7, wherein generating the one or more synthetic anatomical images and performing the one or more deformable registrations are performed using a single neural network.

15. One or more non-transitory computer-readable media embodying machine-executable instructions for performing image processing operations, said operations including: (i) Acquire a single real anatomical image of the region of interest, wherein the real anatomical image includes a real computed tomography (CT) or magnetic resonance (MR) image; (ii) Acquire emission images of the region of interest, wherein the emission images include positron emission tomography (PET) or single-photon emission computed tomography (SPECT) images; (iii) Generate a synthetic anatomical image based on the emission image; (iv) Perform deformable registration of the real anatomical image to the synthetic anatomical image to generate a registered anatomical image; as well as (v) Attenuation correction is performed on the emission image using the registered anatomical image to generate an attenuated emission image. Steps (ii), (iii), (iv), and (v) are repeated to generate a set of attenuation-corrected emission images based on the single real anatomical image.

Images