Methods and systems for integrating pet data with CT data for enhanced medical imaging analysis

Abstract

Preoperative and intraoperative surgical systems and methods provide tools for analyzing and visualizing active metabolic areas in CT image data or 3D anatomical models generated from the CT image data. Some methods involve receiving computed tomography (CT) image data, displaying the CT image data, receiving positron emission tomography (PET) image data, identifying at least one active metabolic area in the PET image data, and integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data. Other methods may include segmenting anatomical structures from CT image data, generating a 3D anatomic model based on segmented anatomical structures, displaying the 3D anatomic model, and integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomic model.

Description

METHODS AND SYSTEMS FOR INTEGRATING PET DATA WITH CT DATA FOR ENHANCED MEDICAL IMAGING ANALYSISFIELD

[0001] The technology of the disclosure is generally related to methods and systems for enhancing medical imaging analysis by presenting Positron Emission Tomography (PET) data integrated with CT image data.BACKGROUND

[0002] Thoracic surgeries, including those surgeries involving resection procedures, often rely on precise planning and visualization of the surgical area to ensure optimal patient outcomes. The current standard of care for preoperative visualization of resection areas in thoracic surgeries may be limited in providing comprehensive and real-time insights for surgeons. Current methods may have limitations in providing a comprehensive and seamless integration of metabolic activity with anatomical details. Thus, there is a need for advanced tools to integrate PET data with CT images for enhanced medical imaging analysis.SUMMARY

[0003] The techniques of this disclosure generally relate to advanced tools to integrate PET data with CT images, which provide a comprehensive and seamless integration of metabolic activity with anatomical details, ultimately improving accuracy in diagnosis, treatment planning, and patient monitoring. The methods and systems of the disclosure leverage advanced image fusion techniques to seamlessly integrate PET data with anatomical information from CT image, thereby providing a comprehensive visualization of metabolic activity overlaid onto structural details. This integrated approach offers medical professionals an enhanced understanding of the correlation between anatomical and functional data, enabling more accurate diagnosis, treatment planning, and monitoring of various medical conditions.

[0004] In one aspect, this disclosure provides a method. The method includes receiving computed tomography (CT) image data and displaying the CT image data. The method also includes receiving positron emission tomography (PET) image data, identifying at least one active metabolic area in the PET image data, and integrating thePET image data corresponding to the identified at least one active metabolic area with the displayed CT image data.

[0005] Implementations of the method may include one or more of the following features. The CT image data may be a CT volume.

[0006] Identifying at least one active metabolic area in the PET image data may include determining radiotracer concentrations from pixel values in the PET image data and determining the at least one active metabolic area based on the determined radiotracer concentrations.

[0007] Identifying at least one active metabolic area in the PET image data includes analyzing at least one standard uptake value (SUV) tag within Digital Imaging and Communications in Medicine (DICOM) data of the PET image data. The at least one SUV tag may include at least one of pixel values, rescale slope, rescale intercept, units, decay factor, amount of radiotracer injected into the patient, a patient’s weight, start time of radiotracer administration, PET scan start time, decay correction, a patient’s height, or radionuclide half-life.

[0008] The CT image data may be cone-beam computed tomography (CBCT) image data.

[0009] Integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data may include overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed CT image data.

[0010] Integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data may be performed preoperatively.

[0011] The method may include segmenting anatomical structures from CT image data, generating a 3D model based on segmented anatomical structures, displaying the 3D model, and overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed 3D model. The overlaying may be performed intraoperatively.

[0012] In another aspect, this disclosure provides another method. The other method includes receiving computed tomography (CT) image data, segmenting anatomical structures from CT image data, generating a 3D anatomical model based on segmented anatomical structures, and displaying the 3D anatomical model. The other method alsoincludes receiving positron emission tomography (PET) image data, identifying at least one active metabolic area in the PET image data, and integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model.

[0013] Implementations of the other method may include one or more of the following features. Integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D model may include overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed 3D anatomical model.

[0014] The overlaying may be performed intraoperatively.

[0015] Integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model may include generating a 3D metabolic area model of the identified at least one active metabolic area based on the PET image data and integrating the 3D metabolic area model into the displayed 3D anatomical model.

[0016] The method may include determining a position of the at least one active metabolic area in the 3D anatomical model and integrating the PET image data corresponding to the identified at least one active metabolic area at the determined position in the displayed 3D anatomical model.

[0017] In another aspect, this disclosure provides a system including a display, a processor, and a memory. The memory has stored thereon instructions which, when executed by the processor, cause the processor to receive computed tomography (CT) image data and cause the display to display the CT image data. The instructions also cause the processor to receive positron emission tomography (PET) image data, identify at least one active metabolic area in the PET image data, and integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data.

[0018] Implementations of the system may include one or more of the following features. The instructions may cause the processor to identify at least one active metabolic area in the PET image data by determining radiotracer concentrations from pixel values in the PET image data and determining the at least one active metabolic area based on the determined radiotracer concentrations.

[0019] The instructions may cause the processor to identify at least one active metabolic area in the PET image data by analyzing at least one standard uptake value (SUV) tag within DICOM data of the PET image data.

[0020] The instructions may cause the processor to integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data by overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed CT image data.

[0021] In another aspect, this disclosure provides another system including a display, a processor, and a memory. The memory has stored thereon instructions which, when executed by the processor, cause the processor to receive computed tomography (CT) image data, segment anatomical structures from CT image data, generate a 3D anatomical model based on segmented anatomical structures, and cause the display to display the 3D anatomical model. The instructions also cause the processor to receive positron emission tomography (PET) image data, identify at least one active metabolic area in the PET image data, and integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model.

[0022] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims.BRIEF DESCRIPTION OF DRAWINGS

[0023] FIG. l is a block diagram that illustrates a surgical environment for performing thoracic surgery.

[0024] FIG. 2 is a block diagram that illustrates a computer system for use with the surgical environment of FIG. 1.

[0025] FIG. 3 is a flow diagram that illustrates a method of integrating PET image data into CT image data.

[0026] FIG. 4 is a graphical diagram that illustrates a surgical application user interface displaying a CT comparison screen in which PET image data is overlaid on a CT image.

[0027] FIG. 5 is a flow diagram that illustrates a method of integrating PET image data into a 3D anatomical model based on CT image data.

[0028] FIG. 6 is a graphical diagram that illustrates a surgical application user interface displaying a main anatomy screen showing a lung model on which PET image data is overlaid.

[0029] FIG. 7 is a graphical diagram that illustrates a surgical application user interface of FIG. 6 displaying a lobe layer removed from the lung model.

[0030] FIG. 8 is a graphical diagram that illustrates another surgical application user interface displaying a main anatomy screen showing a lung model on which PET image data is overlaid.DETAILED DESCRIPTION

[0031] The methods and systems of this disclosure enhance medical imaging analysis by presenting Positron Emission Tomography (PET) data integrated into CT image data preoperatively and intraoperatively. The methods and systems of this disclosure involve advanced image processing techniques. The techniques include data input, in which medical professionals upload PET and CT image data into a system. The system loads PET scan data and identifies the active metabolic areas. The active metabolic areas may be identified by analyzing the SUV tag within the Digital Imaging and Communications in Medicine (DICOM) data as described in greater detail hereinbelow. The system may utilize advanced image fusion algorithms to seamlessly integrate previously analyzed PET image data with the anatomical information from given CT image data of the same patient.

[0032] In aspects, the methods and systems of this disclosure provide preoperative and / or intraoperative overlays. For example, the system may overlay the analyzed PET data onto the volume generated from CT scans, both preoperatively and intraoperatively, providing comprehensive visualization. Preoperatively, the physician can plan the surgery based on the overlayed PET data over the volume generated from the CT scan. This method enables the visualization and understanding of the metabolic and anatomical data before the surgery. Intraoperatively, the physician can analyze the overlayed PET data over the virtual anatomical structures visualized to the physician during the surgery and understand the metabolic and anatomical data during the surgery.

[0033] The methods and systems of this disclosure also provide comprehensive visualization and enhanced understanding. Medical professionals and / or a suitable software application may analyze the integrated images to gain insights into the correlation between anatomical and functional data, leading to more accurate diagnosis, treatment planning, and monitoring of medical conditions. The system offers medical professionals a comprehensive visualization and enhanced understanding of metabolic activity overlaid onto structural details, enhancing their understanding of the correlation between anatomical and functional data. The methods and systems also provide user- friendly interface. The graphical interface is intuitive and user-friendly for easy navigation and interpretation of the integrated PET-CT image data.

[0034] The system also improves medical imaging analysis by providing medical professionals with a powerful tool for enhanced visualization and understanding of both metabolic and anatomical data.

[0035] As illustrated in FIG. 1, the methods described hereinbelow utilize a system 100 including a navigation system capable of guiding a surgical tool 80 within the thoracic cavity and the patient’s (P) lungs (L) to a region of interest (ROI), which may include one or more lesions. The navigation system may be integrated with a robotic or laparoscopic system. The navigation system includes a tracking system 110 that is configured for use with the surgical tool 80 and enables monitoring of the position and orientation of a distal portion of the surgical tool 80. The system 100 further includes a computer system 10 and a user interface 20 displayed on a display associated with the computer system 10 or suitable monitoring equipment 30 (e.g., a video display). The role and use of the system 100 with the methods of this disclosure are described herein.

[0036] Reference is now made to FIG. 2, which is a block diagram of the computer system 10 of FIG. 1 configured for implementing the methods of the disclosure including the methods of FIGS. 3 and 5. The computer system 10 may include a workstation. In aspects, the computer system 10 may be coupled with an imaging system (e.g., a PET, CT, CBCT, fluoroscopic imaging system or other suitable radiographic imaging system), directly or indirectly, e.g., by wireless communication.

[0037] The computer system 10 may include a memory 202, a processor 204, a display 206, and an input device 210. The processor 204 may include one or more hardware processors. The computer system 10 may optionally include an output module 212 and anetwork interface 208. The memory 202 may store radiographic image data 214 and a surgical application 218 for planning and executing a surgery, e.g., a thoracic surgery. The surgical application 218 may include instructions executable by the processor 204 for executing the methods of the disclosure including the methods of FIGS. 3 and 5.

[0038] The surgical application 218 may further include a user interface 216. The image data 214 may include preoperative CT image data or preoperative CBCT image data and PET image data. The processor 204 may be coupled with the memory 202, the display 206, the input device 210, the output module 212, the network interface 208, and the imaging system. The computer system 10 may be a stationary computer system, such as a personal computer, or a portable computer system such as a tablet computer. The computer system 10 may be implemented by multiple computers.

[0039] The memory 202 may include any non-transitory computer-readable storage media for storing data and / or software including instructions that are executable by the processor 204 and which, among other things, control the operation of the computer system 10, process data from a laparoscopic or robotic tool, process radiographic imaging data, and display a 3D anatomical model and tools for interacting with the 3D anatomical model to perform surgical planning or perform a surgical procedure. The imaging system may be used to capture a series of preoperative CT images of a portion of a patient’s body, e.g., the lungs, as the portion of the patient’s body moves, e.g., as the lungs move during a respiratory cycle.

[0040] Optionally, the imaging system may include a CBCT imaging system that captures a series of images based on which a 3D reconstruction is generated. In aspects, the memory 202 may include one or more storage devices such as solid-state storage devices, e.g., flash memory chips. Alternatively, or in addition to the one or more solid- state storage devices, the memory 202 may include one or more mass storage devices connected to the processor 204 through a mass storage controller (not shown) and a communications bus (not shown).

[0041] Although the description of computer-readable media contained herein refers to solid-state storage, it should be appreciated by those skilled in the art that computer- readable storage media can be any available media that can be accessed by the processor 204. That is, computer readable storage media may include non-transitory, volatile and non-volatile, removable and non-removable media implemented in any method ortechnology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media may include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state memory technology, CD-ROM, DVD, Blu-Ray or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which may be used to store the desired information, and which may be accessed by computer system 10.

[0042] The surgical application 218 may, when executed by the processor 204, cause the display 206 to present the user interface 216. The user interface 216 may be configured to present to the user a screen including a view of a 3D model of at least a lung and a lesion. The view may be from the perspective of a tip of a laparoscopic or robotic tool. The user interface 216 may be configured to display the critical structures in different colors to help the clinician to distinguish different critical structures. The user interface 216 may be configured to display

[0043] The network interface 208 may be configured to connect to a network such as a local area network (LAN) consisting of a wired network and / or a wireless network, a wide area network (WAN), a wireless mobile network, a Bluetooth network, and / or the Internet. The network interface 208 may be used to connect between the computer system 10 and the imaging system 515. The network interface 208 may also be used to receive the image data 214. The input device 210 may be any device by which a user may interact with the computer system 10, such as, for example, a mouse, keyboard, foot pedal, touch screen, and / or voice interface. The output module 212 may include any connectivity port or bus, such as, for example, parallel ports, serial ports, universal serial busses (USB), or any other similar connectivity port known to those skilled in the art. From the foregoing and with reference to the various figures, those skilled in the art will appreciate that certain modifications can be made to the disclosure without departing from the scope of the disclosure.

[0044] PET imaging is a type of nuclear medicine imaging using radioactive substances referred to as radiotracers to visualize and measure metabolic processes in the body. This method provides detailed images of the function of tissues and organs. According to PET imaging, a small amount of a radiotracer is injected into the patient’s bloodstream. The radiotracer may be a biologically active molecule tagged with apositron-emitting radionuclide. The radiotracer may be fluorodeoxyglucose (FDG), which accumulates in high-energy consuming cells like cancer cells.

[0045] As the radiotracer accumulates in lesions, the radionuclide undergoes radioactive decay and emits positrons. When these positrons encounter electrons in the body, they annihilate each other, resulting in the emission of gamma photons. A PET imaging system includes a ring of detectors surrounding the patient. The ring of detectors detects the gamma photons and output detection data. The detection data is used to generate detailed 3D images of the distribution of the radiotracer in the body. Specifically, the detection data is processed by a computer to generate cross-sectional images that reflect the concentration and location of the radiotracer. The cross-sectional images provide functional information about the one or more lesions.

[0046] Metabolic areas within the PET image (e.g., a DICOM scan) may be identified using the Standard Uptake Value (SUV). The SUV is a quantitative measure used in PET oncology imaging to assess the concentration of a radiotracer in a region of interest (ROI) within the body. The SUV evaluates metabolic activity in tissues, which can help diagnose and monitor conditions such as cancer, neurological disorders, and cardiovascular diseases. In PET imaging, the SUV provides a standardized way to measure and compare metabolic activity in tissues, thereby aiding in diagnosis and treatment planning. The SUV may be calculated using the following expression:Radiotracer Concentration in ROI suv = _ VUInjected dose (MBq) Patient's body weight (kg)To improve accuracy, this expression may be adjusted based on lean body mass or body surface area instead of body weight.

[0047] Higher SUVs may indicate higher metabolic activity, which can be associated with malignancies or active disease processes. Lower SUVs may indicate lower metabolic activity. The units for SUV are generally dimensionless because they are ratios, but the input values have units such as kilobecquerels per milliliter (kBq / mL) for concentration, megabecquerels (MBq) for dose, and kg for weight. There are several types of SUV calculations, which may include:SUVmax: The maximum SUV within the ROI.SUVmean: The mean SUV within the ROI.• SUVpeak: The peak SUV within a small, consistent region of the ROI.

[0048] In DICOM files, the SUVs are often stored as part of the image metadata or pixel data in PET images. The DICOM standard includes specific tags and attributes to store and describe SUV calculations, ensuring consistency and interoperability between different imaging systems and software. To calculate the SUV in DICOM, specific tags may be needed to retrieve information from the DICOM files. Some DICOM tags for SUV calculation may include the following:• Radiotracer Concentration: May be extracted from the pixel values in the PET image, which represent the activity concentration in kBq / mL. The radiotracer concentration in PET imaging is not directly stored in a specific DICOM tag but is derived from the pixel values of the PET image itself. These pixel values represent the activity concentration in kBq / mL after appropriate scaling factors are applied. Some DICOM tags related to the pixel values and scaling factors are as follows: o Pixel Values: The pixel values in the PET image represent the raw data which can be scaled to obtain the radiotracer concentration. o Rescale Slope: This tag represents the slope used to convert the stored pixel values to the actual concentration values. o Rescale Intercept: This tag represents the intercept used in the conversion equation. o Units: This tag specifies the units of the pixel values. For PET images, the units may be becquerels per milliliter (Bq / mL). o Decay Factor: This tag provides information about the decay correction applied to the PET data.The radiotracer concentration can be calculated from the pixel values using the following expression: / kBq\Radiotracer Concentration I — — 1 = Pixel Value x Rescale Slope + Rescale InterceptWhile no single tag directly represents the radiotracer concentration, the concentration can be derived from the pixel values in combination with the rescale slope and intercept tags.• Injected Dose: The amount of radiotracer injected into the patient and may include the following tags:o Radiopharmaceutical Information Sequence. o Radionuclide Total Dose: The total dose of the radionuclide in Bq or MBq.• Patient’s Body Weight: The weight of the patient, typically in kilograms. This tag may be referred to as Patient's Weight.• Start Time of Radiotracer Administration: The time when the radiotracer injection started. This tag may be referred to as Radiopharmaceutical Start Time.• Scan Start Time: The time when the PET scan started and may include the following tags: o Series Time o Acquisition Time• Decay Correction: Information about whether decay correction has been applied to the PET image.• Patient's Height: This tag may be needed for body surface area calculations. This tag may be referred to as Patient’s Size• Radionuclide Half-Life: The half-life of the radiotracer used.

[0049] An example of how the tags may be used in the SUV calculation process is as follows:• Extract pixel values from the PET image representing radiotracer concentration in kBq / mL.• Retrieve the injected dose (Radionuclide Total Dose) in MBq.• Retrieve the patient's weight (Patient’s Weight) in kilograms.• Determine the timing of the radiotracer injection (Radiopharmaceutical Start Time) and the scan (Series Time / Acquisition Time).• Check for decay correction and apply, if necessary, using the radionuclide half-life (Radionuclide Half-Life).• Calculate the SUV with the following expression:Radiotracer Concentration in ROI suv = _ Injected dose (MBq) UVPatient's body weight (kg)A software application 218 executed by the computer system 10 may process DICOM images, automatically read tags, and perform calculations to provide SUV values.

[0050] FIG. 3 illustrates an example method for integrating PET image data with CT image data. At block 302, computed tomography (CT) image data is received. Receiving CT image data may include receiving DICOM-compliant image files from various sources, such as a local imaging system or a storage medium. The received CT image data may represent a volumetric reconstruction formed from axial slices and may be pre- processed to improve contrast, reduce noise, or normalize intensity values. For example, the CT image data may include a CT volume constructed based on CT images. At block 304, the CT image data is displayed. Displaying the CT image data may include rendering 2D or 3D visualizations of the CT volume within a surgical planning user interface. The visualization may support clinician interaction through GUI tools for adjusting brightness and zoom, and for scrolling slice-by-slice or rotating volumetric views. In aspects, the CT image data may be cone-beam computed tomography (CBCT) image data or any other tomographic image data suitable for showing anatomical structures.

[0051] At block 306, positron emission tomography (PET) image data is received. Receiving PET image data may include receiving PET scans encoded in DICOM format, with relevant tags for SUV computation, decay correction, and / or dose information. The PET image data may be registered to match the coordinate space of the CT image data. At block 308, one or more active metabolic area(s) in the PET image data are identified. Identifying the active metabolic area(s) in the PET image data may include segmenting ROIs using voxel-level thresholds, pattern recognition algorithms, or learning models trained to detect metabolic activity. The system may calculate SUV metrics, such as SUVmax or SUVmean, and generate bounding volumes around detected lesions. In aspects, the active metabolic area(s) may be identified in the PET image data by determining radiotracer concentrations from pixel values in the PET image data and determining the active metabolic area(s) based on the determined radiotracer concentrations.

[0052] In one example, identifying active metabolic area(s) in the PET image data may include analyzing at least one standard uptake value (SUV) tag within DICOM data of the PET image data. As described above, the at least one SUV tag may include at least one of pixel values, rescale slope, rescale intercept, units, decay factor, amount of radiotracer injected into the patient, patient’s weight, start time of radiotraceradministration, PET scan start time, decay correction, patient’s height, or radionuclide half-life

[0053] Before ending at block 312, the PET image data corresponding to the identified active metabolic area(s) is integrated with the displayed CT image data at block 312. Integrating the PET image data corresponding to the identified active metabolic area(s) with the displayed CT image data may include overlaying the PET image data corresponding to the identified active metabolic area(s) on the displayed CT image data. Integrating the PET image data corresponding to the identified active metabolic area(s) with the displayed CT image data may include spatial registration, followed by overlay, blending, or fusing of PET metabolic signal intensities with CT grayscale data. The user interface may include transparency controls, ROI highlighting, lesion measurement overlays, and toggling between PET-only, CT-only, or fused views. The integration may allow clinicians to visually assess anatomical structures and metabolic abnormality simultaneously.

[0054] FIG. 4 illustrates an example of how PET image data 402 may be integrated with (e.g., overlaid on) a CT image 404 of a CT comparison screen 400 in a surgical application user interface. The CT comparison screen 400 may include one or more graphical controls that allow a user to manipulate the displayed images. For example, the user interface may include a graphical control enabling the user to adjust the transparency of the PET image data 402 to reveal underlying anatomical structures in the CT image 404. As shown in FIG. 4, the user interface may display lesion measurements, including lesion size and a maximum axis dimension, as shown in association with the overlaid PET image data 402. The CT comparison screen 400 may include graphical controls to toggle between different imaging views (e.g., PET-only, CT-only, or fused view), adjust image brightness and contrast, and / or select or annotate ROIs for closer inspection. The user interface tools may facilitate preoperative planning by enabling the clinician to visualize metabolic activity within anatomical structures, assess lesion boundaries, and plan for biopsy and / or treatment based on the integrated imaging data.

[0055] In aspects, the PET image data 402 may be fused with a 3D anatomical model constructed using CT images. In these aspects, the method 300 of FIG. 3 may include segmenting anatomical structures from CT image data, generating a 3D anatomical model based on segmented anatomical structures, displaying the 3D anatomical model, andoverlaying the PET image data 402 corresponding to the identified active metabolic area(s) on the displayed 3D anatomical model. The overlaying may be performed intraoperatively.

[0056] FIG. 5 illustrates an example method of integrating PET image data with a 3D anatomical model based on CT image data. At block 502, computed tomography (CT) image data is received. Receiving CT image data may involve accessing DICOM files representing a volumetric scan of a body region, e.g., the thoracic region. Preprocessing may include denoising, spatial resampling, and reducing artifacts in the CT image data. The CT image data may be imported automatically from a connected CT scanner or selected from prior imaging studies stored in a storage medium.

[0057] At block 504, anatomical structures are segmented from CT image data. The segmentation may use deep-learning algorithms or models to segment anatomical structures, which may include lesions, airways, blood vessels (arteries and / or veins), pleura, lobes, and / or segments. The deep-learning models may include neural networks such as convolutional neural networks trained on imaging data sets. For example, the deep-learning model may train on appropriate radiographic imaging data sets, e.g., CT data sets, which may be annotated by suitable clinicians, e.g., medical students in the final stages of their education. The deep-learning model may then be tested on dedicated test data sets, and model performance may be compared to human medical professionals. The segmentation by the deep-learning model may occur in real-time as a clinician, e.g., a physician, is loading the radiographic imaging data into a system application.

[0058] At block 506, a 3D anatomical model is generated based on the segmented anatomical structures. Generating the 3D anatomical model may include constructing mesh-based or voxel-based renderings of the segmented regions. Segmented anatomical structures may be assigned labels and / or colors for clarity. At block 508, the 3D anatomical model is displayed as illustrated in FIG. 6. Displaying the 3D anatomical model may include a rendering window within a surgical planning user interface. The surgical planning user interface may include graphical controls for adjusting view angle, hiding or highlighting anatomical structures, changing opacity or color, and setting resection boundaries.

[0059] For example, the user interface shown in FIG. 6 may include graphical controls that enable a clinician to manipulate the 3D anatomical model, including adjustingmargins, selecting or deselecting anatomical structures such as airways, arteries, veins, lobes, and segments, and customizing the view by adding or removing specific layers. The user interface may also present lesion size measurements, labels, and guidance tools for planning a surgical procedure (e.g., a resection procedure) or marking or annotating regions of interest (ROIs).

[0060] At block 510, positron emission tomography (PET) image data is received. The PET image data may include metadata for SUV normalization and timestamps for decay correction. The PET image data may be pre-registered to the CT coordinate space or registered upon receiving the PET image data. At block 512, one or more active metabolic area(s) are identified in the PET image data. Identifying the active metabolic area(s) may involve applying a thresholding algorithm, a trained deep-learning model, and / or lesion detection. Identified regions may be represented as 3D volumes and tagged with SUV values.

[0061] Then, at block 514, the PET image data corresponding to the identified active metabolic area(s) is integrated with the displayed 3D anatomical model. Integrating the PET image data with the 3D anatomical model may include projecting or mapping active metabolic areas onto the 3D anatomical model. This may include generating a 3D metabolic sub-model from the PET image data, aligning the 3D metabolic sub-model to the 3D anatomical model, and rendering the 3D metabolic sub-model with colors, textures, and / or transparency.

[0062] In aspects, the image data integration may be implemented by overlaying the PET image data corresponding to the identified active metabolic area(s) on the displayed 3D anatomical model. This is illustrated, for example, by FIG. 8, which shows a PET image 802 overlaid on the displayed 3D anatomical model 804. The user interface of FIG.8 may include anatomical segmentation and labeling controls, along with visualization tools, e.g., tools for adjusting transparency or visibility of layers. In aspects, the transparency of the PET image 802 may be adjusted to enable a clinician to view both the PET image 802 and the portion of the 3D anatomical model 804 beneath the PET image 802. Also, in aspects, the overlaying may be performed intraoperatively.

[0063] As illustrated in FIGS. 6-8, the user interface may include a graphical layer control for toggling anatomical structures, such as the “Lobe” layer control, which is shown as selected. When the lobe layer control or the segment layer control is selected, thesystem may render specific lobes or segments in different colors or in partially transparent views or remove them entirely to expose underlying anatomical structures. The segmented anatomical structures, which may include airways, arteries, and veins, may be depicted in color (e.g., color-coded) to enhance contrast and visual differentiation.

[0064] For example, one or more lesions may be shown in one color (e.g., green), airways may be shown in another color (e.g., light pink), arteries in another color (e.g., blue), and veins in yet another color (e.g., red), enabling the clinician to distinguish between different structures (e.g., critical structures). One or more of the colors may be applied to the anatomical model, the preoperative images (e.g., CT images), and / or the PET image data. Also, in the Views & Overlays panel, the circle next to each of the listed structures may be shown in the corresponding color, thereby providing a visual legend that enables the clinician to easily identify structures in the anatomical model, the preoperative images (e.g., CT images), and / or the PET image data. These visualization features may assist the clinician in identifying the spatial relationships between active metabolic areas, lesion locations, and anatomical structures and / or landmarks during surgical planning or execution.

[0065] In aspects, integrating the PET image data corresponding to the identified active metabolic area(s) with the displayed 3D anatomical model may include generating a metabolic area model, e.g., a 2D or 3D metabolic area model, of the identified active metabolic area(s) based on the PET image data and integrating the metabolic area model into the displayed 3D anatomical model. An example of this is illustrated in FIGS. 6 and 7, in which a metabolic area model 602 based on the PET image data 402 is overlaid on the displayed 3D anatomical model 604. As shown in FIG. 6, the user interface may present the metabolic area model 602 in spatial relation to segmented anatomical structures of the lung, including arteries, veins, and airways, each of which may be rendered in different colors and / or labeled with corresponding identifiers. The user may interact with graphical tools to adjust the anatomical layers, including selectively hiding or displaying specific lobes or segments.

[0066] FIG. 7 illustrates an example of the 3D anatomical model 604 in which a lobe layer has been removed to expose anatomical structures beneath the overlaid metabolic area model 602. This layered visualization approach enables the clinician to assess the metabolic area model 602 in its spatial context, including determining whether a lesion isnear vasculature, located within a particular segment or lobe, or adjacent to other anatomical structures. In aspects, the method 500 may include determining a position of the active metabolic area(s) in the 3D anatomical model 604 and integrating the PET image data 402 corresponding to the identified active metabolic area(s) at the determined position in the displayed 3D anatomical model 604.

[0067] Various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the techniques). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a medical device.

[0068] In one or more examples, the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).

[0069] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

[0070] The invention may be further described by reference to the following numbered paragraphs:1. A method comprising: receiving computed tomography (CT) image data; displaying the CT image data; receiving positron emission tomography (PET) image data; identifying at least one active metabolic area in the PET image data; and integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data.2. The method according to paragraph 1, wherein the CT image data is a CT volume.3. The method according to any of the preceding paragraphs, wherein identifying at least one active metabolic area in the PET image data includes: determining radiotracer concentrations from pixel values in the PET image data; and determining the at least one active metabolic area based on the determined radiotracer concentrations.4. The method according to any of the preceding paragraphs, wherein identifying at least one active metabolic area in the PET image data includes analyzing at least one standard uptake value (SUV) tag within Digital Imaging and Communications in Medicine (DICOM) data of the PET image data.5. The method according to paragraph 4, wherein the at least one SUV tag includes at least one of pixel values, rescale slope, rescale intercept, units, decay factor, amount of radiotracer injected into the patient, patient’s weight, start time of radiotracer administration, PET scan start time, decay correction, patient’s height, or radionuclide half-life.6. The method according to any of the preceding paragraphs, wherein the CT image data is cone-beam computed tomography (CBCT) image data.7. The method according to any of the preceding paragraphs, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data includes overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed CT image data.8. The method according to any of the preceding paragraphs, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data is performed preoperatively.9. The method according to any of the preceding paragraphs, further comprising: segmenting anatomical structures from CT image data; generating a 3D model based on segmented anatomical structures; displaying the 3D model; and overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed 3D model.10. The method according to paragraph 9, wherein the overlaying is performed intraoperatively.11. A method comprising: receiving computed tomography (CT) image data; segmenting anatomical structures from CT image data; generating a 3D anatomical model based on segmented anatomical structures; displaying the 3D anatomical model; receiving positron emission tomography (PET) image data; identifying at least one active metabolic area in the PET image data; and integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model.12. The method according to paragraph 11, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3Dmodel includes overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed 3D anatomical model.13. The method according to paragraph 12, wherein the overlaying is performed intraoperatively.14. The method according to any of the preceding paragraphs, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model includes: generating a 3D metabolic area model of the identified at least one active metabolic area based on the PET image data; and integrating the 3D metabolic area model into the displayed 3D anatomical model.15. The method according to any of the preceding paragraphs, further comprising: determining a position of the at least one active metabolic area in the 3D anatomical model; and integrating the PET image data corresponding to the identified at least one active metabolic area at the determined position in the displayed 3D anatomical model.16. A system comprising: a display; a processor; and a memory having stored thereon instructions which, when executed by the processor, cause the processor to: receive computed tomography (CT) image data; cause the display to display the CT image data; receive positron emission tomography (PET) image data; identify at least one active metabolic area in the PET image data; and integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data.17. The system according to paragraph 16, wherein the instructions further cause the processor to identify at least one active metabolic area in the PET image data by determining radiotracer concentrations from pixel values in the PET image data and determining the at least one active metabolic area based on the determined radiotracer concentrations.18. The system according to any of the preceding paragraphs, wherein the instructions further cause the processor to identify at least one active metabolic area in the PET image data by analyzing at least one standard uptake value (SUV) tag within Digital Imaging and Communications in Medicine (DICOM) data of the PET image data.19. The system according to any of the preceding paragraphs, wherein the instructions further cause the processor to integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data by overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed CT image data.20. A system comprising: a display; a processor; and a memory having stored thereon instructions which, when executed by the processor, cause the processor to: receive computed tomography (CT) image data; segment anatomical structures from CT image data; generate a 3D anatomical model based on segmented anatomical structures; cause the display to display the 3D anatomical model; receive positron emission tomography (PET) image data; identify at least one active metabolic area in the PET image data; and integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model.

Claims

WHAT IS CLAIMED IS:

1. A method comprising: receiving computed tomography (CT) image data; displaying the CT image data; receiving positron emission tomography (PET) image data; identifying at least one active metabolic area in the PET image data; and integrating, with the displayed CT image data, the PET image data corresponding to the identified at least one active metabolic area.

2. The method according to claim 1, wherein identifying the at least one active metabolic area in the PET image data includes: determining radiotracer concentrations from pixel values in the PET image data; and determining the at least one active metabolic area based on the determined radiotracer concentrations.

3. The method according to any of the preceding claims, wherein identifying the at least one active metabolic area in the PET image data includes analyzing at least one standard uptake value (SUV) tag within Digital Imaging and Communications in Medicine (DICOM) data of the PET image data, and wherein the at least one SUV tag includes at least one of pixel values, rescale slope, rescale intercept, units, decay factor, amount of radiotracer injected into a patient, patient’s weight, start time of radiotracer administration, PET scan start time, decay correction, patient’s height, or radionuclide half-life.

4. The method according to any of the preceding claims, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data includes overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed CT image data.

5. The method according to any of the preceding claims, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed CT image data is performed preoperatively.

6. The method according to any of the preceding claims, further comprising: segmenting anatomical structures from CT image data; generating a 3D model based on segmented anatomical structures; displaying the 3D model; and overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed 3D model.

7. A method comprising: receiving computed tomography (CT) image data; segmenting anatomical structures from CT image data; generating a 3D anatomical model based on segmented anatomical structures; displaying the 3D anatomical model; receiving positron emission tomography (PET) image data; identifying at least one active metabolic area in the PET image data; and integrating, with the displayed 3D anatomical model, the PET image data corresponding to the identified at least one active metabolic area.

8. The method according to claim 7, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D model includes overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed 3D anatomical model, and wherein the overlaying is performed intraoperatively.

9. The method according to any of the preceding claims, wherein integrating the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model includes: generating a 3D metabolic area model of the identified at least one active metabolic area based on the PET image data; andintegrating the 3D metabolic area model into the displayed 3D anatomical model.

10. The method according to any of the preceding claims, further comprising: determining a position of the at least one active metabolic area in the 3D anatomical model; and integrating the PET image data corresponding to the identified at least one active metabolic area at the determined position in the displayed 3D anatomical model.

11. A system comprising: a display; a processor; and a memory having stored thereon instructions which, when executed by the processor, cause the processor to: receive computed tomography (CT) image data; cause the display to display the CT image data; receive positron emission tomography (PET) image data; identify at least one active metabolic area in the PET image data; and integrate, with the displayed CT image data, the PET image data corresponding to the identified at least one active metabolic area.

12. The system according to claim 11, wherein the instructions further cause the processor to identify the at least one active metabolic area in the PET image data by determining radiotracer concentrations from pixel values in the PET image data and determining the at least one active metabolic area based on the determined radiotracer concentrations.

13. The system according to any of the preceding claims, wherein the instructions further cause the processor to identify the at least one active metabolic area in the PET image data by analyzing at least one standard uptake value (SUV) tag within Digital Imaging and Communications in Medicine (DICOM) data of the PET image data.

14. The system according to any of the preceding claims, wherein the instructions further cause the processor to integrate the PET image data corresponding to the identified at leastone active metabolic area with the displayed CT image data by overlaying the PET image data corresponding to the identified at least one active metabolic area on the displayed CT image data.

15. A system comprising: a display; a processor; and a memory having stored thereon instructions which, when executed by the processor, cause the processor to: receive computed tomography (CT) image data; segment anatomical structures from CT image data; generate a 3D anatomical model based on segmented anatomical structures; cause the display to display the 3D anatomical model; receive positron emission tomography (PET) image data; identify at least one active metabolic area in the PET image data; and integrate the PET image data corresponding to the identified at least one active metabolic area with the displayed 3D anatomical model.

Images