Subtraction radiography for on-board imaging in radiotherapy

Abstract

Two-dimensional X-ray images with enhanced tumor contrast are generated from computed tomography (CT) image data that were previously acquired from a subject. A simulated X-ray image is generated from the CT image data, where the simulated X-ray image depicts tissues that are non-essential for a radiotherapy plan. An X-ray image of the subject is acquired with an on-board imaging system of a radiotherapy system. A subtraction X-ray image is then generated by subtracting the simulated X-ray image from the X-ray image. The subtraction X-ray image has improved visualization of tumor tissues relative to the X-ray image by way of removing the non-essential tissues from the X-ray image.

Description

Mayo 2024-390630666.01652SUBTRACTION RADIOGRAPHY FOR ON-BOARD IMAGING IN RADIOTHERAPYCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 728,655, filed on December 5, 2024, and entitled “SUBTRACTION RADIOGRAPHY FOR ON-BOARD IMAGING IN RADIOTHERAPY ’ which is herein incorporated by reference in its entirety.BACKGROUND

[0002] Radiotherapy plays an important role in the ongoing battle against cancer. More than half of cancer patients receive radiotherapy as part of their treatment. To deliver a lethal radiation dose accurately to the tumor while minimizing radiation to surrounding organs-at- risk (OARs), radiotherapy utilizes on-board imaging to set up patients such that the tumor is accurately aligned with preoptimized radiation fields during daily treatments. The ability to visualize the tumor using on-board imaging directly impacts patient setup, and thus dose delivery accuracy.

[0003] In current clinical practice, the most commonly used on-board imaging is X-ray based imaging due to its compatibility with radiation dose delivery. However, on-board X-ray imaging faces significant limitations. In most cases, visualizing tumors on two-dimensional (2D) X-ray images is not feasible due to low tumor contrast and anatomy overlapping with the tumor. Three-dimensional (3D) X-ray (e.g., cone-beam computed tomography (CT)) allows better visualization of internal anatomy, but has a higher imaging dose and is slow to acquire, making it incompatible with real-time tumor tracking.SUMMARY OF THE DISCLOSURE

[0004] It is an aspect of the present disclosure to provide a method for generating an image with enhanced tumor contrast. The method includes accessing computed tomography (CT) image data with a computer system, where the CT image data have been previously- acquired from a subject using a CT system. A simulated X-ray image is generated from the CT image data with the computer system, where the simulated X-ray image depicts non-essential tissues in the subject for a radiotherapy plan. An X-ray image of the subject is acquired with an on-board imaging system of a radiotherapy system. A subtraction X-ray image is then1QB\630666.01652\99810157.1Mayo 2024-390630666.01652 generated with the computer system by subtracting the simulated X-ray image from the X-ray image. The subtraction X-ray image has improved visualization of tumor tissues relative to the X-ray image by way of removing signal contributions the non-essential tissues from the X-ray image. The subtraction X-ray image is then outputted with the computer system.

[0005] It is another aspect of the present disclosure to provide a method for generating an image with enhanced contrast for an anatomical target to receive treatment. The method includes accessing a first medical image with a computer system, where the first medical image has been previously acquired from a subject using a first medical imaging system. A non- essential tissue image is generated from the first medical image by removing signal contributions from the anatomical target in the first medical image. The non-essential tissue image depicts tissues that are non-essential for aligning the anatomical target to receive the treatment. A second medical image of the subject is acquired with a second medical imaging system. A subtraction image is generated with the computer system by subtracting the non- essential tissue image from the second medical image. The subtraction image has improved visualization of the anatomical target relative to the second medical image by way of removing signal contributions of the non-essential tissues from the second medical image. The subtraction image can then be outputted with the computer system.

[0006] It is yet another aspect of the present disclosure to provide a method for generating an image with enhanced contrast for an anatomical target to receive treatment. The method includes accessing medical image data with a computer system, where the medical image data have been acquired from a subject using a medical imaging system. A machine learning model is also accessed with the computer system, where the machine learning model has been trained on training data to remove signal contributions from medical images that are associated with tissues that are non-essential to aligning the anatomical target with a treatment modality. The medical image data are applied or otherwise input to the machine learning model, generating an enhanced medical image as an output. The enhanced medical image has improved visualization of the anatomical target relative to the medical image data by way of removing signal contributions of non-essential tissues from the medical image data. The enhanced medical image can then be outputted with the computer system.BRIEF DESCRIPTION OF THE DRAWINGS

[0007] FIG. 1 is a flowchart of an example method for generating an X-ray image with enhanced tumor contrast and / or visualization.2QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0008] FIG. 2 illustrates a workflow of an example method for generating subtraction X-ray images to enhance tumor contrast and / or visualization for radiotherapy on-board imaging.

[0009] FIGS. 3A-3F illustrate images generated using the methods described in the present disclosure, where the images depict the ability of subtracting X-ray images to visualize tumor (yellow arrow) by subtracting an X-ray image of non-essential tissues from an X-ray image, such as a treatment day X-ray image.

[0010] FIG. 4 is a block diagram of an example system for generating X-ray images with enhanced tumor contrast and / or visualization.

[0011] FIG. 5 is a block diagram of example components that can implement the system of FIG. 4.DETAILED DESCRIPTION

[0012] Described here are systems and methods for enhancing tumor contrast and visualization in X-ray images, such as those acquired using the on-board imaging system of a radiotherapy system. In general, tumor contrast is enhanced by leveraging prior knowledge about patient anatomy. By enhancing tumor contrast and visualization, the resulting images can be advantageously used for radiotherapy treatment.

[0013] As a non-limiting example, tumor visualization can be substantially improved by removing signals from tissues that remain stable during treatment, and which are otherwise non-essential to alignment for radiotherapy. During the radiotherapy treatment planning process, planning CT data are acquired using a planning CT scan, from which a treatment plan is created. During the course of treatment, certain tissues remain relatively stable. For example, when treating lung cancer these tissues can include bones and muscles outside of the thoracic cavity7. These tissues that remain relatively stable during the treatment process and are otherwise non-essential to patient alignment for radiotherapy planning purposes can be referred to as non-essential tissues.

[0014] Information about non-essential tissues can be obtained from planning CT data. This information can then be treated as prior information that is known on treatment days. Using this prior information, simulated X-ray images of the non-essential tissues can be generated and subtracted from the X-ray images, such as X-ray images obtained on treatment days. The resulting subtraction X-ray images will have contributions from the non-essential3QB\630666.01652\99810157.1Mayo 2024-390630666.01652 tissues removed, thereby greatly enhancing tumor contrast and making the tumor directly visible on the subtraction X-ray images.

[0015] The disclosed systems and methods provide a low cost framework for enhancing tumor visualization on X-ray images within existing radiotherapy systems. Advantageously, the systems and methods enable more accurate patient setup and direct tracking of the moving tumor, providing new abilities for better and more effective treatment of challenging cancers. The disclosed systems and methods are compatible with existing radiotherapy systems and can be added at a very low cost.

[0016] Referring now to FIG. 1, a flowchart is illustrated as setting forth the steps of an example method for generating subtraction X-ray images with enhanced tumor contrast and / or visualization. As a non-limiting example, FIG. 2 illustrates an overview of an example workflow for generating subtraction X-ray images with enhanced tumor contrast and / or visualization. In FIG. 2, the left two columns illustrate an example process for simulating 2D X-ray images of non-essential tissues, while the third column illustrates an example subtraction process on treatment days. The vertical dashed line in FIG. 2 distinguishes between steps performed before treatment starts (left: simulating 2D X-ray of non-essential tissues based on the planning CT or other prior CT image data), and those performed on treatment days (right: subtraction process), which correspond to the subcomponents described below.

[0017] Referring again to FIG. 1, the method includes accessing prior CT image data of a patient with a computer system, as indicated at step 102. Accessing the prior CT image data may include retrieving such data from a memory’ or other suitable data storage device or medium. Additionally or alternatively, accessing the prior CT image data may include acquiring such data with a CT system and transferring or otherwise communicating the data to the computer system, which may be a part of the CT system. As described above, in some instances the prior CT image data may include planning CT image data acquired for radiotherapy treatment planning.

[0018] From the prior CT image data, non-essential tissue images are generated, as indicated at step 104. These non-essential tissue images can include simulated 2D X-ray images that depict tissues that are stable and otherwise non-essential to the radiotherapy planning process (e.g., non-essential for treatment target alignment). As a non-limiting example, for a patient receiving treatment for lung cancer, the non-essential tissues may include bones and muscles outside of the thoracic cavity.4QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0019] The non-essential tissue images may be simulated from the prior CT image data.As illustrated in FIG. 2, non-essential tissue CT image data is first generated from the prior CT image data. As a non-limiting example, the non-essential tissue CT image data can be generated by removing essential tissues from the prior CT image data. Essential tissues can be removed by overriding the CT Hounsfield unit (HU) of essential tissues to that of air (e.g., HU value of -1000). In these instances, the essential tissues can be identified as those tissues not deemed to be non-essential; that is, those tissues not identified as non-essential tissues can be removed from the prior CT image data. For example, essential tissues may include tumors and surrounding tissues for generating contrast. Primary and secondary' (e.g., scatter) signals can then be simulated from the non-essential tissue CT image data and can subsequently be combined to generate the simulated images that depict non-essential tissues.

[0020] Primary' signal X-ray images are generated from the non-essential tissue CT image data. The primary signal can be simulated from digitally reconstructed radiographs (DRRs) generated from the non-essential CT image data. DRRs correspond to the primary X- ray signal component. Simulating 2D X-rays of non-essential tissues may be based on the CT of non-essential tissues, obtained from the prior CT by overriding CT Hounsfield Units (HU) of essential tissues to that of air (-1000).

[0021] As a non-limiting example, a software package such as Plastimatch, which can be enhanced with spectral response matching and has a built-in library for generating DRRs from CT image data, can be used to simulate the primary X-ray signals. Plastimatch is based on Siddon ray tracing and accelerated by GPU and multicore CPU. It is very efficient, and tasks can be completed within milliseconds.

[0022] DRRs generated directly from Plastimatch, or other such algorithms or techniques, however, are not accurate enough for the purpose of generating subtraction 2D X- ray images because they do not consider the spectral response difference between CT and onboard imaging. Given that the spectral response is affected by multiple components (e.g., X- ray tube, filtration, detector response) and these components often contain proprietary information, accurate modeling of each component can be challenging. To achieve effective spectral response matching, a phantom calibration technique is used, in which the entire imaging system is considered as a whole, instead of individual components. As a non-limiting example, a CIRS 062M electron density phantom can be used to perform the calibration. The CIRS 062M phantom has twelve tissue equivalent inserts with known dimensions to emulate human tissues of various densities ranging from 0.205 g / cm3to 2.15 g / cm3. The phantom can5QB\630666.01652\99810157.1Mayo 2024-390630666.01652 be scanned using clinically relevant voltages on both the CT scanner and the radiotherapy onboard imaging unit. Linear attenuation coefficients of the individual inserts can then be obtained on both systems and calibration curves can be established to map the linear attenuation coefficients obtained with the two systems. The calibration curves can then be used to scale HU values of the prior CT image data to match the spectral response of the on-board imaging system. These scaled HU values can then be used as an input to Plastimatch to accurately calculate the primary X-ray signal.

[0023] Secondary signal X-ray images are also generated from the non-essential tissue CT image data. In some instances, secondary X-ray images may be generated from the non- essential tissue CT image data using scaled HU values after spectral response matching. As a non-limiting example, a GPU-based Monte Carlo (MC) method can be used to simulate the secondary (e g., scatter) signal. For instance, the scatter signal can be obtained using a GPUbased Metropolis Monte Carlo (gMMC) simulation. Given the slow varying nature of the scatter signal, the simulation matrix size can be reduced (e g., 4: 1 per dimension) to expedite simulation. To further improve efficiency, the entire photon transport path can be sampled and only photons that reach the detector can be focused on.

[0024] The noise in the MC simulated scatter signal can be removed through a denoise process specifically designed for Poisson noise. The rationale for excluding noise in the simulated non-essential tissue images is based on the subtraction operation. Considering two variables A and B with noises of (JAand (JB, respectively, the difference A — B has noise of. Adding noise to a simulated 2D X-ray image may give it (variable B ) a more realistic appearance, but it will add unnecessary noise to the subtraction X-ray image ( A — B )•

[0025] The simulated primary and secondary’ signals are then combined to generate the simulated non-essential tissue X-ray images. Relative weighting of simulated primary and scatter signals can be calibrated using anthropomorphic phantoms. For example, anthropomorphic phantoms can be scanned using the CT scanner used to acquire the prior CT image data. From the resulting phantom CT image data, 2D X-rays can be simulated at projection angles matching those of the on-board imaging system. These simulated images can be compared with real 2D X-ray images acquired using the on-board imaging system and a linear square fitting can be performed to find the optimal weightings.6QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0026] On-board X-ray image data of the subject are also accessed with the computer system, as indicated at step 106. Accessing the on-board X-ray image data may include retrieving such data from a memory or other suitable data storage device or medium. Additionally or alternatively, accessing the on-board X-ray image data may include acquiring such data with an on-board imaging system and transferring or otherwise communicating the data to the computer system, which may be a part of the on-board imaging system and / or the radiotherapy system.

[0027] Co-registered X-ray image data that are effectively coregistered with the non- essential tissue images are then generated, as indicated at step 108. As illustrated in FIG. 2, on treatment days, the patient is first setup on the treatment couch. As a non-limiting example, the patient can be setup on the treatment couch based on external lasers or using other radiotherapy alignment techniques. One set of orthogonal X-rays is acquired to capture the initial position of the patient.

[0028] Registration is performed betw een the initial X-ray image(s) and the prior CT images. As one example, registration can be performed based on bony structures in the initial X-ray image(s) and the prior CT images. In some instances, bony registration may not be preferred for organs away from the tumor or affected by respiratory' motion. In such instances, other registration techniques can also be used, such as deformable image registration.

[0029] Then, a couch shift is applied to reposition the patient to match the patient anatomy depicted in the prior CT image data, which serves as the reference from which a radiotherapy treatment plan is created. To confirm the patient is in the correct position, clinically, another set of X-rays can be acquired after the couch shift. The second set of X-ray images is effectively coregistered with the simulated X-rays of the non-essential tissues; that is, the non-essential tissue images that were generated based on the prior CT image data. Therefore, the subtraction can be performed directly between the two. Notably, the registration, couch shift, and second (shifted) X-ray imaging are routinely performed as a standard-of-care practice.

[0030] Subtraction X-ray images are then generated, as indicated at step 110, by subtracting the non-essential tissue images from the co-registered X-ray images. As described above, these subtraction X-ray images have an enhanced tumor contrast and / or enhanced visualization of tumor based on the removal of non-essential tissues from the on-board X-ray images.7QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0031] The subtraction X-ray images can then be displayed to a user, stored for later use or further processing, or both, as indicated at step 112. For example, the subtraction X-ray images can be displayed to a user on a display of the radiotherapy system, treatment planning system, or the like. Additionally or alternatively, the subtraction X-ray images can be provided to the radiotherapy system to facilitate delivery of radiotherapy to the tumor(s) depicted in the subtraction X-ray images, and / or provided to a treatment planning system to update, adjust, or verily a treatment plan for the tumor(s) depicted in the subtraction X-ray images.

[0032] In some implementations, the quality of the subtraction X-ray images can be improved through effective handling of respiratory' motion. In general, non-essential tissue images can be simulated for each individual phase of a 4DCT scan and the one that matches the diaphragm position can be selected to further improve the quality of the subtraction X-ray images.

[0033] For disease sites affected by respiratory' motion (e.g., lung cancer), if the simulated non-essential tissue images and on-board X-ray images are in different respiratory states, direct subtraction of the two may leave a high residual signal around the diaphragm region. This high residual signal may cover up the tumor and make it invisible. To further improve the quality of subtraction X-ray images, the residual signal around the diaphragm can be suppressed by utilizing 4DCT image data of the patient acquired in the same session and setup as the prior CT image data. Each 4DCT image data set includes a number of respiratory phases, such as ten phases (or ten volumetric CT datasets), corresponding to that number of respiratory states. Clinically, 4DCT image data are routinely acquired in the standard-of-care practice.

[0034] Non-essential tissue images can be generated for each individual phase of the 4DCT image data using the methods described above. These images can be referred to as phases of non-essential tissue images, following standard 4DCT naming conventions. Given that primary bony structures (e.g., vertebral bodies) remain stable for all phases of non-essential tissue images, applying bony registration can bring all phases of non-essential tissue images into alignment with on-board X-ray images. Subsequently, subtraction will be performed between on-board X-ray images and each individual phase of non-essential tissue images, resulting in the number of phases (e.g., ten phases) of subtraction X-ray images.

[0035] A region-of-interest (ROI) can be placed around the diaphragm and absolute signal intensity summed over the ROI. The phase with the minimal absolute signal summation8QB\630666.01652\99810157.1Mayo 2024-390630666.01652 has the most matching respiratory state and, therefore, has the minimal residual signal. This phase can thus be selected as the final subtraction X-ray image that is output to the user.

[0036] The methods described above can be adapted to enhance the contrast of anatomical targets other than tumors, and for treatment modalities other than radiotherapy. Additionally or alternatively, the methods described above can be adapted to work with medical imaging modalities other than CT and X-ray imaging.

[0037] As one non-limiting example, an image with enhanced contrast for an anatomical target to receive treatment can be generated by accessing a first medical image, or first set of medical image data. The first medical image can be a prior medical image that has been previously acquired from a subject using a first medical imaging system. As described above, the first medical image may include planning CT image data, or other prior CT image data, acquired with a CT system. In other implementations, the first medical image can include one or more images acquired with other medical imaging modalities, such as X-ray fluoroscopy, magnetic resonance imaging (MRI), ultrasound, or the like.

[0038] A non-essential tissue image is then generated from the first medical image by removing signal contributions from the anatomical target in the first medical image. The non- essential tissue image depicts tissues that are non-essential for aligning the anatomical target to receive treatment from the designated treatment modality. As described above, the treatment modality may include radiotherapy, including external beam radiotherapy. Additionally or alternatively, the treatment modality may include brachytherapy, stereotactic treatments, ablation, robotic assisted surgical treatments, and so on.

[0039] A second medical image of the subject is acquired with a second medical imaging system. As described above, the second medical image may include an X-ray image acquired with an on-board imaging system of a radiotherapy system. In other implementations, the second medical image can be acquired with other medical imaging modalities (e.g., those noted above), and may be acquired with a medical imaging system other than an on-board imaging system.

[0040] A subtraction image is then generated by subtracting the non-essential tissue image from the second medical image. As described above, the subtraction image has improved visualization of the anatomical target relative to the second medical image by way of removing signal contributions of the non-essential tissues from the second medical image. The subtraction image can then be output to a user.9QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0041] In another non-limiting example, an image with enhanced contrast for an anatomical target to receive treatment can be generated using a suitably trained machine learning model. In these examples, medical image data may be accessed with a computer system, where the medical image data have been acquired from a subject using a medical imaging system.

[0042] A machine learning model is also accessed with the computer system. In general, the machine learning model has been trained on training data to remove signal contributions from medical images that are associated with tissues that are non-essential to aligning the anatomical target with a treatment modality. Accessing the trained machine learning model may include accessing model parameters (e.g.. weights, biases, or both) that have been optimized or otherwise estimated by training the machine learning model on training data. In some instances, retrieving the machine learning model can also include retrieving, constructing, or otherwise accessing the particular model architecture to be implemented. For instance, data pertaining to the layers in a neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers, hyperparameters for layers) may be retrieved, selected, constructed, or otherwise accessed.

[0043] The machine learning model may include a neural network. In general, the neural network can implement any number of different neural network architectures. For instance, the neural network could implement a convolutional neural network, a residual neural network, or the like. Alternatively, the neural network could be replaced with other suitable machine learning or artificial intelligence algorithms, such as those based on supervised learning, unsupervised learning, deep learning, ensemble learning, dimensionality reduction, and so on.

[0044] A neural network generally includes an input layer, one or more hidden layers (or nodes), and an output layer. Typically, the input layer includes as many nodes as inputs provided to the artificial neural network. The number (and the type) of inputs provided to the artificial neural network may vary based on the particular task for the artificial neural network.

[0045] The input layer connects to one or more hidden layers. The number of hidden layers varies and may depend on the particular task for the artificial neural network. Additionally, each hidden layer may have a different number of nodes and may be connected to the next layer differently. For example, each node of the input layer may be connected to each node of the first hidden layer. The connection between each node of the input layer and each node of the first hidden layer may be assigned a weight parameter. Additionally, each10QB\630666.01652\99810157.1Mayo 2024-390630666.01652 node of the neural network may also be assigned a bias value. In some configurations, each node of the first hidden layer may not be connected to each node of the second hidden layer. That is, there may be some nodes of the first hidden layer that are not connected to all of the nodes of the second hidden layer. The connections between the nodes of the first hidden layers and the second hidden layers are each assigned different weight parameters. Each node of the hidden layer is generally associated with an activation function. The activation function defines how the hidden layer is to process the input received from the input layer or from a previous input or hidden layer. These activation functions may vary and be based on the type of task associated with the artificial neural network and also on the specific type of hidden layer implemented.

[0046] Each hidden layer may perform a different function. For example, some hidden layers can be convolutional hidden layers which can, in some instances, reduce the dimensionality of the inputs. Other hidden layers can perform statistical functions such as max pooling, which may reduce a group of inputs to the maximum value; an averaging layer; batch normalization; and other such functions. In some of the hidden layers each node is connected to each node of the next hidden layer, which may be referred to then as dense layers. Some neural networks including more than, for example, three hidden layers may be considered deep neural networks.

[0047] The last hidden layer in the artificial neural network is connected to the output layer. Similar to the input layer, the output layer typically has the same number of nodes as the possible outputs.

[0048] As described above, the machine learning model is trained on training data. As a non-limiting example where the machine learning model is a neural network, the neural network can be trained by first accessing training data with a computer system. Accessing the training data may include retrieving such data from a memory or other suitable data storage device or medium. Alternatively, accessing the training data may include acquiring such data with one or more medical imaging systems and transferring or otherwise communicating the data to the computer system.

[0049] In general, the training data can include medical images and enhanced medical images. For instance, the enhanced medical images can be generated from the medical images in the training data and can include images in which signal contributions from non-essential tissues have been removed or otherwise reduced. The method can include assembling training data from such medical images using a computer system. This step may include assembling11QB\630666.01652\99810157.1Mayo 2024-390630666.01652 the medical images and enhanced medical images into an appropriate data structure on which the neural network or other machine learning algorithm can be trained. In general, the neural network can then be trained by optimizing network parameters (e.g., weights, biases, or both) based on minimizing a loss function. As one non-limiting example, the loss function may be a mean squared error loss function.

[0050] Training a neural network may include initializing the neural network, such as by computing, estimating, or otherwise selecting initial network parameters (e.g., weights, biases, or both). During training, an artificial neural netw ork receives the inputs for a training example and generates an output using the bias for each node, and the connections between each node and the corresponding weights. For instance, training data can be input to the initialized neural network, generating output as an enhanced medical image. The artificial neural network then compares the generated output with the actual output of the training example in order to evaluate the quality of the output image. For instance, the output image can be passed to a loss function to compute an error. The current neural network can then be updated based on the calculated error (e.g., using backpropagation methods based on the calculated error). For instance, the current neural network can be updated by updating the network parameters (e.g., weights, biases, or both) in order to minimize the loss according to the loss function. The training continues until a training condition is met. The training condition may correspond to, for example, a predetermined number of training examples being used, a minimum accuracy threshold being reached during training and validation, a predetermined number of validation iterations being completed, and the like. When the training condition has been met (e.g., by determining whether an error threshold or other stopping criterion has been satisfied), the current neural network and its associated network parameters represent the trained neural network. Different types of training processes can be used to adjust the bias values and the w eights of the node connections based on the training examples. The training processes may include, for example, gradient descent, Newton's method, conjugate gradient, quasi-Newton, Levenberg-Marquardt, among others.

[0051] In the inference stage, the medical image data are applied to the machine learning model to generate an enhanced medical image of the subject as an output. As described above, the enhanced medical image has improved visualization of the anatomical target relative to the medical image data by way of removing signal contributions of non-essential tissues from the medical image data. This enhanced medical image can then be output to a user.12QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0052] In an example study, the disclosed systems and methods were evaluated by generating subtraction X-ray images with improved tumor contrast. In this example study, two sets of CT images acquired from the same patient but at different dates were utilized. The first CT image data set was treated as the prior CT image data acquired at the very beginning of the process. This first CT image data set provides prior knowledge about patient anatomy, as illustrated in FIG. 3A. Non-essential tissues were extracted from the prior CT image data by overriding the Hounsfield unit values of everything in the thoracic cavity to air, as shown in FIG. 3B. Contributions of non-essential tissues to X-ray images were approximated using the digitally reconstructed radiograph (DRR) of this CT image of non-essential tissues, as shown in FIG. 3C. Patient anatomy on the treatment day was represented by the second CT image data set, as shown in FIG. 3D. The DRR of the treatment day CT (FIG. 3E) corresponded to the 2D X-ray image on the treatment day. After bony registration, the contributions of non-essential tissues were removed from the treatment day X-ray image by subtracting the X-ray image in FIG. 3C from the X-ray image in FIG. 3E. The resulting subtraction X-ray image, shown in FIG. 3F, had superior tumor contrast (yellow arrow) compared to the treatment day X-ray image (FIG. 3E).

[0053] The disclosed systems and methods demonstrate enhanced tumor contrast to improve patient setup accuracy, and to direct tumor tracking for more effective motion management strategies in radiotherapy treatment of moving targets such as lung tumors. The disclosed systems and methods have applications in other image-guided procedures such as lung biopsy, surgical removal of lung nodules, and so on.

[0054] FIG. 4 shows an example of a system 400 for generating subtraction X-ray images with enhanced tumor contrast and / or visualization in accordance with some embodiments described in the present disclosure. As shown in FIG. 4, a computing device 450 can receive one or more types of data (e g., planning CT image data or other prior CT image data, phantom image data, on-board X-ray image data or other X-ray image data) from data source 402. In some embodiments, computing device 450 can execute at least a portion of an enhanced tumor contrast image generation system 404 to generate subtraction X-ray images with enhanced tumor contrast and / or visualization from data received from the data source 402.

[0055] Additionally or alternatively, in some embodiments, the computing device 450 can communicate information about data received from the data source 402 to a server 452 over a communication network 454, which can execute at least a portion of the enhanced tumor contrast image generation system 404. In such embodiments, the server 452 can return13QB\630666.01652\99810157.1Mayo 2024-390630666.01652 information to the computing device 450 (and / or any other suitable computing device) indicative of an output of the enhanced tumor contrast image generation system 404.

[0056] In some embodiments, computing device 450 and / or server 452 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. The computing device 450 and / or server 452 can also reconstruct images from the data.

[0057] In some embodiments, data source 402 can be any suitable source of data (e.g., measurement data, images reconstructed from measurement data, processed image data), such as a CT system, an on-board imaging system, another computing device (e g., a server storing measurement data, images reconstructed from measurement data, processed image data), and so on. In some embodiments, data source 402 can be local to computing device 450. For example, data source 402 can be incorporated with computing device 450 (e.g., computing device 450 can be configured as part of a device for measuring, recording, estimating, acquiring, or otherwise collecting or storing data). As another example, data source 402 can be connected to computing device 450 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, data source 402 can be located locally and / or remotely from computing device 450, and can communicate data to computing device 450 (and / or server 452) via a communication network (e.g., communication network 454).

[0058] In some embodiments, communication network 454 can be any suitable communication network or combination of communication networks. For example, communication network 454 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), other types of wireless network, a wired network, and so on. In some embodiments, communication network 454 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG. 4 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.14QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0059] Referring now to FIG. 5, an example of hardware 500 that can be used to implement data source 402, computing device 450, and server 452 in accordance with some embodiments of the systems and methods described in the present disclosure is show n.

[0060] As shown in FIG. 5, in some embodiments, computing device 450 can include a processor 502, a display 504, one or more inputs 506, one or more communication systems 508, and / or memory 510. In some embodiments, processor 502 can be any suitable hardware processor or combination of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), and so on. In some embodiments, display 504 can include any suitable display devices, such as a liquid cry stal display (LCD) screen, a light-emitting diode (LED) display, an organic LED (OLED) display, an electrophoretic display (e.g., an “e-ink” display), a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 506 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0061] In some embodiments, communications systems 508 can include any suitable hardware, firmware, and / or software for communicating information over communication network 454 and / or any other suitable communication networks. For example, communications systems 508 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 508 can include hardware, firmware, and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0062] In some embodiments, memory 510 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 502 to present content using display 504, to communicate with server 452 via communications system(s) 508, and so on. Memory 510 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 510 can include random-access memory' (RAM), read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), other forms of volatile memory, other forms of non-volatile memory, one or more forms of semivolatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 510 can have encoded thereon, or otherw ise stored therein, a computer program for controlling operation of computing device 450. In such embodiments, processor 502 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive15QB\630666.01652\99810157.1Mayo 2024-390630666.01652 content from server 452, transmit information to server 452, and so on. For example, the processor 502 and the memory 510 can be configured to perform the methods described herein (e.g., the method of FIG. 1, the workflow illustrated in FIG. 2).

[0063] In some embodiments, server 452 can include a processor 512, a display 514, one or more inputs 516, one or more communications systems 518, and / or memory7520. In some embodiments, processor 512 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 514 can include any suitable display devices, such as an LCD screen, LED display, OLED display, electrophoretic display, a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 516 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0064] In some embodiments, communications systems 518 can include any suitable hardware, firmware, and / or software for communicating information over communication network 454 and / or any other suitable communication networks. For example, communications systems 518 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 518 can include hardware, firmware, and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0065] In some embodiments, memory 520 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 512 to present content using display 514, to communicate with one or more computing devices 450, and so on. Memory7520 can include any suitable volatile memory, non-volatile memory7, storage, or any suitable combination thereof. For example, memory 520 can include RAM. ROM, EPROM, EEPROM, other ty pes of volatile memory, other ty pes of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 520 can have encoded thereon a server program for controlling operation of server 452. In such embodiments, processor 512 can execute at least a portion of the server program to transmit information and / or content (e.g., data, images, a user interface) to one or more computing devices 450, receive information and / or content from one or more computing devices 450, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.16QB\630666.01652\99810157.1Mayo 2024-390630666.01652

[0066] In some embodiments, the server 452 is configured to perform the methods described in the present disclosure. For example, the processor 512 and memory 520 can be configured to perform the methods described herein (e g., the method of FIG. 1, the workflow illustrated in FIG. 2).

[0067] In some embodiments, data source 402 can include a processor 522, one or more data acquisition systems 524, one or more communications systems 526, and / or memory 528. In some embodiments, processor 522 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more data acquisition systems 524 are generally configured to acquire data, images, or both, and can include a CT system, an on-board imaging system, etc.. Additionally or alternatively, in some embodiments, the one or more data acquisition systems 524 can include any suitable hardware, firmware, and / or software for coupling to and / or controlling operations of a CT system, an onboard imaging system, or the like. In some embodiments, one or more portions of the data acquisition system(s) 524 can be removable and / or replaceable.

[0068] Note that, although not shown, data source 402 can include any suitable inputs and / or outputs. For example, data source 402 can include input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, data source 402 can include any suitable display devices, such as an LCD screen, an LED display, an OLED display, an electrophoretic display, a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.

[0069] In some embodiments, communications systems 526 can include any suitable hardware, firmware, and / or software for communicating information to computing device 450 (and, in some embodiments, over communication network 454 and / or any other suitable communication networks). For example, communications systems 526 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 526 can include hardware, firmware, and / or software that can be used to establish a wired connection using any suitable port and / or communication standard (e.g.. VGA, DVI video, USB, RS-232, etc.). Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0070] In some embodiments, memory 528 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 522 to control the one or more data acquisition systems 524. and / or receive data from the one or more data acquisition systems 524; to generate images from data;17QB\630666.01652\99810157.1Mayo 2024-390630666.01652 present content (e.g., data, images, a user interface) using a display; communicate with one or more computing devices 450; and so on. Memory 528 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 528 can include RAM, ROM, EPROM, EEPROM, other ty pes of volatile memory, other types of non-volatile memory, one or more types of semi-volatile memory, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 528 can have encoded thereon, or otherwise stored therein, a program for controlling operation of data source 402. In such embodiments, processor 522 can execute at least a portion of the program to generate images, transmit information and / or content (e.g., data, images, a user interface) to one or more computing devices 450, receive information and / or content from one or more computing devices 450, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.

[0071] In some embodiments, any suitable computer-readable media can be used for storing instructions for performing the functions and / or processes described herein. For example, in some embodiments, computer-readable media can be transitory or non-transitory. For example, non-transitory' computer-readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., RAM, flash memory’, EPROM, EEPROM), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and / or any suitable tangible media. As another example, transitory’ computer- readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and / or any suitable intangible media.

[0072] As used herein in the context of computer implementation, unless otherwise specified or limited, the terms “component,” “system,” “module,” “framework,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process or thread of execution, may be localized on one18QB\630666.01652\99810157.1Mayo 2024-390 630666.01652 computer, may be distributed between two or more computers or other processor devices, or may be included within another component (or system, module, and so on).

[0073] In some implementations, devices or systems disclosed herein can be utilized or installed using methods embodying aspects of the disclosure. Correspondingly, description herein of particular features, capabilities, or intended purposes of a device or system is generally intended to inherently include disclosure of a method of using such features for the intended purposes, a method of implementing such capabilities, and a method of installing disclosed (or otherwise known) components to support these purposes or capabilities. Similarly, unless otherwise indicated or limited, discussion herein of any method of manufacturing or using a particular device or system, including installing the device or system, is intended to inherently include disclosure, as embodiments of the disclosure, of the utilized features and implemented capabilities of such device or system.

[0074] The present disclosure has described one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.19QB\630666.01652\99810157.1

Claims

Mayo 2024-390630666.01652CLAIMS1. A method for generating an image with enhanced tumor contrast, the method comprising: accessing computed tomography (CT) image data with a computer system, wherein the CT image data have been previously acquired from a subject using a CT system; generating a simulated X-ray image from the CT image data with the computer system, wherein the simulated X-ray image depicts non-essential tissues for a radiotherapy plan in the subject; acquiring an X-ray image of the subject with an on-board imaging system of a radiotherapy system; generating a subtraction X-ray image with the computer system by subtracting the simulated X-ray image from the X-ray image, wherein the subtraction X-ray image has improved visualization of tumor tissues relative to the X-ray image by way of removing the non-essential tissues from the X-ray image; and outputting the subtraction X-ray image with the computer system.

2. The method of claim 1 , wherein generating the simulated X-ray image from the CT image data comprises generating non-essential tissue CT image data by retaining only- image signals in the CT image data associated with the non-essential tissues and generating the simulated X-ray image from the non-essential tissue CT image data.

3. The method of claim 2, wherein retaining only image signals in the CT image data associated with the non-essential tissues comprises keeping Hounsfield unit values for image pixels associated with the non-essential tissues and setting the Hounsfield unit values of other image pixels to the Hounsfield unit value for air.

4. The method of claim 2. further comprising: simulating a primary X-ray signal image from the non-essential tissue CT image data, wherein the primary X-ray' signal image is indicative of primary X-ray signals; simulating a secondary- X-ray signal image from the non-essential tissue CT image data, wherein the secondary X-ray signal image is indicative of X-ray scattering signals; and20QB\630666.01652\99810157.1Mayo 2024-390630666.01652 generating the simulated X-ray image by combining the primary' X-ray signal image and the secondary X-ray signal image.

5. The method of claim 4, wherein the primary X-ray signal image is simulated by generating a digitally reconstructed radiography (DRR) from the non-essential tissue CT image data.

6. The method of claim 4, wherein the primary X-ray signal image is simulated by generating a ray tracing from the non-essential tissue CT image data.

7. The method of claim 6, wherein the ray tracing comprises a Siddon ray tracing.

8. The method of claim 4. wherein the primary X-ray signal image is calibrated to account for differences in a spectral response of the CT system and a spectral response of the on-board imaging system.

9. The method of claim 8, wherein the primary X-ray signal image is calibrated using calibration data generated from phantom CT image data acquired from a phantom with the CT system and phantom X-ray image data acquired from the phantom with the on-board imaging system.

10. The method of claim 9. wherein the calibration data are generated by: generating first linear attenuation coefficient data from the phantom CT image data; generating second linear attenuation coefficient data from the phantom X-ray image data; and generating calibration curves by mapping the first linear attenuation coefficient data to the second linear attenuation coefficient data.

11. The method of claim 10, wherein calibrating the primary X-ray signal image using calibration data comprises scaling Hounsfield unit values in the primary X-ray signal image using the calibration curves of the calibration data.21QB\630666.01652\99810157.1Mayo 2024-390630666.0165212. The method of claim 4. wherein simulating the secondary X-ray signal image comprises generating a noisy secondary X-ray signal image from the non-essential tissue CT image data, denoising the noisy secondary' X-ray signal image; and storing the denoised secondary X-ray signal image as the secondary X-ray signal image.

13. The method of claim 12, wherein the noisy secondary X-ray signal image is denoised using a denoising process designed for Poisson noise.

14. The method of claim 4 or 12, wherein the secondary X-ray signal image is simulated by a Monte Carlo simulation to estimate X-ray scattering based on the non- essential tissue CT image data.

15. The method of claim 14, wherein the Monte Carlo simulation comprises a GPU-based Metropolis Monte Carlo (gMMC) simulation.

16. The method of claim 1, wherein acquiring the X-ray image comprises: acquiring an initial X-ray image; co-registering the initial X-ray image with the CT image data; determining patient couch shift values based on co-registering the initial X-ray image with the CT image data; shifting a patient couch of the radiotherapy system using the patient couch shift values; acquiring a co-registered X-ray image with the on-board imaging system of the radiotherapy system after shifting the patient couch; and storing the co-registered X-ray image as the X-ray image.

17. The method of claim 16, wherein the initial X-ray image is co-registered with the CT image data using a bony registration.

18. The method of claim 16, wherein the initial X-ray image is co-registered with the CT image data using a deformable registration.22QB\630666.01652\99810157.1Mayo 2024-390630666.0165219. The method of claim 1. further comprising reducing residual signals around a diaphragm of the subject in the subtraction X-ray image.

20. The method of claim 19, wherein reducing residual signals around the diaphragm of the subject in the subtraction X-ray image comprises: generating a plurality of subtraction X-ray images, each of the plurality of subtraction X-ray images being generated for a different one of a plurality of respiratory phases of the subject; and selecting one of the plurality of subtraction X-ray images with minimal residual signal around the diaphragm as the subtraction X-ray image.

21. The method of claim 20, wherein the CT image data comprises CT image data acquired in each of the plurality of respiratory' phases of the subject and generating the plurality of subtraction X-ray images comprises: generating a plurality of simulated X-ray images, wherein the plurality of simulated X-ray images includes a different simulated X-ray image for each of the plurality of respiratory7phases; and generating the plurality of subtraction X-ray images by subtracting each of the plurality of simulated X-ray images from the X-ray image.

22. The method of claim 21, wherein generating the plurality of subtraction X-ray images comprises co-registering the plurality of simulated X-ray images with the X-ray image before subtracting each of the plurality of simulated X-ray images from the X-ray image.

23. The method of claim 22, wherein each of the plurality of simulated X-ray images are co-registered with the X-ray image using a bony registration.

24. The method of claim 20, wherein selecting the one of the plurality of subtraction X-ray images with minimal residual signal around the diaphragm comprises: placing a region-of-interest around the diaphragm in each of the plurality of subtraction X-ray images; summing an absolute signal intensity’ over the region-of-interest; and23QB\630666.01652\99810157.1Mayo 2024-390630666.01652 selecting the one of the plurality of subtraction X-ray images as the subtraction X-ray image having a minimal absolute signal intensity.

25. The method of claim 1, wherein the non-essential tissues comprise tissue not used for alignment with the radiotherapy plan.

26. The method of claim 25, wherein the non-essential tissues comprise bones.

27. The method of claim 25 or 26, wherein the non-essential tissues comprise muscles.

28. The method of claim 1, wherein outputting the subtraction X-ray image comprises updating a treatment couch position of a radiotherapy system based on the subtraction X-ray image to guide radiotherapy treatment to the tumor tissues with the radiotherapy system.

29. A method for generating an image with enhanced contrast for an anatomical target to receive treatment, the method comprising: accessing a first medical image with a computer system, wherein the first medical image has been previously acquired from a subject using a first medical imaging system; generating a non-essential tissue image from the first medical image by removing signal contributions from the anatomical target in the first medical image, wherein the non-essential tissue image depicts tissues that are non-essential for aligning the anatomical target to receive the treatment; acquiring second medical image of the subject with a second medical imaging system; generating a subtraction image with the computer system by subtracting the non- essential tissue image from the second medical image, wherein the subtraction image has improved visualization of the anatomical target relative to the second medical image by way of removing signal contributions of the non-essential tissues from the second medical image; and outputting the subtraction image with the computer system.24QB\630666.01652\99810157.1Mayo 2024-390630666.0165230. The method of claim 29, wherein the first medical image comprises a computed tomography (CT) image.

31. The method of claim 29 or 30, wherein the second medical image comprises an X-ray image.

32. The method of claim 29, wherein the treatment comprises radiotherapy.

33. A method for generating an image with enhanced contrast for an anatomical target to receive treatment, the method comprising: accessing medical image data with a computer system, wherein the medical image data have been acquired from a subject using a medical imaging system; accessing a machine learning model with the computer system, wherein the machine learning model has been trained on training data to remove signal contributions from medical images that are associated with tissues that are non-essential to aligning the anatomical target with a treatment modality; applying the medical image data to the machine learning model, generating an enhanced medical image as an output, wherein the enhanced medical image has improved visualization of the anatomical target relative to the medical image data by way of removing signal contributions of non-essential tissues from the medical image data; and outputting the enhanced medical image with the computer system.

34. The method of claim 33, wherein the machine learning model comprises a neural network.

35. The method of claim 33 or 34, wherein the medical image comprises a computed tomography (CT) image.

36. The method of claim 33 or 34, wherein the medical image comprises an X-ray image.25QB\630666.01652\99810157.1Mayo 2024-390630666.0165237. The method of claim 33 or 34, wherein the medical image comprises a magnetic resonance image.

38. The method of any one of claims 33-37, wherein he treatment modality comprises radiotherapy.26QB\630666.01652\99810157.1

Images