Adaptive radiation therapy using composite imaging slices

Abstract

A system and method for registering images are provided. The system and method perform operations including: receiving a first imaging slice including an object at a first time point during a given radiation phase, the first imaging slice corresponding to a first plane; accessing a composite imaging slice corresponding to the first plane at a first time point during the given radiation phase, the composite imaging slice generated using a plurality of imaging slices acquired before the first time point; spatially registering the first imaging slice with the composite imaging slice; determining movement of the object using the spatially registered first imaging slice and the composite imaging slice; and generating an updated treatment plan to control delivery of a treatment beam based on the determined movement.

Description

[0001] Priority claim

[0002] This patent application claims the benefit of priority to U.S. patent application serial number 16 / 824,276, filed on March 19, 2020, entitled “ADAPTIVE RADIATION THERAPYUSING COMPOSITE IMAGING SLICES,” and U.S. provisional patent application serial number 62 / 886,030, filed on August 13, 2019, entitled “AUTOMATIC GATING WITH A MAGNETIC RESONANCE LINEARACCELERATOR (MR LINAC),” the entire contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The present disclosure relates generally to radiation therapy or radiotherapy. More specifically, the present disclosure relates to systems and methods for adjusting a radiation therapy treatment plan to compensate for changes in the position of a target tumor during the delivery of the radiation therapy. Background Art

[0004] Radiation therapy, or "radiotherapy," can be used to treat cancer or other diseases in mammalian (e.g., human and animal) tissue. One such radiotherapy technique is the Gamma Knife, whereby a patient is irradiated with a large number of low-intensity gamma rays that are focused at a target (e.g., a tumor) with high intensity and precision. In another embodiment, radiotherapy is provided using a linear accelerator, whereby the tumor is irradiated with high-energy particles (e.g., electrons, protons, ions, high-energy photons, etc.). The placement and dosage of the radiation beam must be accurately controlled to ensure that the tumor receives the prescribed radiation and that the beam is positioned to minimize damage to surrounding healthy tissue, often referred to as organs at risk (OARs). The radiation is referred to as a "prescription" because the physician customizes a predetermined amount of radiation to the tumor and surrounding organs, similar to a prescription for a medication.

[0005] Typically, ionizing radiation in the form of a collimated beam is directed toward the patient from an external radiation source.

[0006] Specified or selectable beam energies may be used, for example, to deliver a range of diagnostic energy levels or a range of therapeutic energy levels. Modulation of the radiation beam may be provided by one or more attenuators or collimators (e.g., a multi-leaf collimator). The intensity and shape of the radiation beam may be adjusted by collimation to avoid damaging healthy tissue adjacent to the target tissue (e.g., an organ at risk) by conforming the projected beam to the contours of the target tissue.

[0007] The treatment planning process can include using three-dimensional images of the patient to identify the target area (e.g., a tumor) and identify critical organs near the tumor. Creating a treatment plan can be a time-consuming process in which the planner attempts to adhere to various treatment goals or constraints (e.g., dose volume histogram (DVH) goals), taking into account their individual importance (e.g., weighting) in order to develop a clinically acceptable treatment plan. This task can be a time-consuming trial-and-error process that is complicated by the various organs at risk (OARs) because the complexity of the process increases as the number of OARs increases (e.g., up to 13 OARs for head and neck treatment). OARs far from the tumor may be easily irradiated, while OARs close to or overlapping the target tumor may be difficult to avoid.

[0008] Traditionally, for each patient, the initial treatment plan can be generated in an "offline" manner. The treatment plan can be formulated well before the delivery of radiation therapy, for example, using one or more medical imaging techniques. Imaging information can include images such as from X-rays, computed tomography (CT), magnetic resonance (MR), positron emission tomography (PET), single photon emission computed tomography (SPECT) or ultrasound. A healthcare provider, such as a doctor, can use three-dimensional (3D) imaging information indicating the patient's anatomical structure to identify one or more target tumors and organs at risk near the tumor. The healthcare provider can use manual techniques to depict the target tumor to be treated with a prescribed radiation dose, and the healthcare provider can similarly depict nearby tissues such as organs, which are at risk of damage from radiation therapy. Alternatively or additionally, an automated tool (e.g., ABAS provided by Elekta AB, Sweden) can be used to help identify or depict target tumors and organs at risk. A radiation therapy treatment plan ("treatment plan") can then be created using optimization techniques based on clinical and dosimetric goals and constraints (eg, maximum, minimum, and mean radiation dose to the tumor and critical organs).

[0009] The treatment plan can then be executed later by positioning the patient and delivering the prescribed radiation therapy. Radiation therapy treatment plans can include dose "fractionation," whereby a series of radiation therapy deliveries are provided over a predetermined period of time (e.g., 45 fractions), where each treatment delivery includes a specified fraction of the total prescribed dose. However, during treatment, the position of the patient and the target tumor relative to the treatment machine (e.g., a linear accelerator—"linac") is very important to ensure that the target tumor, and not healthy tissue, is irradiated.

[0010] The initial treatment plan can be adapted to the 3D pre-treatment images to compensate for any anatomical changes and setup differences. During the radiation therapy beam on period, additional two-dimensional (2D) cine images can be acquired and used to determine whether to manually pause the beam if the target moves outside of tolerance.

[0011] Overview

[0012] The inventors have recognized that, among other things, radiotherapy treatment plans can be adjusted simultaneously in an adaptive manner to compensate for changes in the position of the target tumor during the delivery of radiotherapy to the tumor. For example, the position of the desired target (e.g., a tumor or other object) may be displaced to such an extent that if a dedicated "offline" treatment planning method is used, the position of the tumor indicated by the medical image taken before the treatment may be significantly different from the position of the tumor target during the radiotherapy treatment. For example, the tumor may shrink, move, or be misaligned compared to the expected or desired position indicated in the treatment plan. The movement of the target may be caused by one or more sources such as cardiac movement, breathing, reflexes (e.g., coughing), or other movements. Therefore, when the radiotherapy is finally delivered, the position where the radiotherapy should be delivered based on the image taken before the treatment may be significantly misaligned with the desired target. In some cases, a "target site" is the position of a target or object, such as the position of the target or object before treatment, and a "treatment site" is the position where the radiotherapy is delivered during treatment (and ideally, the treatment site is aligned with the actual position of the target when the radiotherapy is delivered).

[0013] In one approach, imaging can be performed simultaneously with the delivery of radiation therapy, such as by acquiring imaging immediately prior to initiating radiation therapy delivery during a treatment session, or by using a series of corresponding treatment deliveries and then acquiring one or more images of the tumor immediately during a radiation therapy delivery session. Such imaging can provide information that helps identify the location of the target or identify the movement of the target. Such simultaneous imaging can often be referred to as "real time," but there is often a delay or time lag between image acquisition and the delivery of radiation therapy, which is typically about 100 to 500 milliseconds (ms).

[0014] Although many cine imaging strategies can be considered, this disclosure focuses on methods using sagittal and coronal 2D images centered at or near the target centroid. Other strategies can be employed, such as using a single slice or multiple parallel slices with arbitrary orientations.

[0015] A problem exists if the acquisition of 3D volumetric imaging information is delayed or the imaging acquisition rate is unacceptable (e.g., imaging acquisition is too slow to allow treatment guidance or control, such as greater than about 500 ms for a respiratory target). The subject matter described herein can address such issues, for example, by facilitating more rapid acquisition of imaging slices (including one or more of one-dimensional contours, two-dimensional slices, or three-dimensional volumes including sub-volumes or sub-regions of earlier imaged volumetric regions), including comparing information indicative of a portion of a target site rapidly acquired from one or more imaging slices with information obtained from an earlier acquired volumetric reference image.

[0016] In an example, an adaptive image-guided therapy delivery system can receive imaging information including a volumetric image that includes a target in a patient undergoing radiation therapy, and can receive imaging information corresponding to one or more imaging slices that include different portions of the target, the imaging slices being acquired at different times after the volumetric image is acquired.

[0017] According to various examples, the system receives a first imaging slice of an object depicted in a 3D pre-treatment volumetric image at a first time point during a given radiation phase. The first imaging slice is defined along a first plane (e.g., a sagittal plane). At a first time point during the given radiation phase, a second imaging slice of the object obtained at a second time point prior to the first time point during the given radiation phase is accessed. The second imaging slice is defined along a second plane (e.g., a coronal plane). The system determines movement of the object at the first time point in a first direction and a second direction (e.g., along the sagittal plane and left / right sides) based on registering the first imaging slice with the volumetric image. Movement of the object at the first time point in a third direction (e.g., a coronal plane) is determined based on the second imaging slice.

[0018] In an example, the registration between two images finds a transformation that maps one image to another. Typically, a deformable vector field (DVF) maps each point in the first image to a corresponding position in the second image. The first image may be referred to as a fixed image and the second image may be referred to as a moving image. The registration may be calculated using an optimization technique that attempts to find a registration value (DVF) that corresponds to an optimal value of a metric. The metric is a quantity that is considered to represent the success of the registration. As the optimization iterations get closer to a solution, it may use the metric to determine whether each potential step is better or worse than the previous one. In an example, the metric may be the sum of the differences between the moving image and the fixed image, but more complex metrics such as normalized cross-correlation or mutual information may also be included. The optimizer may iteratively improve the registration until the optimal value of the metric is found. Other techniques for registration may rely on machine learning. Rigid registration techniques may constrain the registration so that the moving image may be mapped to the fixed image by a global translation and rotation or by translation alone. If deformation is actually present, the algorithm attempts to find the best fit.

[0019] In some examples, the system receives a first imaging slice including the object at a first time point during a given radiation phase, the first imaging slice corresponding to a first plane (e.g., a sagittal plane or a coronal plane). The system accesses a composite imaging slice corresponding to the first plane at the first time point. The composite imaging slice is generated using a plurality of imaging slices acquired before the first time point. The first imaging slice is spatially registered with the composite imaging slice to determine movement of the object.

[0020] In some examples, the system receives a first imaging slice including an object at a first time point during a given radiation phase, the first imaging slice corresponding to a first plane (e.g., a sagittal plane). The system encodes the first imaging slice into a lower dimensional representation, and applies the trained machine learning model to the encoded first imaging slice to estimate an encoded version of a second imaging slice corresponding to a second plane (e.g., a coronal plane). The first imaging slice and the second imaging slice form a pair of imaging slices for the first time point. The pair of imaging slices are spatially registered to a volumetric image (e.g., a 3D pre-treatment image) to calculate the displacement of the object. In some implementations, the one or more imaging slices (e.g., a pair of imaging slices) are registered to the volumetric image by inserting the one or more imaging slices (e.g., a pair of imaging slices) into the 3D volume and filling the voxels of the 3D volume at the intersection of the slices and the voxels. In this way, the registration is a 3D-3D registration, but only the filled voxels of the 3D volume are considered. In another implementation, the one or more imaging slices (e.g., a pair of imaging slices) are directly registered to the 3D volume by optimizing a cost function that considers how different the slices in the one or more slices are from the volumetric image.

[0021] An updated treatment plan can be generated in an adaptive manner to align a treatment site established by a treatment beam with an updated target site or gate the beam so that treatment is delivered as the treatment site passes through the target site. As illustrative examples, updating the treatment plan can include one or more of the following: (a) adjusting an actuator coupled to a movable platform, such as a couch or table supporting a treatment recipient; (b) adjusting one or more apertures configured to collimate or shape the treatment beam; (c) adjusting one or more actuators configured to position the treatment output to establish a specified treatment beam direction; or (d) gating treatment delivery, such as using imaging or information obtained from other sensors.

[0022] This summary is intended to provide an overview of the subject matter of this patent application. It is not intended to provide an exclusive or exhaustive description of the subject matter of the present invention. The detailed description is included to provide additional information about this patent application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1A Examples of radiation therapy systems that may include radiation therapy outputs configured to provide treatment beams are generally shown in accordance with some embodiments.

[0024] Figure 1B Another example of a radiation therapy system in accordance with some embodiments is generally shown.

[0025] Figure 2 An example of a system is generally shown that may include a combined radiation therapy system and an imaging system, which may include, for example, a computed tomography (CT) imaging system, in accordance with some embodiments.

[0026] Figure 3 A partial cross-sectional view of an example of a system according to some embodiments is generally shown, which may include a combined radiation therapy system and an imaging system, such as may include a magnetic resonance (MR) imaging system.

[0027] Figure 4 Examples of collimator configurations in accordance with some embodiments are generally shown, such as may be used to one or more of shape, direct, or modulate the intensity of a radiotherapy beam.

[0028] Figure 5 An example of a radiation therapy system according to some embodiments is generally shown, which may include, for example, a radiation therapy device and an imaging acquisition device.

[0029] Figure 6 Generally shown is a view of a target site, such as a tumor region, according to some embodiments, such as may be obtained using volumetric imaging techniques.

[0030] Fig. 7A and Figure 7B An imaging acquisition plane (in Fig. 7A ) and the corresponding acquired imaging slices corresponding to the orientation of the first imaging plane (in Figure 7B in).

[0031] Figure 7C and Fig.7D An imaging acquisition plane (in Figure 7C ) and the corresponding acquired imaging slices corresponding to the orientation of the second imaging plane (in Fig.7D In the view of FIG. 1 , the orientation of the second imaging plane may be, for example, the same as that described above with respect to FIG. Fig. 7A and Figure 7B The first imaging plane is oriented orthogonally.

[0032] Fig. 7E and Figure 7F An imaging acquisition plane (in Fig. 7E ) and the corresponding acquired imaging slices corresponding to the orientation of the third imaging plane (in Figure 7F In the view of FIG. 1 , the third imaging plane orientation can be, for example, the same as that described above with respect to FIG. Fig. 7A , Figure 7B , Figure 7C and Fig.7D One or more of the first imaging plane orientation and the second imaging plane orientation are orthogonal.

[0033] Fig. 8A Generally shown are a series of imaging acquisition planes, such as may be obtained as a target site moves from one imaging acquisition instance to another, in accordance with some embodiments.

[0034] Figure 8B A series of imaging acquisition regions, which may include curved shapes, for example, are generally shown in accordance with some embodiments.

[0035] Figure 8C A series of imaging acquisition planes are generally shown according to some embodiments, which may, for example, include different orientations that need not each be orthogonal to the others.

[0036] Fig. 9A A series of two imaging slices is generally shown according to some embodiments, for example including a target site displaced between a first imaging slice and a second imaging slice.

[0037] Fig. 9B A series of two spatially registered imaging slices is generally shown, for example after adjusting a segmented portion of a target site in one or more imaging slices, according to some embodiments.

[0038] Fig.10 Techniques that may include spatially registering segmented portions of a target site according to some embodiments are generally shown.

[0039] Fig.11A Generally shown is a composite segmented representation of a target site according to some embodiments, such as after spatially registering portions of the target site acquired at different times.

[0040] Fig. 11B Techniques for determining a difference between a position of a target site from acquired spatially registered imaging information and an earlier position of the target site, such as represented by earlier acquired volumetric imaging information, in accordance with some embodiments are generally presented.

[0041] Fig. 11C Techniques for updating a treatment plan, such as to shift a treatment site to a new location, according to some embodiments are generally shown.

[0042] Fig.12 A second technique for determining movement of an object is shown in accordance with some implementations.

[0043] Fig.13 A timing diagram for determining movement of an object according to a second technique is shown in accordance with some implementations.

[0044] Fig.14 is a flow chart for determining movement of an object according to a second technique, according to some implementations.

[0045] Fig.15 A third technique for determining movement of an object is shown in accordance with some implementations.

[0046] Fig.16 Techniques for generating a composite slice for determining movement according to a third technique are shown in accordance with some implementations.

[0047] Fig.17 is a flow chart for determining movement of an object according to a third technique, in accordance with some implementations.

[0048] Fig.18 A fourth technique for determining movement of an object is shown in accordance with some implementations.

[0049] Fig.19 is a flow chart for determining movement of an object according to a fourth technique, in accordance with some implementations.

[0050] In the accompanying drawings, which are not necessarily drawn to scale, like numbers may describe similar components in different views. Like numbers with different letter suffixes may represent different instances of similar components. The accompanying drawings generally illustrate various embodiments discussed in this document by way of example and not limitation. DETAILED DESCRIPTION

[0051] Figure 1A An example of a radiation therapy system 102 is generally shown, which may include a radiation therapy output 104 configured to provide a treatment beam 108. The radiation therapy output 104 may include one or more attenuators or collimators, such as in Figure 4 A multi-leaf collimator (MLC) is described in the illustrative example of FIG. Return to reference Figure 1A , a patient can be positioned in area 112, such as on a platform 116 (e.g., a table or couch), to receive a radiation therapy dose in accordance with a radiation therapy treatment plan. The radiation therapy output 104 can be located on a gantry 106 or other mechanical support, such as to allow the treatment output 104 to rotate about an axis ("A"). One or more of the platform 116 or the radiation therapy output 104 can be moved to other positions, such as in a transverse direction ("T") or a lateral direction ("L"). Other degrees of freedom are possible, such as rotation about one or more other axes, such as rotation about a transverse axis (indicated as "R").

[0052] Figure 1A 1, including axes A, T, and L, can have an origin at an isocenter 110. The isocenter 110 can be defined as the location where the radiotherapy beam 108 intersects the origin of the coordinate axes, such as to deliver a prescribed radiation dose to or within a patient. For example, the isocenter 110 can be defined as the location where the radiotherapy beam 108 intersects the patient for various rotational positions of the radiotherapy output 104 positioned by the gantry 106 about the axis A.

[0053] In an example, the detector 114 can be located within the field of the treatment beam 108, and can include, for example, a flat panel detector (e.g., a direct detector or a scintillation-based detector). The detector 114 can be mounted on the gantry 106, opposite the radiation therapy output 104, for example, to maintain alignment with the treatment beam 108 as the gantry 106 rotates. In this manner, the detector 114 can be used to monitor the treatment beam 108, or the detector 114 can be used for imaging, such as portal imaging.

[0054] In the illustrative example, one or more of the platform 116, the treatment output 104, or the gantry 106 can be automatically positioned, and the treatment output 104 can establish the treatment beam 108 according to the specified dose for a particular treatment delivery instance. A series of treatment deliveries can be specified according to the radiotherapy treatment plan, for example, using one or more different orientations or positions of the gantry 106, the platform 116, or the treatment output 104. The treatment deliveries can occur sequentially, but can intersect at a desired treatment site on or in the patient, such as at the isocenter 110. The prescribed cumulative dose of radiation therapy can thereby be delivered to the treatment site while reducing or avoiding damage to tissue near the treatment site.

[0055] As mentioned with respect to other examples herein, the radiation therapy system 102 may include or may be coupled to an imaging acquisition system, for example to provide one or more of magnetic resonance (MR) imaging, X-ray imaging, for example, the radiation therapy system 102 may include computed tomography (CT) imaging, or ultrasound imaging. In an example, MR imaging information or other imaging information may be used to generate imaging information or visualization equivalent to CT imaging without the need for actual CT imaging. Such imaging may be referred to as "pseudo-CT" imaging.

[0056] Figure 1B Another example of a radiation therapy system 102 (e.g., a Leksell Gamma Knife manufactured by Elekta, AB, Stockholm, Sweden) is generally shown in accordance with some embodiments of the present disclosure. Figure 1B As shown, during a radiotherapy treatment session, a patient 152 can wear a coordinate frame 154 to stabilize a portion of the patient's anatomy (e.g., the head) for surgery or radiotherapy. The coordinate frame 154 and the patient positioning system 156 can establish a spatial coordinate system that can be used when imaging the patient or during radiosurgery.

[0057] The radiation therapy system 102 may include a protective housing 164 that surrounds a plurality of radiation sources 162. The radiation sources 162 may generate a plurality of radiation beams (e.g., beamlets) through beam channels 166. The plurality of radiation beams may be configured to focus on an isocenter 110 from different directions. Although each individual radiation beam may have a relatively low intensity, the isocenter 110 may receive a relatively high level of radiation when multiple doses from different radiation beams are accumulated at the isocenter 110. In some embodiments, the isocenter 110 may correspond to a target in surgery or treatment, such as a tumor.

[0058] Figure 2An example of a system is generally shown that can include a combined radiation therapy system 102 and an imaging system, which can include, for example, a computed tomography (CT) imaging system. The CT imaging system can include an imaging X-ray source 118, for example, providing X-ray energy in the kiloelectronvolt (keV) energy range. The imaging X-ray source 118 provides one or more fan or cone beams 120 that are directed to an imaging detector 122, such as a flat panel detector. The radiation therapy system 102 can be similar to the one described with respect to Figure 1A The system 102 depicted, for example, includes a radiation therapy output 104, a gantry 106, a platform 116, and another flat panel detector 114. Figure 1A , Figure 1B and Figure 3 As shown in the example of , the radiation therapy system 102 can be coupled to or can include a high energy accelerator configured to provide a therapeutic radiation beam. The X-ray source 118 can provide a relatively low energy X-ray diagnostic beam for imaging.

[0059] exist Figure 2 In the illustrative example of FIG. 1 , the radiotherapy output 104 and the x-ray source 118 can be mounted on the same rotating gantry 106, rotated 90 degrees apart from each other. In another example, two or more x-ray sources can be mounted along the circumference of the gantry 106, such as each x-ray source having its own detector arrangement to provide diagnostic imaging at multiple angles simultaneously. Similarly, multiple radiotherapy outputs 104 can be provided.

[0060] Figure 3 A partial cross-sectional view of an example of a system is generally shown, which system may include a combined radiotherapy system 102 and an imaging system, such as may include a magnetic resonance (MR) imaging system 130. The MR imaging system 130 may be arranged to define an "aperture" about an axis ("A"), and the radiotherapy system may include a radiotherapy output 104, such as to provide a radiotherapy beam 108 directed to an isocenter 110 within the aperture along the axis A. The radiotherapy output 104 may include a collimator 124, such as to control, shape, or modulate the radiotherapy beam 108 to direct the beam 108 to a treatment site aligned with a desired target site in a patient. The patient may be supported by a platform, such as a platform positionable in one or more of an axial direction A, a lateral direction L, or a transverse direction T. One or more portions of the radiotherapy system 102 may be mounted on a gantry 106, such as to rotate the radiotherapy output 104 about the axis A.

[0061] Figure 1A , Figure 1B , Figure 2 and Figure 3Examples are generally shown including configurations in which the treatment output can rotate about a central axis (e.g., axis "A"). Other radiation treatment output configurations can be used. For example, the radiation treatment output can be mounted on a robotic arm or manipulator, such as with multiple degrees of freedom. In yet another example, the treatment output can be fixed, such as located in an area laterally separated from the patient, and a platform supporting the patient can be used to align the radiation treatment isocenter with a designated target site in the patient.

[0062] Figure 4 An example of a multi-leaf collimator (MLC) 132 is generally shown, such as may be used to one or more of shape, direct, or modulate the intensity of a radiation therapy beam. Figure 4 In the embodiment of the present invention, the leaves 132A to 132J can be automatically positioned to define an aperture that approximates a cross-section or projection of the tumor 140. According to the radiation treatment plan, the leaves 132A to 132J can be made of a material that is designated to attenuate or block the radiation beam in areas other than the aperture. For example, the leaves 132A to 132J can include a metal plate, such as tungsten, with the long axis of the plate oriented perpendicular to the beam direction and the ends oriented parallel to the beam direction (such as Figure 4 The "state" of the MLC 132 may be adaptively adjusted during the course of radiation therapy, for example to establish a treatment beam that better approximates the shape or location of a tumor 140 or other target site, as compared to using a static collimator configuration or as compared to using an MLC 132 configuration determined specifically using "offline" treatment planning techniques. Radiation therapy techniques that include the use of an MLC 132 to produce a prescribed radiation dose distribution to a tumor or a specific region within a tumor may be referred to as Intensity Modulated Radiation Therapy (IMRT).

[0063] Figure 5 An exemplary radiation therapy system 500 is shown that is adapted to perform radiation therapy planning processing operations using one or more of the methods discussed herein. These radiation therapy planning processing operations are performed to enable the radiation therapy system 500 to provide radiation therapy to a patient based on captured medical imaging data and treatment dose calculations or specific aspects of radiation therapy machine configuration parameters.

[0064] The radiotherapy system 500 includes a radiotherapy processing computing system 510, which carries a treatment processing logic 520. Specifically, the treatment processing logic 520 implements Figures 12 to 19Various techniques discussed to detect object movement and update treatment plans during real-time delivery of radiotherapy phases. However, it should be understood that many variations and use cases of the following treatment processing logic 520 can be provided, including in data validation, visualization, and other medical assessment and diagnostic settings. The radiotherapy treatment computing system 510 can be connected to a network (not shown), and such a network can be connected to the Internet. For example, the network can connect the radiotherapy treatment computing system 510 to one or more medical information sources (e.g., a radiology information system (RIS), a medical record system (e.g., an electronic medical record (EMR) / electronic health record (EHR) system), an oncology information system (OIS)), one or more image data sources 550, an image acquisition device 570 (e.g., an imaging modality), a treatment device 580 (e.g., a radiotherapy device, also referred to herein as a radiotherapy device) and a treatment data source 560.

[0065] As an example, the radiotherapy treatment computing system 510 can be configured to receive a first imaging slice of an object depicted in a 3D pre-treatment volumetric image at a first time point during a given radiation phase. The first imaging slice is defined along a first plane (e.g., a sagittal plane). At a first time point during a given radiation phase, a second imaging slice of an object obtained at a second time point prior to the first time point during the given radiation phase is accessed. The second imaging slice is defined along a second plane (e.g., a coronal plane). The system determines the movement of the object at the first time point along a first direction and a second direction (e.g., along the sagittal plane and left / right sides) based on registering the first imaging slice with the volumetric image. The movement of the object at the first time point along a third direction (e.g., a coronal plane) is determined based on the second imaging slice.

[0066] Specifically, the radiotherapy treatment computing system 510 can determine the movement of the object along three directions (e.g., sagittal plane, coronal plane, and left / right plane) during the radiotherapy treatment stage according to the second technology. It achieves this by registering the image (e.g., movie image) received along one plane (e.g., sagittal plane) at a specific point in the treatment stage with the 3D volume image before treatment. The result provides the movement of the object along the sagittal plane and the left / right plane. At the same specific point, the radiotherapy treatment computing system 510 retrieves the previously obtained image (e.g., movie image) along another plane (e.g., coronal plane). The radiotherapy treatment computing system 510 uses the position of the object in the previously obtained image slice to determine the movement of the object along the coronal plane. Therefore, the movement of the object along three directions at a given time point in the treatment stage can be determined (e.g., the 3D movement of the object can be determined). For example, the first direction corresponds to the front / back direction, the second direction corresponds to the up / down direction, and the third direction corresponds to the left / right direction. The object may include at least one of the anatomical part, pixel, or any defined image part of the patient. The first imaging slice and the second imaging slice are planar or orthogonal, but may be any two imaging slices along any other parallel or perpendicular directions.

[0067] In some cases, to determine movement of the object along a current planar orientation (e.g., a direction corresponding to a currently acquired image rather than a previously acquired image), the radiation therapy treatment computing system 510 extracts a first two-dimensional (2D) slice along a first plane from the volumetric image. The radiation therapy treatment computing system 510 spatially registers the first imaging slice with the first 2D slice (e.g., by moving a portion of the object in the first imaging slice to register the object in the first imaging slice with a portion of the object in the first 2D slice).

[0068] In some embodiments, the radiation therapy treatment computing system 510 registers the second imaging slice (corresponding to an earlier time point in the treatment phase) with the pre-treatment image to determine the movement along the third direction. In such a case, the radiation therapy treatment computing system 510 defines the first imaging slice and the second imaging slice (defined along the respective first and second planes) as moving images, and extracts 2D slices from the volumetric pre-treatment image as fixed images to perform 2D / two-dimensional image registration. The radiation therapy treatment computing system 510 then determines the movement along the three directions based on the first image slice and the second image slice registered with the 3D pre-treatment volumetric image.

[0069] In some embodiments, the radiotherapy treatment calculation system 510 calculates the in-plane displacement of the object along the first direction and the second direction based on the first imaging slice and the first 2D slice. The radiotherapy treatment calculation system 510 calculates the out-of-plane displacement value along the third direction based on the second imaging slice. For example, the radiotherapy treatment calculation system 510 calculates the in-plane displacement value of the object along the sagittal direction and the left / right direction based on the first imaging slice defined along the sagittal direction and the first 2D slice extracted from the pre-treatment volumetric image. The radiotherapy treatment calculation system 510 calculates the out-of-plane displacement value along the coronal direction based on the second imaging slice defined along the coronal direction.

[0070] In some cases, the radiotherapy treatment computing system 510 predicts the difference between the second imaging slice defined along the second plane at the first time point and the expected imaging slice defined along the second plane. For example, the radiotherapy treatment computing system 510 applies machine learning techniques to the second imaging slice (obtained at the previous time point) to predict or estimate the second imaging slice at the current time point in the radiotherapy phase. The predicted or estimated second imaging slice defined along the second plane can then be used in conjunction with the first imaging slice defined along the first plane to determine the movement of the object in three directions at the current time point.

[0071] In some embodiments, the radiation therapy treatment computing system 510 determines a metric indicative of an alignment between a portion of an object from a first imaging slice and a portion of an object from a first 2D slice. The radiation therapy treatment computing system 510 adjusts a position of a second imaging slice to improve the alignment between a portion of an object from the first imaging slice and a portion of an object from the first 2D slice. In some cases, the metric is determined by masking a portion of the first imaging slice and a slice of the volumetric image. In some cases, the metric is determined by determining a similarity between intersection lines between a portion of an object from the first imaging slice and a portion of an object from a slice of the volumetric image.

[0072] In some embodiments, the radiotherapy treatment computing system 510 determines movement during a radiotherapy treatment phase according to a third technique. For example, the radiotherapy treatment computing system 510 receives a first imaging slice including an object at a first time point during a given radiation phase. The first imaging slice can correspond to a first plane (e.g., a sagittal plane). The radiotherapy treatment computing system 510 accesses a composite imaging slice corresponding to the first plane at a first time point during a given radiation phase. The composite imaging slice is generated using multiple imaging slices acquired before the first time point. The radiotherapy treatment computing system 510 spatially aligns the first imaging slice with the composite imaging slice to determine the movement of the object. The radiotherapy treatment computing system 510 generates an updated treatment plan to control the delivery of the treatment beam based on the determined movement.

[0073] In some cases, the radiation therapy treatment computing system 510 registers the first imaging slice with the composite imaging slice by limiting the pixel mask considered in the registration to one or both of the first imaging slice and the composite imaging slice. Prior to registering the first imaging slice to the composite imaging slice, the composite imaging slice is registered to the two-dimensional slice extracted from the volumetric image. In some cases, the composite image registered to the two-dimensional slice extracted from the volumetric image is manually adjusted by user input to specify or change the location or portion of the composite image registered relative to the two-dimensional slice.

[0074] In some embodiments, the composite image is a first composite image. At a second time point during a given radiation session, the radiation therapy treatment computing system 510 receives a second imaging slice corresponding to a second plane (e.g., a coronal plane). The radiation therapy treatment computing system 510 accesses a second composite imaging slice corresponding to the second plane at a second time point during a given radiation session. The second composite imaging slice is generated using a plurality of additional imaging slices acquired before the second time point. The radiation therapy treatment computing system 510 spatially registers the second imaging slice with the second composite imaging slice, including moving a portion of an object in the second imaging slice to register the object in the second imaging slice with a portion of an object in the second composite imaging slice.

[0075] In some embodiments, a composite image is generated by averaging a selected subset of imaging slices. For example, multiple imaging slices are obtained before activating the radiotherapy beam during a given radiation phase. One of the given slices corresponding to the reference breathing phase is selected as the target image. The radiotherapy treatment computing system 510 deformably aligns the remaining set of multiple imaging slices to the selected target image. The radiotherapy treatment computing system 510 averages the multiple imaging slices that are deformably registered and determines the average offset of each imaging slice in the multiple imaging slices. The radiotherapy treatment computing system 510 resamples the imaging slices based on the average offset to select a subset of the resampled imaging slices based on the distance of each resampled imaging slice to the target image (e.g., the selected imaging slice). For example, the radiotherapy treatment computing system 510 selects those imaging slices whose distance between the slice and the target slice is less than a specified threshold. A composite image is then generated by averaging the selected subset of the resampled multiple imaging slices.

[0076] In some embodiments, the radiotherapy treatment computing system 510 determines movement during a radiotherapy treatment phase according to a fourth technique. For example, the radiotherapy treatment computing system 510 receives a first imaging slice corresponding to a first plane (e.g., corresponding to a sagittal plane or a 2D movie image defined along a sagittal plane) at a first time point in a given radiation phase. The radiotherapy treatment computing system 510 encodes the first imaging slice into a lower dimensional representation (e.g., by converting principal component analysis (PCA) components of a deformable vector field (DVF) of the first imaging slice to estimate PCA components of a second imaging slice). In an example, encoding the slice into a lower dimensional representation includes reducing the amount of data used to represent a given slice by converting a standard used to represent the slice from one standard to another. The radiotherapy treatment computing system 510 applies a trained machine learning model (e.g., a support vector machine or a random forest machine or a neural network) to the encoded first imaging slice to estimate an encoded version of a second imaging slice corresponding to a second plane (e.g., a coronal plane) at the first time point, thereby providing a pair of imaging slices for the first time point. The radiotherapy treatment computing system 510 simultaneously spatially registers the pair of imaging slices to a volumetric image received before a given radiation phase (e.g., a 3D pre-treatment volume of a patient), the volumetric image including a time-varying object to calculate the displacement of the object. The radiotherapy treatment computing system 510 generates an updated treatment plan to control the delivery of the treatment beam based on the calculated displacement of the object. In some cases, before registering the images, the radiotherapy treatment computing system 510 decodes the estimated encoded version of the second imaging slice to obtain the second imaging slice at the first time point. For example, the radiotherapy treatment computing system 510 converts or upgrades the data used to represent the imaging slice from one standard to another standard. In order to encode and decode the slices, any suitable compression and decompression algorithm can be used in accordance with the disclosed technology.

[0077] In some cases, a pair of imaging slices includes a pair of stereoscopic imaging slices such that the first plane is orthogonal to the second plane or the first plane is tilted relative to the second plane (e.g., non-orthogonal). In some cases, in order to train the machine learning model to perform predictions in the fourth technique, the radiotherapy treatment computing system 510 receives a first training imaging slice sequence corresponding to the first plane before a given radiation stage. The radiotherapy treatment computing system 510 also receives a second training imaging slice sequence corresponding to the second plane before a given radiation stage. The second training imaging slice sequence can be received simultaneously or alternately with the first imaging slice sequence. The radiotherapy treatment computing system 510 encodes the first sequence and the second sequence into a lower dimensional representation (e.g., converting the data used to represent the first sequence and the second sequence from one standard to another standard to reduce the amount of data used to represent the sequence). In some implementations, the radiotherapy treatment computing system 510 interpolates the first training image slice of the first training imaging slice sequence corresponding to the first training time point to generate a first interpolated training imaging slice corresponding to the second training time point when the second training image slice of the second training imaging slice sequence is received at the second training time point. Specifically, the radiation therapy treatment computing system 510 interpolates or generates an expected version of the first training image slice for the next adjacent time point corresponding to the time point at which one of the training imaging slices of the second plane is received. The radiation therapy treatment computing system 510 trains the machine learning model based on the interpolated image to predict the first image corresponding to the first plane based on the second image corresponding to the second plane. The radiation therapy treatment computing system 510 continuously or periodically trains the machine learning model during a given radiation phase as new imaging slices are captured and received.

[0078] The radiotherapy treatment computing system 510 may include a processing circuit system 512, a memory 514, a storage device 516, and other hardware and software operable features such as a user interface 542, a communication interface (not shown), etc. The storage device 516 may store transient or non-transient computer executable instructions, such as an operating system, a radiotherapy treatment plan (e.g., training data, a treatment plan strategy, a patient movement model, a patient deformation model, beam delivery segment information, 5D and / or 2D image information of the patient, and device adjustment parameters, etc.), a software program (e.g., image processing software, image or anatomical visualization software, etc.), and any other computer executable instructions to be executed by the processing circuit system 512.

[0079] In an example, the processing circuit system 512 may include a processing device, for example, one or more general-purpose processing devices such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), etc. More specifically, the processing circuit system 512 may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that implements other instruction sets, or a processor that implements a combination of instruction sets. The processing circuit system 512 may also be implemented by one or more special-purpose processing devices such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.

[0080] As will be appreciated by those skilled in the art, in some examples, processing circuit system 512 may be a special purpose processor rather than a general purpose processor. Processing circuit system 512 may include one or more known processing devices, such as those from Intel. TM Pentium TM 、Core TM , Xeon TM or series of microprocessors from AMD TM Made by Turion TM , Athlon TM 、Sempron TM 、Opteron TM FX TM 、Phenom TM series of microprocessors, or any of a variety of processors manufactured by Sun Microsystems. Processing circuit system 512 may also include a graphics processing unit, such as a graphics processing unit from Nvidia TM Manufactured series of GPUs from Intel TM GMA, Iris TM series of GPUs, or from AMD TM Made by Radeon TM The processing circuit system 512 may also include an accelerated processing unit, such as an Intel TM Xeon Phi manufactured TMseries. The disclosed embodiments are not limited to any type of processor that is otherwise configured to meet the computational requirements of identifying, analyzing, maintaining, generating and / or providing large amounts of data or manipulating such data to perform the methods disclosed herein. In addition, the term "processor" may include more than one physical (circuitry-based) or software-based processor, for example, a multi-core design or multiple processors each having a multi-core design. The processing circuit system 512 can execute a sequence of transient or non-transient computer program instructions stored in the memory 514 and accessed from the storage device 516 to perform various operations, processes, methods that will be described in more detail below. It should be understood that any component in the radiotherapy system 500 can be implemented separately and operated as a stand-alone device, and can be coupled to any other component in the radiotherapy system 500 to perform the techniques described in the present disclosure.

[0081] The memory 514 may include read-only memory (ROM), phase-change random access memory (PRAM), static random access memory (SRAM), flash memory, random access memory (RAM), dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), electrically erasable programmable read-only memory (EEPROM), static memory (e.g., flash memory, flash disk, static random access memory), and other types of random access memory, cache, registers, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage device, cassette tape, other magnetic storage device, or any other non-transitory medium that can be used to store information, including images, training data, ML technique parameters, device adaptation functions, data, or transient or non-transitory computer-readable instructions (stored in any format) that can be accessed by the processing circuit system 512 or any other type of computer device. For example, computer program instructions can be accessed by the processing circuit system 512, read from ROM or any other suitable memory location, and loaded into RAM for execution by the processing circuit system 512.

[0082] The storage device 516 may constitute a drive unit including a transitory or non-transitory machine-readable medium on which is stored one or more sets of transitory or non-transitory instructions and data structures (e.g., software) embodying or used by any one or more of the methods or functions described herein (including, in various examples, the treatment processing logic 520 and the user interface 542). The instructions may also reside completely or at least partially within the memory 514 and / or within the processing circuit system 512 during execution by the radiation therapy treatment computing system 510, wherein the memory 514 and the processing circuit system 512 also constitute transitory or non-transitory machine-readable media.

[0083] The memory 514 and the storage device 516 may constitute a non-transitory computer-readable medium. For example, the memory 514 and the storage device 516 may store or load transient or non-transitory instructions for one or more software applications on a computer-readable medium. Software applications stored or loaded using the memory 514 and the storage device 516 may include, for example, operating systems for ordinary computer systems and for software-controlled devices. The radiotherapy treatment computing system 510 may also operate various software programs, including software code for implementing the treatment treatment logic 520 and the user interface 542. In addition, the memory 514 and the storage device 516 may store or load the entire software application, a portion of the software application, or code or data associated with the software application that can be executed by the processing circuit system 512. In another example, the memory 514 and the storage device 516 may store, load and manipulate one or more radiotherapy treatment plans, imaging data, segmentation data, treatment visualization, histograms or measurement results, etc. It is contemplated that the software program may be stored not only on storage device 516 and memory 514, but also on removable computer media such as a hard drive, a computer disk, a CD-ROM, a DVD, a Blu-ray DVD, a USB flash drive, an SD card, a memory stick, or any other suitable media; such software programs may also be transmitted or received over a network.

[0084] Although not depicted, the radiotherapy treatment computing system 510 may include a communication interface, a network interface card, and a communication circuit system. Example communication interfaces may include, for example, a network adapter, a cable connector, a serial connector, a USB connector, a parallel connector, a high-speed data transmission adapter (e.g., optical fiber, USB 5.0, thunderbolt interface, etc.), a wireless network adapter (e.g., an IEEE 802.11 / Wi-Fi adapter), a telecommunications adapter (e.g., communicating with 5G / LTE, 4G / LTE and 5G, networks, etc.), etc. Such a communication interface may include one or more digital and / or analog communication devices that allow the machine to communicate with other machines and devices, such as remotely located components, via a network. The network may provide the functionality of a local area network (LAN), a wireless network, a cloud computing environment (e.g., software as a service, platform as a service, infrastructure as a service, etc.), a client-server, a wide area network (WAN), etc. For example, the network may be a LAN or WAN that may include other systems (including additional image processing computing systems or image-based components associated with medical imaging or radiotherapy operations).

[0085] In an example, the radiation therapy treatment computing system 510 may obtain image data 552 from an image data source 550 (e.g., a CT image, a PET image, and / or an MR image) for hosting on the storage device 516 and the memory 514. In yet another example, the software program may substitute a function of the patient image, such as a signed distance function or a processed version of the image that emphasizes certain aspects of the image information.

[0086] In an example, the radiation therapy treatment computing system 510 can obtain image data 552 from or transmit image data 552 to the image data source 550. In another example, the treatment data source 560 receives or updates the planning data 562 as a result of radiation therapy device parameter adjustment or segment adaptation or movement of a detected object (e.g., a target site).

[0087] The processing circuit system 512 may be communicatively coupled to the memory 514 and the storage device 516, and the processing circuit system 512 may be configured to execute computer executable instructions stored thereon from the memory 514 or the storage device 516. The processing circuit system 512 may execute the instructions to cause a medical image from the image data 552 to be received or obtained in the memory 514 and processed using the treatment processing logic 520.

[0088] In addition, the processing circuit system 512 can utilize a software program to generate intermediate data, such as updated parameters to be used by, for example, a neural network model, a deep learning neural network model, a regression technique including a support vector machine or a random forest, a machine learning model, or other aspects related to generating device parameter adjustments or segment adaptation, as discussed herein. In addition, such a software program can utilize the treatment processing logic 520 to generate updated radiation therapy parameters to provide to the treatment data source 560 to modify the dose delivered to the target within a given segment using the techniques further discussed herein and / or for presentation on the output device 546. The processing circuit system 512 can then transmit the updated radiation therapy parameters to the treatment device 580 via the communication interface and network, where the updated parameters will be used to treat the patient with radiation via the treatment device 580. For each segment or portion of a given treatment fraction, radiotherapy parameters (also called control points) may include MLC position and settings, gantry angle, radiation dose amount (e.g., amount of monitor units (MUs)), radiotherapy beam direction, radiation beam size, arc position, beam on and off durations, machine parameters, gantry speed, MRI pulse sequence, any combination thereof, and the like.

[0089] In an example, the image data 552 may include one or more MR images (e.g., 2D MRI, 5D MRI, 2D flow MRI, 4D MRI, 4D volume MRI, 4D cine MRI, etc.), functional MRI images (e.g., fMRI, DCE-MRI, diffusion MRI), computed tomography (CT) images (e.g., 2D CT, 2D cone beam CT, 5D CT, 5D CBCT, 4D CT, 4DCBCT), ultrasound images (e.g., 2D ultrasound, 5D ultrasound, 4D ultrasound), positron emission tomography (PET) images, X-ray images, fluoroscopy images, radiotherapy portal images, single emission computed tomography (SPECT) images, computer-generated synthetic images (e.g., pseudo-CT images), radio beacons, laser scans of a patient's surface, etc. In addition, the image data 552 may also include or be associated with medical image processing data, such as training images, ground truth images, contour images, and dose images. In other examples, equivalent representations of anatomical regions may be represented in non-image formats (eg, coordinates, maps, etc.).

[0090] In an example, the image data 552 may be received from an image acquisition device 570 and stored in one or more of the image data sources 550 (e.g., a picture archiving and communication system (PACS), a vendor neutral archive (VNA), a medical record or information system, a data warehouse, etc.). Thus, the image acquisition device 570 may include an MRI imaging device, a CT imaging device, a PET imaging device, an ultrasound imaging device, a fluoroscopy device, a SPECT imaging device, an integrated linear accelerator and MRI imaging device, a CBCT imaging device, or other medical imaging device for obtaining medical images of a patient. The image data 552 may be received and stored in any type of data or any type of format (e.g., in a Digital Imaging and Communications in Medicine (DICOM) format) that may be used by the image acquisition device 570 and the radiation therapy treatment computing system 510 to perform operations consistent with the disclosed embodiments. Furthermore, in some examples, the models discussed herein may be trained to process the original image data format or a derivative thereof.

[0091] In an example, the image acquisition device 570 can be integrated with the treatment device 580 into a single device (e.g., an MRI device combined with a linear accelerator, also referred to as "MRI-Linac"). Such an MRI-Linac can be used, for example, to determine the location of a target in a patient's body, thereby accurately guiding the radiation beam to the predetermined target according to the radiotherapy treatment plan. For example, a radiotherapy treatment plan can provide information about a specific radiation dose to be applied to each patient. The radiotherapy treatment plan can also include other radiotherapy information and radiotherapy device parameters, such as beam angles, dose-volume-histogram information, the number of radiation beams to be used during treatment, the dose of each beam, etc. MRI-Linac can be used to calculate, generate and / or update a patient deformation model to deform the image portion of a 5D or 2D image of the patient corresponding to a given beam delivery segment. MRI-Linac can be used to provide real-time patient images (or a subset of patient images) and machine settings or parameters at various increments / intervals of the treatment segmentation to continuously or periodically calculate the dose of the increment / interval, and determine the real-time dose accumulation based on such calculated dose.

[0092] The radiotherapy treatment computing system 510 can communicate with an external database over a network to send / receive a variety of different types of data related to image processing and radiotherapy operations. For example, the external database may include machine data (including device constraints) that provide information associated with the treatment device 580, the image acquisition device 570, or other machines related to radiotherapy or medical procedures. The machine data information may include radiation beam size, arc position, beam on and off duration, machine parameters, segments, MLC configurations, gantry speeds, MRI pulse sequences, etc. The external database may be a storage device and may be equipped with an appropriate database management software program. In addition, such a database or data source may include multiple devices or systems located in a central or distributed manner.

[0093] The radiation therapy treatment computing system 510 can collect and obtain data and communicate with other systems via a network using one or more communication interfaces that are communicatively coupled to the processing circuit system 512 and the memory 514. For example, the communication interface can provide a communication connection between the radiation therapy treatment computing system 510 and the radiation therapy system components (e.g., allowing data to be exchanged with external devices). For example, in some examples, the communication interface can have appropriate interface circuit systems from the output device 546 or the input device 548 to connect to the user interface 542, which can be a hardware keyboard, keypad, or touch screen, through which the user can enter information into the radiation therapy system.

[0094] As an example, the output device 546 may include a display device that outputs a representation of the user interface 542 and one or more aspects of a medical image, visualization or representation, a treatment plan, and the status of training, generation, validation, or implementation of such a plan. The output device 546 may include one or more display screens that display to the user a medical image, interface information, treatment plan parameters (e.g., contours, doses, beam angles, labels, maps, etc.), a treatment plan, for a given treatment segment, portions of the image in which targets are identified and deformed, positioning targets, and / or tracking targets, or any relevant information. The output device 546 may provide the user with a visualization of the movement of a target object or treatment site during the delivery of a radiation therapy treatment segment.

[0095] The input device 548 connected to the user interface 542 can be a keyboard, keypad, touch screen, or any type of device that a user can use to input information to the radiation therapy system 500. Alternatively, the output device 546, input device 548, and features of the user interface 542 can be integrated into a single device such as a smartphone or tablet computer (e.g., Apple Lenovo Samsung etc.).

[0096] In addition, any and all components of the radiotherapy system 500 can be implemented as virtual machines (e.g., via a virtualization platform such as VMWare, Hyper-V, etc.) or stand-alone devices. For example, a virtual machine can be software that acts as hardware. Thus, a virtual machine can include at least one or more virtual processors, one or more virtual memories, and one or more virtual communication interfaces that together act as hardware. For example, the radiotherapy treatment computing system 510, the image data source 550, or similar components can be implemented as virtual machines or within a cloud-based virtualization environment.

[0097] The image acquisition device 570 can be configured to acquire one or more images of the patient's anatomical structure of the region of interest (e.g., a target organ, a target tumor, or both). Each image (usually a 2D image or slice) can include one or more parameters (e.g., 2D slice thickness, orientation, and position, etc.). In an example, the image acquisition device 570 can acquire 2D slices at any orientation. For example, the orientation of the 2D slice can include a sagittal orientation, a coronal orientation, or an axial orientation. The processing circuit system 512 can adjust one or more parameters, such as the thickness and / or orientation of the 2D slice, to include the target organ and / or the target tumor. In an example, the 2D slice can be determined based on information such as a 5D CBCT or CT or MRI volume. Such a 2D slice can be acquired by the image acquisition device 570 in "near real time" while the patient is receiving radiotherapy treatment, such as when using a treatment device 580 (where "near real time" means acquiring data without (or with minimal) delay between image acquisition and treatment, as known in the art). In an example, a 5D volume representation of the region of interest can be generated using a stack of one or more 2D slices.

[0098] Figure 6 Generally, a view of an object or target site 616, such as a tumor region to be treated using radiation therapy, is shown. Information indicating the target site 616 may be obtained using volumetric imaging techniques. For illustration purposes only, Figure 6 The representation in is shown with rectangular faces, and the actual target site 616 may include curved or irregular surfaces. The region of interest 612 may be imaged, for example, using one or more of MR imaging, CT imaging, pseudo-CT image visualization, portal imaging, ultrasound imaging, or other techniques. The target site 616 may be identified, for example, using automatic, manual, or semi-automatic image segmentation techniques. For example, a designated label may be assigned to a voxel (e.g., voxel 620) to identify a site or group of voxels that includes a radiation therapy target. As an illustrative example, the contrast value of voxel 620 may be compared to adjacent voxels, and the contrast information may be used, for example, to identify the edges of a tumor or other region. Other regions, such as the location of nearby organs or other features, may also be identified.

[0099] Various features of the target site may be determined, such as one or more edge locations (e.g., using edge detection or edge recognition techniques). For example, the location of the target site 616 may be identified by determining the spatial centroid 608 of the target site 616. The use of the centroid 608 as an extracted feature is illustrative, and other features may be used to track the motion of the target site 616, such as the location of a manually identified or automatically determined point, surface, or edge of the target site 616. In yet another example, an implantable or external seed fiducial may be used, such as to provide a marker in the acquired imaging information.

[0100] The target site 616 and one or more corresponding features such as the centroid 608 may be identified during treatment planning imaging, such as between segments (e.g., between radiation therapy delivery phases) or just before starting a radiation therapy delivery phase. The volumetric imaging information including the target site 616 may be referred to as a "reference image". As described above, there may be challenges because image acquisition and processing delays may hinder the "real-time" acquisition of volumetric imaging information during radiation therapy. Therefore, intra-segment image acquisition (e.g., during radiation therapy delivery in a radiation therapy phase or between radiation therapy delivery in a radiation therapy phase) may include more rapidly acquiring imaging slices (including one-dimensional contours, two-dimensional slices, movie images, or three-dimensional volumes including sub-volumes or sub-regions of earlier imaged volumetric regions). Information indicating a portion of the target site rapidly acquired from one or more imaging slices may be compared with information obtained from an earlier acquired volumetric reference image to update the treatment plan, such as to adjust the treatment due to movement of the target site. The imaging modality used to acquire or generate the volumetric reference image need not be the same as the imaging modality used for intra-segment imaging.

[0101] In another example, the target site 616 need not be segmented or otherwise identified in the volumetric reference image. The use of the phrase "target site" merely describes a target within a region of interest, such as a portion or all of a tumor to be treated or other anatomical structures such as organs near a tumor. In addition to or in lieu of segmentation of the target site 616, various techniques as described herein may be performed, such as including, for example, using voxel values ​​including contrast values ​​or grayscale values ​​to spatially register a portion of the volumetric reference image (e.g., a specified region of interest) or all of the volumetric reference image with other imaging information. For example, such spatial registration may include three-dimensional (e.g., volumetric) registration (e.g., 3D-2D registration) or other techniques with one or more two-dimensional imaging slices.

[0102] Fig. 7A and Figure 7B An imaging acquisition plane 712A of a target site 716 is shown (at Fig. 7A) and a corresponding acquired imaging slice 714A (in Figure 7B As an illustrative example, Fig. 7A The sagittal orientation of the image acquisition plane 712A and the corresponding slice 714A may be indicated. An imaging "slice" may include two-dimensional pixel imaging information or three-dimensional imaging information, for example, having Figure 7B A small finite thickness is illustratively shown. A portion 718A of the target site in the imaged slice 714A may be identified, for example, again using a segmentation technique such as using discrete dynamic contours, snakes, or level sets, or again using a registration-based technique.

[0103] A series of imaging slices may be obtained, including, for example, Figure 7C , Fig.7D , Fig. 7E and Figure 7F Different imaging acquisition plane orientations are shown. For example, Figure 7C and Fig.7D An imaging acquisition plane 712B is shown intersecting a target site 716 along line 706B (at Figure 7C ) and a corresponding acquired imaging slice 714B (in Fig.7D The corresponding acquired imaging slice 714B corresponds to the second imaging plane orientation, for example, orthogonal to the above Fig. 7A and Figure 7B The first imaging plane orientation is mentioned. As an illustration, Figure 7C and Fig.7D May correspond to the coronal plane. Fig. 7E and Figure 7F An imaging acquisition plane 712C is shown intersecting a target site 716 along line 706C (at Fig. 7E ) and a corresponding acquired imaging slice 714C (in Figure 7F The acquired imaging slice 714C corresponds to a third imaging plane orientation, for example, orthogonal to the above Fig. 7A , Figure 7B , Figure 7C and Fig.7D One or more of the first imaging plane orientation and the second imaging plane orientation. As an illustration, Fig. 7E and Figure 7F May correspond to the axial imaging plane. Fig. 7A , Figure 7B , Figure 7C , Fig.7D , Fig. 7E and Figure 7FThe imaging slices are shown as two-dimensional planes that are orthogonal to each other. However, as mentioned with respect to other examples herein, the slices need not be planar nor orthogonal. Each acquired imaging slice may be processed, including processing one or more slices in parallel with the acquisition of the imaging slices. For example, such processing may include sampling information from a volume reference image, such as a plane or region corresponding to the acquired imaging slice.

[0104] In one approach, features extracted from one or more of the imaging slices 714A, 714B, or 714C can be compared to imaging information extracted from a volumetric reference image. For example, a segmented portion of a target site 616 from a volumetric reference image or a region encapsulating the target site and surrounding tissue can be spatially registered with a corresponding imaging slice 714A, 714B, or 714C to determine if the target site has shifted. However, comparing features extracted from a single two-dimensional imaging slice with corresponding features from a volumetric reference image can present challenges. For example, Fig. 8A Generally shown are a series of imaging acquisition planes 812A, 812B, and 812C, such as may be obtained as a target site 816 moves from one imaging acquisition instance to another. i-2 At time t, the line 806A where the imaging acquisition plane intersects the target site 816 is approximately centered, and at time t i-1 At time t i At time t i ,t i-1 ,t i-2 The imaging slices obtained at ,... can usually be marked as S i , S i-1 , S i-2 …….

[0105] If the motion of the target site is out-of-plane, the center of mass or other characteristic locations may not shift significantly between images even if the target site has moved significantly, or the target site may appear deformed, making interpretation of the motion difficult. However, such out-of-plane motion may be appropriately tracked, for example, by one or more of the following: changing the orientation of successively acquired imaging slices (e.g., Figure 8C and elsewhere), or changing or compensating for the non-planar shape of the imaging acquisition region so that the imaging slice is not represented as a completely planar slice in a three-dimensional sense (e.g., as Figure 8B For example, Figure 8B A series of imaging slices 813A, 813B, and 813C are generally shown, which may include curved shapes, for example. In general, even when referring to image acquisition "planes", the techniques described herein may be used with other slice geometries, such as Figure 8B As an illustrative example, MR imaging information can be obtained as planar slices, and transformations can be applied to the planar slices to obtain surfaces that are curved in a three-dimensional sense, such as to help capture or correct distortion of the entire imaging field, such as to compensate for inhomogeneities in the established magnetic field. As another illustrative example, Figure 8C A series of imaging acquisition planes 815A, 815B, and 815C are generally shown, which may include different orientations that need not be orthogonal to each other, for example.

[0106] As discussed above, if imaging slices are acquired time sequentially, the target site is likely to shift between consecutive image acquisitions. Fig. 9A A series of two imaging slices is generally shown, for example including a target site displaced between a first imaging slice and a second imaging slice. The example of two sequentially acquired imaging slices is illustrative. The techniques described herein are also generally applicable to sequences of more than two imaging slices (or even a single imaging slice analyzed with respect to a portion or all of the reference volume imaging information).

[0107] Imaged slice 914A (eg, S i ) may include a portion 918A of the target site, and a second imaging slice 914B (S i-1 , e.g., acquired earlier in time) may include a second portion 918B. The second portion 918B appears to be displaced relative to the first portion 918A by a displacement 920. Because the displacement 920 is typically an unknown value or vector, various techniques may be used, such as to spatially register the first portion 918A with the second portion 918B. Fig. 9B As shown, the spatially registered slice 914C may include the adjusted position of a portion 918C of the target site. In symbology, a group or series of acquired imaging slices may be labeled S i , S i-1 , S i-2 ,...,S i-n , where S i corresponds to the most recently acquired slice for processing, S i-n Corresponds to the earliest slice acquired for processing. A series of spatially registered imaging slices can be labeled S i , S' i-1 , S' i-2 ,……,S' i-n .

[0108] Fig.10 Generally, a technique is shown that can include combining images from two (or more) imaging slices S i , S i-1 , S i-2 ,……,Si-n The segmented portions of the target site are spatially registered. In one approach, a metric can be defined to quantify how well the two portions 1018A and 1018B of the target site are aligned (or more generally, how well two imaging slices or imaging volumes are aligned), such as the similarity between pixels along the intersection line 1022 connecting the first portion 1018A to the displaced second portion 1018B. Examples of such metrics include one-dimensional normalized cross-correlation or mutual information. Automatic optimization techniques can be performed to find the translation of the second portion 1018B that provides the optimal value of the metric. Masks can be used to limit calculations to only the most relevant pixels, such as pixels close to the target of interest. This first approach typically relies on planes with common intersection lines.

[0109] Another method may include creating three-dimensional (3D) voxel images, such as a first image corresponding to a first slice orientation, including filled voxels corresponding to a portion of a target site within the slice and zero-valued voxels outside the slice, and a second image having a second slice orientation, including itself filled voxels corresponding to a portion of the target site in the second image. As an illustrative example, MR imaging slices typically have a finite thickness, such as corresponding to approximately 5 millimeters. The finite slice thickness can be accounted for by assigning slice information to voxels within the actual slice thickness using two-dimensional (2D) pixel information extracted from the MR imaging slices. 3D registration techniques can be used to find the optimal translation between the two 3D images, and masks can be used to exclude voxels that do not contain slice information. 3D registration methods also typically rely on having common intersection lines. An illustrative example of three-dimensionally registered imaging slices is shown in Fig.11A illustratively shown in FIG.

[0110] In yet another approach, prediction techniques can be applied to previously imaged slices to find the most likely location of a portion of a target site oriented for a previously imaged slice but having the same timestamp as a current or recently acquired imaging slice. Various techniques such as kernel density estimation, neural networks, deep learning neural network models, regression techniques including support vector machines or random forests, or template matching techniques can be used for such predictions. Prediction can be particularly effective if the motion of the target site is highly periodic in nature, such as associated with respiratory motion. Although the use of prediction is only an approximation, it can also be used in conjunction with the image-based methods described above and elsewhere herein. Prediction-based methods do not rely on image acquisition planes having common intersection lines, so prediction techniques can be used when acquiring parallel slices.

[0111] In yet another approach, when acquiring rapid perpendicular MR imaging slices, dark lines appear in the current image at locations where the previous image was excited but has not yet fully relaxed to equilibrium. Because the excitation magnetic dipoles from the previous slice follow the moving anatomical structure, the visible previous excitation area (e.g., the excitation line) can be used as an anchor point to align the previous slice with the imaging information acquired later. This can be taken into account implicitly by the other techniques described above, because such darkened areas such as pixel or voxel features can affect the registration, or explicitly, such as by identifying or explicitly determining the location of these darkened areas in the acquired imaging slice. For example, peak detection techniques can be used to find points along the center of the darkened area, and the line (straight or curved) that provides the best fit to these points can be found. Various examples described in this document involve registration between images, such as including registration of a segmented portion of a first image with a segmented portion or other identifying features of a second image (or a series of images). In general, the registration between two images can include a moving image and a fixed image. The goal can be to determine the position or transformation of the moving image so that it best fits the fixed image. "Move" may mean, for example, shifting the moving image in one or more dimensions (translation only), shifting and rotating the moving image, or even deforming the moving image. The degrees of freedom available for moving the moving image may be referred to as "optimization parameters", which may include, as illustrative examples, two degrees of freedom in the case of utilizing only two 2D image shifts, or six degrees of freedom for shift plus rotation of a 3D image. Although we may generally refer to 3D image registration, "optimization parameters" may still be collectively referred to as "shift values" without loss of generality.

[0112] For a given set of optimization parameters, the metric can define the "goodness" of the overlap determined between the moving image and the fixed image. Various examples can include determining a normalized cross-correlation or mutual information (e.g., using voxel or pixel values ​​as input values), and such a technique can provide an indication of the optimal value of the metric when the images are perfectly matched. The properties of the underlying imaging technique can help establish the metric to be used. For example, mutual information techniques can be used for images obtained using different modalities such as positron emission tomography (PET) and CT imaging.

[0113] Optimization techniques can be used to find shift values ​​that optimize a metric between a moving image and a fixed image, for example to identify a shift value that gives an optimal match between the two images. Different optimizers that can be used may include gradient descent, simulated annealing, trust region gradient descent, or one or more other techniques. In some cases, there may be deformation between the images, but it is not always necessary to optimize the deformation vector field and take it into account, because the optimization of the deformation information may cause one or more undesirable delays or may erode the stability of the optimization technique. The calculation of the metric can be limited to a specified region of interest, such as a "mask". For example, assuming that there is negligible deformation in the region of interest, one or more of a translation or rotation without deformation can be used to determine a locally optimal shift value in the specified region of interest.

[0114] Typically, if one or more 2D imaging slices are available, the motion of the radiotherapy target can be determined between the current 2D slice (e.g., MR imaging slice) and the initial 3D volume (e.g., reference volume image). As an illustrative example, the 3D volume can also correspond to a specific respiratory phase to reduce respiratory blur, which can be accomplished by gated or triggered imaging (e.g., triggered imaging), or by extracting a 3D volume image from four-dimensional imaging information at a specific phase (e.g., extracting a 3D volume snapshot from a 4D imaging series, such as 4D imaging information obtained using MR imaging).

[0115] In one approach, the target itself may be identified in the imaging slice. However, while segmentation is mentioned elsewhere herein, the techniques described herein also apply if segmentation or other feature extraction is not performed. As an illustrative example, a best fit of grayscale values ​​may be determined between one or more later acquired imaging slices and an earlier acquired reference 3D image. The target need not actually be in the image, but a region of interest may be specified, which may include, for example, a portion of the target, or the target plus an edge, or simply an arbitrary region of interest without the target.

[0116] In one approach, a 2D / 3D registration may be performed to find a relative shift between a current imaging slice and a reference volume. The 3D image may represent a "moving image" while the 2D slice may represent a "fixed image" associated with the registration technique. Typically, the location of a target may be identified in the 3D image, and applying the relative shift to an earlier target location may provide an estimate of an updated position of the target, such as an estimate of an updated position of the target within a three-dimensional coordinate system. Such an estimate may be performed without the need to determine a deformation, and may even be performed without using rotation parameters, such as to improve the efficiency of execution of an optimization technique. Simplifying assumptions may be applied, such as when registering a 3D reference image to one or more 2D imaging slices.

[0117] For example, a designated region of interest can be used to define a mask around the object plus an edge (although the region of interest need not include the object or all of the object). Deformation may be present, but as an illustrative example, if the edge is small, then the deformation will also be small (e.g., negligible), and a registration technique that ignores the deformation can be used to determine the displacement of the object itself. In this way, a single-plane technique (e.g., one in which the imaging slice is a fixed image) can still provide information indicative of displacement in three dimensions. Such a technique is robust when there are enough features in the image to "lock" the registration in three dimensions - the registration converges to a well-defined optimum - although no features need to be extracted or segmented.

[0118] In one approach, normalized cross-correlation of image gradients may be used as a registration metric, as such techniques tend to match image edges and are less sensitive to absolute grayscale values, which may vary from image to image. In some cases, a single imaging slice or plane may not be robust enough to establish a shift with a clear optimal value (colloquially, "locked"). Therefore, as an illustrative example, additional slices may be used to help lock all directions, such as using the current sagittal and previously acquired coronal slices can be used to help lock all directions, or vice versa. As the number of slices used for registration increases, whether parallel, orthogonal, or in non-parallel and non-orthogonal planes, more information is available for registration between the reference volume image and the later acquired imaging slices, which improves the robustness of the resulting shift parameter determination.

[0119] Fig.12 A second technique for determining the movement of an object according to some embodiments is shown. Fig.12As shown, a first slice (e.g., a 2D movie image) that depicts an object and is defined along a first plane (e.g., a sagittal plane) can be captured in real time at a first time point during a radiotherapy treatment phase. The first slice is registered with a portion of a volumetric image depicting the object to determine the movement of the object along a first direction and a second direction. For example, a corresponding plane (e.g., a sagittal plane) can be extracted from a pre-treatment 3D image and registered with the captured first slice to determine the movement of the object along the first direction and the second direction. As an example, the first direction and the second direction can include an up / down direction and an anterior / posterior direction. The left / right movement of the object (e.g., movement along a third direction) may be missing in the registration of the first slice with the volume portion. In this case, the movement of the object along the missing third direction (e.g., left / right direction) can be obtained based on a second slice (e.g., a 2D movie image) that depicts the object from a previous time point during the radiotherapy treatment phase. The second slice can be defined along a coronal plane and, when registered with a corresponding plane in the volumetric image, provides movement along the left / right direction. In this way, movement of an object along first, second, and third directions at a current point in time of a radiotherapy treatment session may be determined using slices defined along a first plane corresponding to the current point in time and using previously captured slices defined along a second plane.

[0120] In some cases, the volumetric image is defined as a moving image for registration purposes, while the 2D image captured during the radiotherapy treatment phase is defined as a fixed image. At a later or sequential next time point in the radiotherapy treatment phase, a third slice (e.g., a 2D movie image) in which the object is depicted and defined along the second plane is captured. The third slice is registered with a portion of the volumetric image depicting the object to determine the movement of the object along the first direction and the third direction. For example, a corresponding plane (e.g., a coronal plane) can be extracted from the pre-treatment 3D image and registered with the captured third slice to determine the movement of the object along the first direction and the third direction. As an example, the first direction and the third direction can include an up / down direction and a left / right direction, respectively. The forward / backward movement of the object (e.g., movement along the second direction) may be missing in the registration of the third slice with the volume portion. In this case, the movement of the object along the missing second direction (e.g., the forward / backward direction) can be obtained based on a fourth slice (e.g., a 2D movie image) in which the object was depicted at a previous time point during the radiotherapy treatment phase. A fourth slice may be defined along a sagittal plane and, when registered with a corresponding plane in the volumetric image, provide movement in an anterior / posterior direction. In this manner, movement of the subject in the first, second, and third directions at a current point in time of a radiotherapy treatment session may be determined using slices defined along a first plane corresponding to the current point in time and using previously captured slices defined along a second plane.

[0121] Fig.13A timing diagram for determining movement of an object according to a second technique is shown in accordance with some embodiments. Fig.13 As shown, during a first time point 1310 of a radiotherapy treatment session, a sagittal image 1312 (e.g., a 2D movie image along the sagittal plane) is captured. The sagittal image 1312 is registered with the pre-treatment 3D volumetric image. As a result of this registration, the movement of the object depicted in the sagittal image 1312 captured during the first time point can be determined along a first direction 1320 and a second direction 1330 (e.g., along the up / down direction and along the anterior / posterior direction). The movement along the left / right direction 1340 is missing. In order to restore the missing movement information, a coronal image 1350 (e.g., a 2D movie image along the coronal plane) captured during one or more previous time points 1360 in the same radiotherapy treatment session is retrieved. The previously captured coronal image 1350 is registered with the same 3D volumetric image to provide the movement along the left / right direction 1340 (e.g., left / right movement). In this way, the movement of the object along three directions can be determined using the currently captured image defined along the first plane and the previously captured image defined along the second plane.

[0122] Fig.14 is a flow chart 1400 for determining movement of an object according to a second technique, in accordance with some implementations. Fig.14 The operations discussed in can be performed sequentially, in parallel, skipped, or out of order. Fig.14 The operations discussed in the above may be performed by the computing system 510 ( Figure 5 )implement.

[0123] At operation 1401 , the computing system 510 receives a volumetric image including a time-varying object prior to a given radiation session.

[0124] At operation 1402, the computing system 510 receives a first imaging slice including a subject at a first point in time during a given radiation session. The first imaging slice is defined along a first plane.

[0125] At operation 1403 , the computing system 510 accesses a second imaging slice including the object at a first time point during a given radiation session, the second imaging slice having been acquired at a second time point prior to the first time point during the given radiation session and defined along a second plane.

[0126] At operation 1404 , the computing system 510 determines movement of the object in the first direction and the second direction based on the first imaging slice registered with the volumetric image.

[0127] At operation 1405 , the computing system 510 determines movement of the object along a third direction at the first point in time based on the second imaging slice.

[0128] At operation 1406 , the computing system 510 generates an updated treatment plan to control the delivery of the treatment beam based on the determined movement.

[0129] Registration may include scanning the 3D volume over one or more imaging slices, or the slices may be inserted into a 3D volume as mentioned elsewhere herein, in which case 3D to 3D registration techniques may be used (e.g., registration that suppresses or ignores unfilled voxels). Multi-slice techniques may work well, for example, where slices are acquired in rapid succession, e.g., with less than about 50 ms between the instants corresponding to the acquired slices. Older slices may become "stale" if longer durations occur between imaging slice acquisitions. However, such slices may still be temporally realigned, e.g., using one of the following to temporally realign: a) a prediction technique, e.g., where target motion is periodic in nature; b) alignment of excitation lines provided by an imaging modality (e.g., excitation lines visible from MR imaging acquisitions), e.g., by fitting lines that pass through such excitation lines as mentioned elsewhere herein; or c) by spatially registering old imaging slices so that the old imaging slices are matched to the current slice using metrics calculated along the intersection lines.

[0130] Fig.11A A composite 1118 representation of a target site is generally shown, for example, after spatial registration of segmented portions of the target site acquired at different times corresponding to spatially registered imaging slices 1114A and 1114B. After the portions of the target site from two or more imaging slices have been spatially registered, features can be extracted, for example, from the composite 1118. For example, the location of the centroid 1108 or other features of the composite 1118 can be determined, for example, to provide an estimate of the location of the centroid of the target site for updating a radiation treatment plan. In another example, the determined spatial displacement between the portions of the target site in the first imaging slice 1114A and the second imaging slice 1114B can be used to update the radiation treatment plan.

[0131] Fig. 11BGenerally, techniques are shown for determining a difference between a later estimated or predicted position of a composite 1118 of a target site based on acquired spatially registered imaging information and an earlier position of a target site 616, such as represented by earlier acquired volumetric imaging information. The displacement of one or more features can be used to estimate a current or future target site position. For example, the displacement between an earlier determined centroid 608 extracted from reference imaging information and a later determined centroid 1108 can be determined. In another example, a shift parameter from imaging registration can be used to provide an updated target site without extracting or determining features such as centroids. In general, a variety of other techniques can be used, such as to extract information indicating motion or displacement of the target site 616 using spatially registered imaging slices.

[0132] For example, once a set of previously imaged slices are aligned with the current slice to form a slice set S i , S' i-1 , S' i-2 ,……,S' i-n , then the slice set (or composite 1118 generated from such a slice set) can be found to be consistent with a three-dimensional reference volume (e.g., corresponding to Fig. 11B As an illustrative example, sagittal slice acquisition may precede coronal slice acquisition. A spatial translation may be identified that brings the coronal slice to the same time point as the sagittal slice to spatially register the coronal slice with the sagittal slice. The two slices may then be established in 3D voxel coordinate space. This may be accomplished by interpolating the slices as infinitesimally small slices or using finite slices of known slice thickness (e.g., Fig.11A and Fig. 11B as shown in the complex 1118).

[0133] Voxels that are not partially filled by the target site from each slice may not be filled. A 3D to 3D registration may then be determined between a 3D reference image (e.g., corresponding to the target site 616) and a 3D "slice-filled image" (corresponding to the composite 1118). A mask may be used to filter out voxels that have not yet been filled. In another example, a 3D to 2D registration may be performed, such as using multiple 2D slices rather than a single composite. For example, a shift parameter of a 3D reference image (e.g., a set of shift values ​​for the displacement of one or more features such as the center of mass, or a specified translation or rotation of a region of interest) may be evaluated, such as to optimize the value of a similarity metric comparing a voxel of a shifted 3D volume reference image with each slice of a plurality of 2D imaging slices using the registration and optimization techniques described above. In this way, a 3D to 2D registration may be performed, such as to identify an optimal set of shift values. The position of the target may be updated using information obtained from the shift value. 3D to 3D, 2D to 2D or 3D to 2D registration techniques do not require image segmentation (e.g., the target site itself does not need to be segmented), and such registration does not require identification of imaging features, such as centroids or edge locations. Alternatively, grayscale or contrast imaging information (or other extracted values) can be used to perform registration, such as performing registration on a specified region of interest as generally mentioned elsewhere herein.

[0134] Fig. 11C Generally, a technique for updating a treatment plan is shown, for example, to shift a treatment site to a new location. An earlier radiation therapy target area 1110A can be established, for example, corresponding to a target site extracted from a volume reference image or established based on other treatment or dosimetry goals. Such a treatment target area 1110A can be established at least in part using one or more of the following: positioning the radiation therapy output, modulating the treatment beam (including one or more of the modulated intensity or beam shape), or using an actuator such as a movable treatment couch or platform to move the patient. One or more features can be extracted from the treatment target area, such as the center of mass 1108A. An estimated or predicted displacement of a target site (e.g., a tumor or other structure to be treated) can be determined, for example, using other techniques described herein to determine.

[0135] The treatment target area 1110A can be adjusted to provide an updated treatment target area 1110B. For example, if the target site is translated due to patient motion, a similar displacement can be applied to the earlier area 1110A to provide an updated treatment target area 1110B. Similarly, if other features are extracted, other techniques such as rotation or scaling can be used to adjust the treatment target area 1110A. In another example, instead of adjusting the treatment target area 1110A or in addition to adjusting the treatment target area 1110A, other techniques can be used to control the treatment, for example, unless the updated treatment target area 1110B falls within a specified area, the treatment is gated to inhibit the delivery of radiation therapy. In yet another example, an alternative signal such as a sensor output can be used to gate the treatment. For example, the sensor output can be related to the location of the feature extracted from the imaging slice, such as the centroid location of the target site within the imaging slice after the imaging slice is spatially registered. The treatment delivery can be gated in a synchronized manner, such as triggered in response to the sensor output (e.g., to deliver treatment at a specific time during a periodic motion such as breathing).

[0136] Fig.15 A third technique for determining the movement of an object according to some embodiments is shown. Fig.15 As shown, the radiation therapy treatment computing system 510 initially generates a composite (template) imaging slice 1520 that is registered with a corresponding plane of the 3D volumetric image 1540 .

[0137] Fig.16 An illustrative technique for generating a composite imaging slice 1520 according to some embodiments is shown. In some cases, the composite image is generated by averaging a selected subset 1610 of imaging slices. For example, a plurality of imaging slices 1610 are acquired prior to activating a radiation therapy beam during a given radiation phase. A given one of the slices corresponding to a reference breathing phase (e.g., end of exhalation) is selected as a target image 1612. The radiation therapy treatment computing system 510 deformably registers 1614 the remaining set of the plurality of imaging slices to the selected target image 1612.

[0138] The radiotherapy treatment computing system 510 averages the deformably registered plurality of imaging slices and determines an average offset 1620 for each of the plurality of imaging slices. The radiotherapy treatment computing system 510 resamples the imaging slices based on the average offset to select a subset 1630A to 1630N of the resampled imaging slices based on the distance of each resampled imaging slice to the target image (e.g., the selected imaging slice). For example, the radiotherapy treatment computing system 510 calculates the distance between each resampled imaging slice and the target image. The radiotherapy treatment computing system 510 selects those imaging slices whose distance between the slice and the target slice is less than a specified threshold. A composite image 1640 is then generated by averaging the selected subset 1630A to 1630N of the resampled plurality of imaging slices.

[0139] Return to reference Fig.15 , after generating the composite image 1640, the composite imaging slice 1520 is registered with a new imaging slice corresponding to the same plane as the composite imaging slice to determine the movement of the object depicted in the composite imaging slice. As an example, at a first time point during a given radiation phase, a first imaging slice 1510 including an object 1512 is received. The first imaging slice 1510 can correspond to a first plane (e.g., a sagittal plane). The radiotherapy treatment computing system 510 accesses the composite imaging slice 1520 corresponding to the first plane at a first time point during a given radiation phase. The composite imaging slice is generated using a plurality of imaging slices 1530 acquired before the first time point. The radiotherapy treatment computing system 510 spatially registers the first imaging slice with the composite imaging slice to determine the movement of the object. The radiotherapy treatment computing system 510 generates an updated treatment plan to control the delivery of the treatment beam based on the determined movement.

[0140] In some cases, the radiation therapy treatment computing system 510 registers the first imaging slice to the composite imaging slice by limiting the pixel mask considered in the registration to one or both of the first imaging slice and the composite imaging slice. In some cases, before the first imaging slice 1510 is registered to the composite imaging slice 1520, the composite imaging slice 1520 is registered to the two-dimensional slice extracted from the volumetric image 1540. In some cases, the composite imaging slice 1520 registered to the two-dimensional slice extracted from the volumetric image is manually adjusted to select or change the location or portion of the composite image registered relative to the two-dimensional slice.

[0141] In some embodiments, similar operations are performed to determine movement based on a composite image along a different plane (e.g., a coronal plane). In such a case, the composite imaging slice 1520 is a first composite image. At a second time point during a given radiation phase, the radiotherapy treatment computing system 510 receives a second imaging slice corresponding to a second plane (e.g., a coronal plane). The radiotherapy treatment computing system 510 accesses a second composite imaging slice (not shown) corresponding to a second plane (e.g., a coronal composite imaging slice) at a second time point during a given radiation phase. The second composite imaging slice is generated using a plurality of additional imaging slices obtained before the second time point. Specifically, the coronal composite imaging slice is generated in the same manner as described above for the sagittal plane, but using slices received along the coronal plane. For example, a coronal composite image is generated by averaging a selected subset of the coronal imaging slices. For example, multiple coronal imaging slices are obtained before activating the radiotherapy beam during a given radiation phase. A given slice of the coronal slice corresponding to the reference respiratory phase is selected as the target image. In some cases, the same reference respiratory phase used to select a target image for a first imaging plane (eg, sagittal plane) is used to select a target image for a coronal imaging plane.The radiation therapy treatment computing system 510 deformably registers the remaining set of multiple coronal imaging slices to the selected target image.

[0142] The radiotherapy treatment computing system 510 averages the multiple coronal imaging slices of the deformable registration and determines the average offset of each of the multiple coronal imaging slices. The radiotherapy treatment computing system 510 resamples the coronal imaging slices based on the average offset to select a subset of the resampled coronal imaging slices based on the distance of each resampled coronal imaging slice to the target image (e.g., the selected coronal imaging slice). For example, the radiotherapy treatment computing system 510 calculates the distance between each resampled coronal imaging slice and the coronal target image. The radiotherapy treatment computing system 510 selects those coronal imaging slices whose distance between the coronal slice and the target coronary slice is less than a specified threshold. In some cases, the same threshold used to select imaging slices for the first imaging plane (e.g., the sagittal plane) is used to select imaging slices for the coronal imaging plane. In some cases, the thresholds for different planes are also different. A coronal composite image is then generated by averaging the selected subset of the resampled multiple imaging slices.

[0143] The radiation therapy treatment computing system 510 spatially aligns a second imaging slice (e.g., a coronal imaging slice) with a second composite imaging slice (corresponding to the coronal plane), including moving a portion of an object in the second imaging slice to align the object in the second imaging slice with a portion of the object in the second composite imaging slice.

[0144] Fig.17is a flow chart 1700 for determining movement of an object according to a third technique, in accordance with some implementations. Fig.17 The operations discussed in can be performed sequentially, in parallel, skipped, or out of order. Fig.17 The operations discussed in the above may be performed by the computing system 510 ( Figure 5 )implement.

[0145] At operation 1701, the computing system 510 receives a first imaging slice including a subject at a first point in time during a given radiation session. The first imaging slice corresponds to a first plane (eg, a sagittal plane).

[0146] At operation 1702, the computing system 510 accesses a compound imaging slice corresponding to a first plane at a first point in time during a given radiation session. The compound imaging slice is generated using a plurality of imaging slices acquired prior to the first point in time.

[0147] At operation 1703, the computing system 510 spatially registers the first imaging slice with the composite imaging slice. For example, the computing system 510 optimizes a cost function based on the difference between the first imaging slice and the composite imaging slice to register the first imaging slice with the composite imaging slice.

[0148] At operation 1704 , the computing system 510 determines movement of the object using the spatially registered first imaging slice and the composite imaging slice.

[0149] At operation 1705, the computing system 510 generates an updated treatment plan to control the delivery of the treatment beam based on the determined movement.

[0150] Fig.18 A fourth technique for determining movement of an object according to some embodiments is shown. Fig.18 The technology shown, at a first time point in a given radiation phase, receives a first imaging slice 1810 corresponding to a first plane (e.g., corresponding to a sagittal plane or a 2D movie image defined along a sagittal plane). The first imaging slice is encoded into a lower dimensional representation (e.g., by transforming the principal component analysis (PCA) components of the deformable vector field (DVF) of the first imaging slice to estimate the PCA components of the second imaging slice). A trained machine learning model (e.g., a support vector machine or a random forest machine, a deep learning neural network model, or a neural network or regression technique including a support vector machine) is applied to the encoded first imaging slice to estimate an encoded version of a second imaging slice corresponding to a second plane (e.g., a coronal plane) at the first time point, thereby providing a pair of imaging slices for the first time point. As an example, the trained machine learning model is applied to the first imaging slice 1810 (e.g., a sagittal plane) to estimate an encoded version of a coronal plane imaging slice.

[0151] The pair of imaging slices are simultaneously spatially registered to a volumetric image received prior to a given radiation phase (e.g., a 3D pre-treatment volume of a patient) to generate an updated treatment plan to control the delivery of a treatment beam based on the calculated displacement of the object. In some cases, before registering the images, the estimated encoded version of the second imaging slice is decoded to obtain a second imaging slice at a first point in time. The decoded version is then spatially registered with the first imaging slice 1810 against the volumetric image to determine the movement or displacement of the object in three dimensions.

[0152] In some embodiments, the machine learning model is trained during the initial stage of the radiotherapy segmentation, before the radiotherapy segmentation, or during the delivery of the radiotherapy segmentation. In order to train the machine learning model, the radiotherapy treatment computing system 510 receives a first slice sequence 1810 and 1830 (e.g., a movie image) along the same plane (e.g., a sagittal plane image). The radiotherapy treatment computing system 510 also receives a second slice sequence 1820 (e.g., a movie image) along the same plane (e.g., a coronal plane image). In some cases, the coronal plane image is received as an alternative to the sagittal plane image, so that when the sagittal plane image is received, the coronal plane image is not received, and vice versa. The radiotherapy treatment computing system 510 encodes the first slice sequence 1810 and 1830 into a lower dimensional representation and interpolates a sagittal image slice 1812 between the first slice sequence 1810 and 1830. The interpolated sagittal image slice 1812 can represent the expected sagittal image slice when the coronal image slice 1820 is received. When sagittal image slices are received, similar operations are performed to interpolate coronal image slices 1822 to generate the expected coronal image slices.

[0153] The radiotherapy treatment computing system 510 applies the coronal image slice 1820 to the machine learning model to predict the sagittal image slice. During training, the radiotherapy treatment computing system 510 calculates the deviation between the predicted sagittal image slice and the interpolated sagittal image slice 1812. The radiotherapy treatment computing system 510 updates the parameters of the machine learning model based on the calculated deviation. The radiotherapy treatment computing system 510 determines whether the calculated deviation meets the stopping criteria to determine whether to continue training based on additional coronal image slices or output the trained machine learning model. Similarly, the radiotherapy treatment computing system 510 applies the sagittal image slice 1830 to the machine learning model to predict the coronal image slice. During training, the radiotherapy treatment computing system 510 calculates the deviation between the predicted coronal image slice and the interpolated coronal image slice 1822. The radiotherapy treatment computing system 510 updates the parameters of the machine learning model based on the calculated deviation. The radiotherapy treatment computing system 510 determines whether the calculated deviation meets the stopping criteria to determine whether to continue training based on additional sagittal image slices or output the trained machine learning model. In some cases, the machine learning model is updated indefinitely during the radiation therapy treatment phase as new sagittal and coronal image slices are received. Although sagittal image counterparts and coronal image counterparts are described as being used to train the machine learning model, any parallel, perpendicular, or other angled image slices can similarly be used to train the machine learning model to predict slices along one angle based on slices received along a different angle.

[0154] Fig.19 is a flowchart 1900 for determining movement of an object according to a fourth technique, in accordance with some implementations. Fig.19 The operations discussed in can be performed sequentially, in parallel, skipped, or out of order. Fig.19 The operations discussed in the above may be performed by the computing system 510 ( Figure 5 )implement.

[0155] At operation 1901 , the computing system 510 receives a first imaging slice corresponding to a first plane at a first point in time in a given radiation phase.

[0156] At operation 1902 , the computing system 510 encodes a first imaged slice into a lower dimensional representation.

[0157] At operation 1903, the computing system 510 applies the trained machine learning model to the encoded first imaging slice to estimate the encoded version of the second plane at the first time point to provide a set of imaging slices (e.g., a pair of imaging slices) for the first time point. In some cases, the set of imaging slices includes three imaging slices that are orthogonal to each other or along planes that are tilted relative to each other.

[0158] At operation 1904, the computing system 510 simultaneously spatially registers the set of imaging slices (e.g., the pair of imaging slices) to a volumetric image received before a given radiation phase, the volumetric image including the time-varying object to calculate the displacement of the object. For example, the computing system 510 inserts the set of imaging slices into a 3D volume including the volumetric image and fills the voxels of the 3D volume where the set of imaging slices intersects the voxels of the 3D volume. As another example, the computing system 510 optimizes a cost function based on the difference between the set of imaging slices and the volumetric image.

[0159] At operation 1905 , the computing system 510 generates an updated treatment plan to control the delivery of the treatment beam based on the calculated displacement of the object.

[0160] The above techniques (including the second technique, the third technique, and the fourth technique for determining the movement of an object during a given radiotherapy phase) can be performed in a serial manner, for example, including acquiring images and processing such images, then acquiring another series of images, and processing a set of subsequently acquired images. In another method, one or more parts of the techniques described herein can be performed in parallel or in an interleaved manner, for example to provide displacement or other information that is continuously generated for updating the treatment plan in an adaptive manner. In some cases, the movement of the object can be determined using a third technique (for example, using a composite image). If the movement of the object exceeds a specified threshold, the treatment plan can be updated immediately. Alternatively, if the movement determined based on the third technique exceeds the threshold, the fourth technique can be applied to the same set of images to determine the movement of the object before updating the treatment plan. If the movement of the object exceeds the specified threshold again, the treatment plan can be updated. In some cases, the third technique can be applied before the fourth technique based on the priority assigned to the third technique. The fourth technique can be skipped and the fourth technique is applied only when the movement of the object is determined to exceed the specified threshold according to the third technique.

[0161] As an illustrative example, the imaging slices are shown as two-dimensional planes that are orthogonal to each other. However, as mentioned with respect to other examples herein, the slices need not be planar nor orthogonal. Each acquired imaging slice can be processed, including processing one or more slices in parallel with the acquisition of the imaging slices. For example, such processing can include sampling information from a volume reference image, such as a plane or region corresponding to the acquired imaging slice. The extent of a target site (e.g., a tumor or other anatomical structure that can be used as a landmark to adaptively guide radiation therapy) can be extracted from one or more segmented structures in the volume reference image.

[0162] Margins can be added to the range of the target site, for example to define an area in which the target site must be located in subsequently acquired imaging slices. The margins can be adjusted based on factors such as image acquisition rate (the lower the acquisition rate, the farther the tumor or other target can travel between image acquisitions, and the margins can be enlarged accordingly). Tumors or other targets can be segmented in the acquired imaging slices. The segmented target information extracted from one or more imaging slices can be spatially registered with the target site identified in the planning image. As described in other examples herein, such registration can be used to determine an updated target site or to otherwise control or adjust treatment. For example, as an illustrative example, an updated target site can be used to gate treatment, such as pausing treatment when the updated target site indicates that the target site is partially or completely outside a specified spatial gating window; or the updated target site can be used to adjust the beam aperture shape to track the tumor.

[0163] Various notes and examples

[0164] Each non-limiting example described in this document may stand alone or may be combined with one or more of the other examples in various permutations or combinations.

[0165] The above detailed description includes reference to the accompanying drawings that form a part of the detailed description. The accompanying drawings show specific embodiments in which the subject matter of the present invention can be practiced by way of illustration. These embodiments are also referred to as "examples" in this article. Such examples may include elements other than those elements shown or described. However, the inventors also contemplate examples that only provide those elements shown or described. In addition, the inventors also contemplate examples of any combination or arrangement of those elements (or one or more aspects thereof) shown or described relative to the specific examples (or one or more aspects thereof) shown or described herein or relative to other examples (or one or more aspects thereof) shown or described herein.

[0166] In the event of an inconsistency in usage between this document and any document incorporated by reference, the usage in this document controls.

[0167] In this document, as is common in patent literature, the terms "a" or "an" are used to include one or more than one, independent of any other instance or usage of "at least one" or "one or more". In this document, unless otherwise indicated, the term "or" is used to refer to a non-exclusive "or", such that "A or B" includes "A but not B", "B but not A", and "A and B". In this document, the terms "including" and "in..." are used as the plain English equivalents of the corresponding terms "comprising" and "wherein". In addition, in the appended claims, the terms "including" and "comprising" are open-ended, that is, a system, device, article, combination, formulation or process that includes elements other than those listed after such terms in the claim is still considered to fall within the scope of the claim. In addition, in the appended claims, the terms "first", "second", and "third", etc. are used only as labels and are not intended to impose numerical requirements on their objects.

[0168] The method examples described herein may be at least partially machine or computer-implemented. Some examples may include computer-readable media or machine-readable media that can be operated to configure an electronic device to perform instructions encoding the method described in the above example. The implementation of such a method may include code, such as microcode, assembly language code, higher-level language code, etc. Such code may include computer-readable instructions for performing various methods. The code may form a part of a computer program product. In addition, in an example, for example, during execution or at other times, the code may be tangibly stored on one or more volatile, non-transient or non-volatile tangible computer-readable media. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (e.g., high-density disks and digital video disks), cassettes, memory cards or sticks, random access memory (RAM), read-only memory (ROM), etc.

[0169] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more aspects thereof) can be used in combination with each other. Other embodiments can be used, such as those used by those of ordinary skill in the art after reading the above description. The abstract is provided to comply with 37C.FR§1.72(b), enabling readers to quickly determine the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above specific embodiments, various features can be combined together to simplify the present disclosure. This should not be interpreted as intended that the disclosed features that are not claimed for protection are essential to any claim. More precisely, the subject matter of the invention may not lie in all the features of a particular disclosed embodiment. Therefore, the attached claims are incorporated into the specific embodiments as examples or embodiments herein, wherein each claim exists independently as a separate embodiment, and it is envisioned that such embodiments can be combined with each other in various combinations or arrangements. The scope of the subject matter of the present invention should be determined with reference to the attached claims and the full scope of equivalents given by such claims.

Claims

1. An apparatus for controlling an adaptive radiation therapy delivery system, the apparatus comprising: means for receiving a first imaging slice comprising the object at a first point in time during a given radiation phase, the first imaging slice corresponding to a first plane; means for collecting a plurality of imaging slices prior to activating the treatment beam, wherein the plurality of imaging slices are obtained prior to the first time point; means for accessing, at said first point in time during said given radiation phase, a composite imaging slice corresponding to said first plane, said composite imaging slice being generated using said plurality of imaging slices; means for spatially registering the first imaging slice with the composite imaging slice; means for determining movement of the object using the spatially registered first imaging slice and the composite imaging slice based at least in part on a difference between the first imaging slice and the composite imaging slice; and means for generating an updated treatment plan to control delivery of the treatment beam based on the determined movement, A given one of the plurality of imaging slices corresponds to a reference respiratory phase and is selected as a target image.

2. The device according to claim 1, wherein: The object includes a tumor in a patient's body that is a target of the treatment beam, and the apparatus further includes a unit for receiving a volumetric image including the object before the given radiation phase, wherein a difference in image contrast between the first imaging slice and one or more slices of the volumetric image exceeds a threshold. 3 . The apparatus of claim 2 , further comprising means for spatially registering the composite imaging slice to a two-dimensional slice extracted from the volumetric image along the first plane.

4. The device according to claim 3, wherein: Registering the first imaging slice with the composite imaging slice includes limiting a pixel mask considered in the registration to one or both of the first imaging slice and the composite imaging slice to limit registration calculations to only pixels associated with the object, wherein the composite imaging slice is registered to the two-dimensional slice before the first imaging slice is registered to the composite imaging slice.

5. The apparatus of claim 3, further comprising means for receiving user input to manually adjust a position or portion of the composite imaging slice registered relative to the two-dimensional slice.

6. The device according to claim 1, wherein: The composite image slice is a first composite image, the apparatus further comprising: means for receiving a second imaging slice including the object at a second point in time during the given radiation phase, the second imaging slice corresponding to a second plane; means for accessing, at said second point in time during said given radiation phase, a second composite imaging slice corresponding to said second plane, said second composite imaging slice being generated using a further plurality of imaging slices acquired prior to said second point in time; and The means for spatially registering the second imaging slice with the second composite imaging slice includes moving a portion of the object in the second imaging slice to register the object in the second imaging slice with a portion of the object in the second composite imaging slice.

7. The apparatus according to claim 1 further comprises a unit for deformably registering the remaining set of the plurality of imaging slices to the given one of the plurality of imaging slices to generate a corresponding transformation of each of the remaining set of the plurality of imaging slices.

8. The apparatus according to claim 7, further comprising: means for calculating a transformed average of each of said residual set of said plurality of imaging slices; means for adjusting the transform by removing a mean value from the transform of each of the remaining set of the plurality of imaging slices; means for resampling the plurality of imaging slices based on the adjusted transform of each of the remaining set of the plurality of imaging slices; means for selecting a subset of the resampled plurality of imaging slices based on a distance of each of the resampled plurality of imaging slices to the given one of the plurality of imaging slices; as well as Means for generating the composite imaging slice by averaging the selected subset of the resampled plurality of imaging slices.

9. The device according to claim 8, wherein: A distance between each selected subset of the resampled plurality of imaging slices and the given one of the plurality of imaging slices is less than a threshold value.

10. The device according to claim 1, wherein: The composite imaging slice is generated prior to activating the therapy beam, the first imaging slice is received prior to activating the therapy beam, and the movement is determined prior to activating the therapy beam, and wherein the therapy beam is activated after generating the updated therapy plan.

11. The device according to claim 1, wherein: The compound imaging slice is updated while the therapy beam is active, accessing the compound imaging slice includes retrieving an updated compound imaging slice, the first imaging slice is received while the therapy beam is active, and the movement is determined while the therapy beam is active.

12. The device according to claim 1, wherein: The composite imaging slice is generated prior to activation of the therapy beam, the first imaging slice is received after activation of the therapy beam, and the movement is determined after activation of the therapy beam.

13. A non-transitory computer readable medium comprising non-transitory computer readable instructions for controlling an adaptive radiation therapy delivery system, the non-transitory computer readable instructions performing operations comprising: receiving a first imaging slice including the object at a first point in time during a given radiation phase, the first imaging slice corresponding to a first plane; collecting a plurality of imaging slices prior to activating the treatment beam, the plurality of imaging slices being obtained prior to the first time point; accessing a composite imaging slice corresponding to the first plane at the first point in time during the given radiation phase, the composite imaging slice being generated using the plurality of imaging slices; spatially registering the first imaging slice with the composite imaging slice; determining movement of the object using the spatially registered first imaging slice and the composite imaging slice based at least in part on a difference between the first imaging slice and the composite imaging slice; as well as generating an updated treatment plan to control delivery of the treatment beam based on the determined movement, A given one of the plurality of imaging slices corresponds to a reference respiratory phase and is selected as a target image.

14. The non-transitory computer readable medium of claim 13, wherein: The object includes a tumor in a patient that is a target of the treatment beam, and also includes an operation for receiving a volumetric image including the object before the given radiation phase, wherein a difference in image contrast between the first imaging slice and one or more slices of the volumetric image exceeds a threshold.

15. The non-transitory computer readable medium of claim 14, further comprising an operation for spatially registering the composite imaging slice to a two-dimensional slice extracted from the volumetric image along the first plane.

16. The non-transitory computer readable medium of claim 15, wherein: Registering the first imaging slice with the composite imaging slice includes limiting a pixel mask considered in the registration to one or both of the first imaging slice and the composite imaging slice to limit registration calculations to only pixels associated with the object, wherein the composite imaging slice is registered to the two-dimensional slice before the first imaging slice is registered to the composite imaging slice.

17. The non-transitory computer readable medium of claim 15, further comprising an operation for receiving user input to manually adjust a position or portion of the composite imaging slice registered relative to the two-dimensional slice.

18. The non-transitory computer readable medium of claim 13, wherein: The composite image slice is a first composite image and further comprises operations for: receiving a second imaging slice including the object at a second point in time during the given radiation phase, the second imaging slice corresponding to a second plane; accessing a second composite imaging slice corresponding to the second plane at the second time point during the given radiation phase, the second composite imaging slice being generated using a further plurality of imaging slices acquired prior to the second time point; as well as The second imaging slice is spatially registered with the second composite imaging slice, including moving a portion of the object in the second imaging slice to register the object in the second imaging slice with a portion of the object in the second composite imaging slice.

19. The non-transitory computer readable medium of claim 13, further comprising operations for: deformably registering a remaining set of the plurality of imaging slices to the given one of the plurality of imaging slices to generate a transformation of each of the remaining set of the plurality of imaging slices; calculating a mean of the transforms for each of the remaining sets of the plurality of imaging slices; adjusting the transform by removing a mean value from the transform of each of the remaining set of the plurality of imaging slices; resampling the plurality of imaging slices based on the adjusted transform of each of the remaining set of the plurality of imaging slices; selecting a subset of the resampled plurality of imaging slices based on a distance of each of the resampled plurality of imaging slices to the given one of the plurality of imaging slices; as well as The composite imaging slice is generated by averaging a selected subset of the resampled plurality of imaging slices.

20. The non-transitory computer readable medium of claim 13, wherein: The composite imaging slice is generated prior to activating the therapy beam, the first imaging slice is received prior to activating the therapy beam, and the movement is determined prior to activating the therapy beam, and wherein the therapy beam is activated after generating the updated therapy plan.

21. The non-transitory computer readable medium of claim 13, wherein: The compound imaging slice is updated while the therapy beam is active, accessing the compound imaging slice includes retrieving the updated compound imaging slice, the first imaging slice is received while the therapy beam is active, and the movement is determined while the therapy beam is active.

22. The non-transitory computer readable medium of claim 13, wherein: The composite imaging slice is generated prior to activation of the therapy beam, the first imaging slice is received after activation of the therapy beam, and the movement is determined after activation of the therapy beam.

23. A system for controlling an adaptive radiation therapy delivery system, the system comprising: One or more processors configured to perform operations including: receiving a first imaging slice including the object at a first point in time during a given radiation phase, the first imaging slice corresponding to a first plane; collecting a plurality of imaging slices prior to activating the treatment beam, the plurality of imaging slices being obtained prior to the first time point; accessing a composite imaging slice corresponding to the first plane at the first point in time during the given radiation phase, the composite imaging slice being generated using the plurality of imaging slices; spatially registering the first imaging slice with the composite imaging slice; determining movement of the object using the spatially registered first imaging slice and the composite imaging slice based at least in part on a difference between the first imaging slice and the composite imaging slice; as well as generating an updated treatment plan to control delivery of the treatment beam based on the determined movement, A given one of the plurality of imaging slices corresponds to a reference respiratory phase and is selected as a target image.

24. The system of claim 23, wherein: The object includes a tumor in a patient that is a target of the treatment beam, and also includes an operation for receiving a volumetric image including the object before the given radiation phase, wherein a difference in image contrast between the first imaging slice and one or more slices of the volumetric image exceeds a threshold.

25. The system of claim 24, further comprising an operation for spatially registering the composite imaging slice to a two-dimensional slice extracted from the volumetric image along the first plane.

26. The system of claim 25, wherein: Registering the first imaging slice with the composite imaging slice includes limiting a pixel mask considered in the registration to one or both of the first imaging slice and the composite imaging slice to limit registration calculations to only pixels associated with the object, wherein the composite imaging slice is registered to the two-dimensional slice before the first imaging slice is registered to the composite imaging slice.

27. The system of claim 23, wherein: The composite imaging slice is generated prior to activating the therapy beam, the first imaging slice is received prior to activating the therapy beam, and the movement is determined prior to activating the therapy beam, and wherein the therapy beam is activated after generating the updated therapy plan.

28. The system of claim 23, wherein: The compound imaging slice is updated while the therapy beam is active, accessing the compound imaging slice includes retrieving an updated compound imaging slice, the first imaging slice is received while the therapy beam is active, and the movement is determined while the therapy beam is active.

29. The system of claim 23, wherein: The composite imaging slice is generated prior to activation of the therapy beam, the first imaging slice is received after activation of the therapy beam, and the movement is determined after activation of the therapy beam.

30. The system of claim 23, wherein: The operations also include: deformably registering a remaining set of the plurality of imaging slices to a given one of the plurality of imaging slices to generate a transformation of each of the remaining set of the plurality of imaging slices; calculating a mean of the transforms for each of the remaining sets of the plurality of imaging slices; adjusting the transform by removing a mean value from the transform of each of the remaining set of the plurality of imaging slices; resampling the plurality of imaging slices based on the adjusted transform of each of the remaining set of the plurality of imaging slices; selecting a subset of the resampled plurality of imaging slices based on a distance of each of the resampled plurality of imaging slices to the given one of the plurality of imaging slices; and The composite imaging slice is generated by averaging a selected subset of the resampled plurality of imaging slices.

Images