Cherenkov-imaging-based system and method for verifying position of deformable tissue undergoing radiotherapy

Abstract

Systems and methods verify position of deformable tissue undergoing radiotherapy using Cherenkov radiation. A Cherenkov image is captured of a treatment area of a subject during the radiotherapy. The Cherenkov image is segmented to generate a bio-morphological feature image defining vasculature of the deformable tissue. A transfer learning strategy is used to train a convolutional neural network to segment the Cherenkov image. A rigid registration of the bio-morphological feature image and a reference image is performed to determine a rigid displacement. A non-rigid registration of the bio-morphological feature image and the reference image is performed to determine a deformation map indicating locoregional deformation of the deformable tissue. A repositioning alert is generated when the locoregional deformation is greater than a deformation threshold, and an abort alert is generated when registration of the bio-morphological feature image and a reference image us unsuccessful.

Description

CHERENKOV-IMAGING-BASED SYSTEM AND METHOD FOR VERIFYINGPOSITION OF DEFORMABLE TISSUE UNDERGOING RADIOTHERAPYGOVERNMENT RIGHTS

[0001] This invention was made with government support under EB023909, R44 CA265654, and 5P30CA023108-40, awarded by the National Institutes of Health. The government has certain rights in the invention.FIELD

[0002] The present invention relates to the field of medical imaging and radiotherapy. More specifically, it pertains to a method and system for enhancing the precision of target positioning in radiotherapy of targets in soft tissue, such as the abdomen and breasts, through the use of Cherenkov imaging. The invention leverages Cherenkov-imaged vasculature as patient-specific biological fiducial markers to enable precise quantification of locoregional tissue deformations, thereby improving the accuracy of radiotherapy treatments.RELATED APPLICATIONS

[0003] This application claims priority to US Patent Application Serial Number 63 / 774,049, titled “Cherenkov Imaged Bio-morphological Features Verify Patient Positioning with Deformable Tissue Translocation in Breast Radiotherapy,” filed July 22, 2024, which is incorporated herein by reference.BACKGROUND

[0004] Optical imaging systems have been adapted to observe Cherenkov radiation emitted from targets exposed to high-energy radiation treatment and Cherenkov radiation has been measured to verify radiation dose.

[0005] Many objects exposed to radiation treatment have soft, or flexible, portions that are easily deformed and easily move relative to harder portions of the same object. For example, soft tissues, including breast and abdominal tissue, are easily deformed and may not be in the same position between a first radiation treatment and a subsequent radiation treatment session, even when more rigid parts (e.g., bone) of the object are in a same position. Further, targets, such as tumors, located within the soft tissues may also shift relative to rigid parts along with deformation of the soft tissue and / or movement of flexibleportions of the object. That is, a tumor in a breast shifts relative to bony parts of the same organism when soft tissue of the breast deforms.

[0006] Traditional methods of target positioning in radiotherapy often rely on external markers on the subject or imaging modalities such as X-rays or CT scans. These methods are limited by their inability to detect subtle tissue deformations or changes in positioning that occur between, or during, imaging and treatment.SUMMARY

[0007] Methods and systems use Cherenkov imaging to enhance the precision of target positioning in radiotherapy of targets having soft portions. In certain embodiments, the target is a human subject with soft tissues such as abdomen, including the liver or breasts, where the embedded structures include bio-morphological features such as vasculature.

[0008] One aspect of the present embodiments includes the realization that even when the more rigid parts (e.g., bones) of the object are correctly positioned and do not move, the soft tissues may still deform. Where this deformation occurs during radiotherapy, the treatment may not be as effective as intended. The present embodiments solve this problem by using Cherenkov-imaged vasculature as patient-specific biological fiducial markers that enable precise quantification of locoregional tissue deformations. Advantageously, the radiotherapy may be adjusted based on the tissue deformation and / or may be halted to prevent the treatment affecting tissue outside a desired treatment area.

[0009] In certain embodiments, the techniques described herein relate to a system for verifying positioning of deformable tissue during radiotherapy of a subject, including: a first camera configured to capture a Cherenkov image of a target area of the subject during the radiotherapy; and a controller having a processor and a memory storing software executable by the processor to cause the controller to: segment the Cherenkov image to form a bio- morphological feature image of bio-morphological features of the deformable tissue; and register the bio-morphological feature image to a reference image of the bio-morphological features to determine (a) a rigid displacement and (b) a nonrigid displacement.

[0010] In certain embodiments, the techniques described herein relate to a method for verifying position of deformable tissue undergoing radiotherapy, including: capturing a Cherenkov image of a treatment area of a subject during the radiotherapy; segmenting the Cherenkov image to generate a bio-morphological feature image defining vasculature of the deformable tissue; performing a rigid registration of the bio-morphological feature image and a reference image to determine a rigid displacement; and performing a non-rigid registrationof the bio-morphological feature image and the reference image to determine a deformation map indicating locoregional deformation of the deformable tissue.BRIEF DESCRIPTION OF THE FIGURES

[0011] In the drawings, identical reference numbers identify similar elements or acts. The sizes and relative positions of elements in the drawings are not necessarily drawn to scale. For example, the shapes of various elements and angles are not drawn to scale, and some of these elements are arbitrarily enlarged and positioned to improve drawing legibility. Further, the particular shapes of the elements as drawn, are not intended to convey any information regarding the actual shape of the particular elements, and have been solely selected for ease of recognition in the drawings.

[0012] FIG. 1 is a schematic diagram illustrating one example Cherenkov-imagingbased system for verifying position of deformable tissue undergoing radiotherapy, in embodiments.

[0013] FIG. 2 is a schematic illustrating further example detail of the controller of FIG. 1, in embodiments.

[0014] FIG. 3 shows one example of the Cherenkov image of FIG. 2, in embodiments.

[0015] FIG. 4 shows the reference image of FIG. 2 overlaid with the subsequently captured bio-morphological feature image to illustrate displaced vasculature, in embodiments.

[0016] FIG. 5 shows one example grid representation of the deformation map of FIG. 2, in embodiments.

[0017] FIG. 6 is a schematic diagram representing operation of the registration algorithm to determine the rigid displacement and the deformation map using rigid and non- rigid registration, in embodiments.

[0018] FIG. 7 shows one example deformation output image generated by the registration algorithm of FIG. 2 to simultaneously illustrate both rigid displacement and deformation map of tissue in the treatment area, in embodiments.

[0019] FIG. 8 is a flowchart illustrating one example method for verifying position of deformable tissue undergoing radiotherapy, in embodiments.

[0020] FIG. 9 is a schematic diagram illustrating one example transfer learning strategy for the convolutional neural network of FIG. 2, in embodiments.DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In the following description, certain specific details are set forth in order to provide a thorough understanding of various disclosed embodiments. However, one skilled in the relevant art will recognize that embodiments may be practiced without one or more of these specific details, or with other methods, components, materials, etc.

[0022] Unless the context requires otherwise, throughout the specification and claims which follow, the word “comprise” and variations thereof, such as, “comprises” and “comprising” are to be construed in an open, inclusive sense that is as “including, but not limited to.”

[0023] Reference throughout this specification to “one implementation” or “an implementation” or “one embodiment” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one implementation or embodiment. Thus, the appearances of the phrases “one implementation” or “an implementation” or “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same implementation or embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more implementations or one or more embodiments.

[0024] As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. It should also be noted that the term “or” is generally employed in its sense including “and / or” unless the context clearly dictates otherwise.Purpose:

[0025] Fractional radiotherapy is a treatment method where radiation is delivered to a tumor in a subject in small doses called fractions. Each fraction is typically given once a day, five days a week, over several weeks. This approach allows time for normal cells to repair themselves between treatments, reducing side effects. Each fraction involves directing a radiation beam, possibly from multiple different directions, at the tumor within the subject to deliver a specified dose of radiation (e.g., 266 cGy). Consistent positioning of the subject relative to the radiation beam is critical to ensure the radiation beam affects the tumor with minimal effect on other tissue. Repeatable consistency in positioning of the subject, and particularly the tumor, is critical at (a) the start of each fraction of the treatment (e.g., inter-fraction), and (b) during the radiation (e.g., intra-fraction). Physical positioning of the subject is usually based on rigid components (e.g., bone structure) of the subject; however, soft tissue is less positionable.

[0026] The systems and methods disclosed herein ensure that the subject positioning variances — both inter- and intra-fraction — are minimized and that precise radiotherapy doses are delivered effectively. By capturing Cherenkov images during the radiotherapy and detecting bio-morphological features (e.g., vasculature, blood vessels, etc.) in the Cherenkov images, the systems and methods quantify tissue deformation and positional variations of a treatment area of the subject. Treatment outcome is then improved by allowing brief treatment interruptions to reposition the subject when deformation and / or positional variations are detected, or by changing a delivery angle of the radiation beam to compensate for detected deformation and / or positional variations are detected.

[0027] The method involves segmenting bio-morphological features (e.g., vasculature, blood vessels, etc.) in the Cherenkov images and employing both rigid and non- rigid registration techniques to quantify positional differences in deformable tissue. These systems and methods have been validated with submillimeter precision using an anthropomorphic chest phantom to simulate radiotherapy treatment, including simulated patient movement and breathing dynamics.Basics of Cherenkov-Imaging

[0028] FIG. 1 is a schematic diagram illustrating one example Cherenkov-imagingbased system 100 for verifying position of deformable tissue undergoing radiotherapy, in embodiments. FIG. 2 is a schematic illustrating further example detail of controller 101 of FIG. 1, in embodiments. FIGs. 1 and 2 are best viewed together with the following description.

[0029] A radiotherapy machine 150 is positioned in a treatment room 151 and includes a linear accelerator (LINAC) 108 with a positioning apparatus 132 for delivering a radiation beam 110 (e.g., high-energy radiation). Radiotherapy machine 150 also includes a positioning table 130 for supporting and positioning a subject 102 receiving radiotherapy. Radiotherapy machine 150 includes a control apparatus 152 that controls operation of positioning apparatus 132 and LINAC 108. For example, control apparatus 152 controls patient positioning table 130, positioning apparatus 132, and LINAC 108 to cause radiation beam 110 to pass through normal tissue 112 of subject 102 and to a tumor 104 of subject 102. In one example of operation, subject 102 is positioned on patient positioning table 130 andcontrol apparatus 152 controls patient positioning table 130 and / or positioning apparatus 132 to adjust a position of subject 102 such that radiation beam 110 intersects tumor 104 from various angles. In one example, positioning apparatus 132 is controlled to rotate LINAC 108 in an arc over or around subject 102. In embodiments where system 100 and radiotherapy machine 150 are combined, controller 101 controls one or more of patient positioning table 130, positioning apparatus 132, and LINAC 108 during the treatment of subject 102.

[0030] Since high radiation doses are desired in tumors, while high doses are not desired in surrounding normal tissue or on skin, to prevent damaged by radiation control apparatus 152 controls one or both of patient positioning table 130 and positioning apparatus 132 to vary arriving beam delivery angles of radiation beam 110 relative to subject 102 by rotating subject 102 (and enclosure 106 if used) relative to radiation beam 110, rotating LINAC 108 about subject 102 (and enclosure 106 if used), and / or by periodically interrupting treatment to reposition subject 102 (and enclosure 106 if used). In certain embodiments, radiation beam 110 has a static shape. In other embodiments, LINAC 108 is controlled to change a shape of radiation beam 110 dynamically as LINAC 108 and / or subject 102 rotates, to allow customized dose delivery based on a shape of tumor 104 and a current delivery angle. As radiation beam 110 penetrates subject 102, Cherenkov light is emitted from a surface of subject 102 at treatment area 105 (e.g., where radiation beam 110 intersects tissue of subject 102).Verifying Position Of Deformable Tissue

[0031] System 100 includes a controller 101, a console 134, and at least one camera 117(1) and optionally a second camera 117(2). Controller 101 includes a processor 218 and a memory 219 storing software 224. Software 224 includes machine-executable instructions that, when executed by processor 218, cause controller 101 to control operation of system 100 to monitor position of deformable tissue of a subject 102 during radiotherapy. In certain embodiments, system 100 is an add-on to radiotherapy machine 150 and includes at least one timing interface 120 that detects operation of LINAC 108, either directly from LINAC 108 or from control apparatus 152. In certain embodiments, timing interfaces 120 include at least one radiation detector for detecting radiation beam 110. In other embodiments, system radiotherapy machine 150 and system 100 are combined into a single improved radiotherapy system. For example, controller 101 and control apparatus 152 are combined as a single controller that controls operation of LINAC 108, cameras 117, patient positioning table 130, and positioning apparatus 132. In certain embodiments, controller 101, console 134, and / orcontrol apparatus 152 are positioned in a control room that is separate from treatment room 151. Subject 102 and cameras 117 are preferably located in a darkened or dimmed environment. In one example, treatment room 151 is a darkened or dimmed room. In another example, cameras 117 and at least the part of subject 102 receiving radiotherapy are positioned within an enclosure 106.

[0032] As shown in FIG. 1, camera 117(1) is positioned with a field-of-view of a treatment area 105 of subject 102 lying on patient positioning table 130. When included, camera 117(2) is positioned with a field-of-view of treatment area 105 from a different angle. Cameras 117 may each include a lens system adapted to collect light from treatment area 105 of subject 102 and a photosensor array for detecting the collected light, allowing camera 117 to be positioned some distance from subject 102. In certain embodiments, cameras 117(1) and 117(2) are fixed to a ceiling of a treatment room and are angled toward the left and right sides of treatment area 105 to capture the Cherenkov light emission from a surface of subject 102 during transmission of radiation beam 110. Controller 101 controls camera 117(1), and camera 117(2) when included, to capture a Cherenkov image 226 of treatment area 105 during output of radiation beam 110 from LINAC 108. In one example of operation, controller 101 detects output of controls LINAC 108 to generate and direct a radiation beam 110 to administer radiotherapy to tumor 104 in subject 102 lying on patient positioning table 130. FIG. 3 shows one example Cherenkov image 226 of FIG. 2, in embodiments. Cherenkov image 226 shows tissue 302 with vasculature 304.

[0033] In certain embodiments, system 100 includes two cameras 117(1) and 117(2) positioned to capture stereo Cherenkov images 226 of treatment area 105 during operation of LINAC 108. Controller 101 may then process the stereo Cherenkov images to determine a three-dimensional (3D) rigid displacement and a 3D nonrigid displacement of deformable tissue in treatment area 105.

[0034] In certain embodiments, system 100 includes an enclosure 106 that excludes ambient light and is positioned to include cameras 117 and portions of a subject 102 near tumor 104. Enclosure 106 may be made of black plastic or cloth and may have a sealing portion 109 that is held against subject 102 (e.g., by an elastomeric band). Advantageously, eyes of subject 102 are permitted access to ambient light, which may prevent claustrophobia while excluding room light from treatment area 105 for improved Cherenkov imaging. Enclosure 106 may be one of a variety of forms according to the location of tumor 104 within subject 102, desired beam angles for treating tumor 104, and desired angles of cameras 117 relative to treatment area 105. In one example. Enclosure 106 is sized and positioned toexclude ambient light from a cranium of subject 102. In another example, enclosure 106 is sized and positioned to exclude ambient light from a chest or abdomen of subject 102. In another example, enclosure 106 is an entire room containing system 100 and subject 102.

[0035] Cameras 117(1) and 117(2) have high sensitivity and are located outside of radiation beam 110 and positioned to capture Cherenkov radiation from treatment area 105. Where enclosure 106 is used, cameras 117 are positioned within enclosure 106. Cameras 117 are sensitive enough to image Cherenkov light emitted from tissue of subject 102 during an interaction of radiation beam 110.

[0036] In certain embodiments, where control apparatus 152 controls LINAC 108 to generate radiation beam 110 in pulses (e.g., 4ms pulses), timing interface 120 receives a control signal indicative of operation of radiation beam 110 and controls pulsed lighting 122 to cause at least one light source 107 of treatment room 151 to be on only when radiation beam 110 is off. Accordingly, pulsing of light source 107 is synchronized with operation of LINAC 108 and radiation beam 110 to avoid interference from room lighting. This takes advantage of visual persistence of the human eye, which causes light source 107 to appear on continuously and thereby may improve comfort of subject 102.

[0037] Controller 101 receives timing information of radiation beam 110 and pulsed room lighting 122 from timing interfaces 120 and controls operation of cameras 117 to capture a Cherenkov image 226 of Cherenkov radiation from treatment area 105 when light source 107 is off and radiation beam 110 is on. Accordingly, acquisition of Cherenkov images 226 is synchronized with operation of both LINAC 108 and light source 107. In one example of operation, cameras 117(1) and 117(2) are synchronized and time-gated to 4- microsecond pulses of radiation beam 110 such that Cherenkov images 226 are captured during each pulse of radiation beam 110. In certain embodiments, each Cherenkov image226 is an accumulation of multiple images captured by one camera 117 and summed together.

[0038] In an alternative embodiment, where light source 107 remains on continuously (e.g., not pulsed, or pulsed but not synchronized with radiation beam 110) but may be subdued, Cherenkov image 226 is captured when radiation beam 110 is on, but also captures ambient lighting. Controller 101 then controls cameras 117 to capture a background image227 during an immediately subsequent period when radiation beam 110 is off (e.g., for an equivalent shutter period) such that background image 227 captures ambient lighting alone. Background image 227 may be processed by median filtering and dark field corrections that respectively remove speckle and electronic noise. Controller 101 then subtracts backgroundimage 227 from Cherenkov images 226, such that ambient light is removed from Cherenkov image 226. This is known as background image subtraction. Cherenkov image 226 is then used for further processing.

[0039] In certain embodiments, controller 101 may also control cameras 117 to capture background image 227 when both radiation beam 110 and light source 107 are off (e.g., when pulsed room lighting 122 turns light source 107 off), whereby background image 227 captures any background noise from uncontrolled ambient light within treatment room 151. Background image 227 may be processed by median filtering and dark field corrections that respectively remove speckle and electronic noise. Background noises are then suppressed within Cherenkov images 226 by subtracting background image 227 from Cherenkov image 226. Cherenkov image 226 is then used for further processing. In embodiments, Cherenkov images 226, either raw or post processed (e.g., de-noised), captured by cameras 117 are recorded in memory 219 (and / or any other suitable digital memory) as documentation of the radiotherapy.

[0040] Radiotherapy machine 150 may control LINAC 108 to generate radiation beam 110 with a beam energy of at least 800 keV because, at beam energies of less than 800 keV (0.2 MeV), Cherenkov radiation is typically of insufficient intensity for imaging. In certain embodiments, radiation beam 110 is a beam of electrons having energy of 6 million electron volts (6 MeV) or greater. In certain embodiments, radiation beam 110 has energy of between 6 and 24 MeV. In certain embodiments, radiation beam 110 is a beam of high- energy photons, where the photons interact with tissue and / or tumor within treatment area 105 to produce charged particles that in turn produce Cherenkov radiation. In certain embodiments, radiation beam 110 is a high-energy proton beam. In another embodiment, a radiation source is implanted within subject 102, inducing Cherenkov emission light directly as charged particles are emitted during radiation decay.

[0041] Controller 101 receives Cherenkov images 226 and background images 227 from camera 117 and stores Cherenkov images 226 and background images 227 in memory 219. In certain embodiments, controller 101 includes one or more algorithms to analyze Cherenkov image 226 and generate indications of heme concentration in tumor, oxygen concentration in tumor, and other parameters (such as metabolic activity and oxygenation) defined by photoluminescent emission captured in Cherenkov images 226.

[0042] In certain embodiments, cameras 117 are enabled in a temporal relationship after each pulse of radiation beam 110 to capture emissions from Cerenkov excited luminescence, such as phosphorescence or fluorescence. Accordingly, these images includelight emitted from a secondarily-emitting chemical or indicating agent, such as a fluorescent or phosphorescent chemical, in or on the tissue and stimulated by Cherenkov radiation emitted as radiation beam 110 interacts with tissue of subject 102. This time-sequenced signal is generated from the Cherenkov radiation but may provide information regarding the bio-molecular environment of the tissue being irradiated. The secondarily emitting chemical in some embodiments is a chemical intrinsic to the body, and in some embodiments is a drug, or a metabolite of a prodrug, that is administered to the subject.

[0043] Radiotherapy involves LINAC 108 generating radiation beam 110 (e.g., a beam of high-energy charged particles) aimed at tumor 104 to deliver a radiation dose to the tumor. In certain embodiments, subject 102 is rotated during the session to distribute radiation absorbed by normal tissues 112 while maintaining targeting of radiation beam 110 at tumor 104. As charged particles of radiation beam 110 decelerate in both normal tissue 112 and tumor 104, these particles generate light by Cherenkov radiation, with broadband spectral constituents decreasing with wavelength to the inverse square power.

[0044] Some of the light generated by Cherenkov radiation is absorbed by fluorophores (or phosphors) within subject 102, including fluorophores (or phosphors) within normal tissue 112 and tumor 104. Light from Cherenkov radiation that is absorbed by fluorophores (or phosphors) in tissue and tumor may stimulate photoluminescent emission by those tissues and tumor.

[0045] Light from both Cherenkov radiation and photoluminescent emission propagates from the beam path to a surface of subject 102, intersecting any tissue between the tumor and the surface, and being attenuated by absorption from molecular absorbers such as deoxyhemoglobin, oxyhemoglobin, proteins, lipids and water before being emitted from a surface of subject 102 at treatment area 105.

[0046] When treating many soft tissue tumors, including breast tumors, a dominant absorption is from deoxyhemoglobin and oxyhemoglobin, which differ in their spectral absorption, and so changes in spectral characteristics of the attenuated light emitted from the subject are a reasonable measure of oxygen saturation of the blood in the region. Hemoglobin, including oxyhemoglobin and deoxyhemoglobin, is concentrated in blood vessels, or vasculature. Lesser concentrations of hemoglobin are found in vascularized tissues, including many tumors. It has been found that subcutaneous vasculature near a surface of a subject absorbs sufficient Cherenkov light to be visible in Cherenkov images 226 captured by cameras 117 during radiation treatment. Myoglobin is also found in some tissues, particularly muscle tissue, and may also absorb Cherenkov radiation.

[0047] It has been observed that a pattern of bio-morphological features (e.g., near- surface subcutaneous vasculature) is unique to each person, in a manner similar to the uniqueness of fingerprints. As such, the bio-morphological features may be used to identify particular subjects.

[0048] In certain embodiments, controller 101 quantifies Cherenkov emission in a tumor and determines total hemoglobin from an amount of light measured at one or more wavelengths to which oxyhemoglobin and deoxyhemoglobin are isosbestic. In this embodiment, controller 101 includes diffuse optical modeling or Monte Carlo modeling software that when executed by processor 218 allows reconstruction of a shape and spectral characteristics of treatment area 105, and allows spectral characteristics of light emitted within tumor 104 ,as opposed to light emitted elsewhere (e.g., in normal tissue 112), to be determined by compensating for changes due to light transport in surrounding tissues.

[0049] In one example of operation, signals from timing interface 120 allow controller 101 to control an effective shutter of cameras 117 to effectively capture only light received during an interval during and / or surrounding pulses of radiation beam 110 and / or during a fluorescent decay interval after pulses of radiation beam 110. Timing interface 120 may also control pulsed room lighting 122 such that the shutter interval does not overlap pulses of light source 107. Light received at the camera 117 during multiple shutter intervals may be totalized, in an embodiment at the camera, and in an alternative embodiment, multiple images are captured, and per-channel spectrographic light totals are totalized by processor 218.

[0050] In an embodiment, LINAC 108 provides a beam of electrons having energy of 6 million electron volts (6 MeV) or greater, as used to provide treatment energy to deep tumors as opposed to treatment of surface skin. In a particular embodiment, the beam energy is between 6 and 24 MeV. In an alternative embodiment, LINAC 108 produces a photon beam of 6 MeV or greater. In another alternative embodiment, LINAC 108 provides a high- energy proton beam. In an alternative embodiment, an electron beam having electron energy of 0.5 MeV or greater is used.Bio-morphological Feature Segmentation

[0051] Controller 101 includes a segmentation algorithm 234 (shown as part of software 224) that processes Cherenkov image 226 to generate a bio-morphological feature image 240 defining bio-morphological features (e.g., subcutaneous vessels within tissue at treatment area 105) of subject 102. Segmentation algorithm 234 may use at least onethreshold 235 to distinguish the bio-morphological features. In certain embodiments, segmentation algorithm 234 may also allow manual adjustments of at least one threshold 235 to achieve a desired segmentation performance.

[0052] In other embodiments, controller 101 invokes a convolutional neural network (CNN) 128 with a trained model 229 that segments bio-morphological features from Cherenkov images 226 with improved consistency and accuracy. In one embodiment, CNN 228 has a ResNet architecture developed for biomedical image segmentation tasks. This ResNet architecture has a distinctive architecture that enables precise localization and segmentation of objects within images, even with limited training data. In another embodiment, CNN 228 is adapted from VoxelMorph, a known neural network architecture for 2D imaging domain applications.

[0053] FIG. 9 is a schematic diagram illustrating one example transfer learning strategy 900 for CNN 228 of FIG. 2, in embodiments. Due to the limited ground-truth bio- morphological features labels available in training datasets for Cherenkov imagery, transfer learning strategy 900 was employed to train CNN 228. CNN 228 is first trained for vessel segmentation using a retinal vessel dataset 902 to generate pre-trained weights 904. Retinal vessel dataset 902 is the color fundus image vessel segmentation (FIVES) dataset, which contains eight hundred high-resolution multi-disease color fundus photographs with pixelwise retinal vessel annotation. Then, starting with pre-trained weights 904, CNN 228 is further trained using a Cherenkov dataset 906 that includes annotated vasculature masks. Accordingly, model 229 is formed using transfer learning strategy 900, which allows CNN 228 and model 229 to learn how to segment more specific features from Cherenkov images 226.

[0054] In certain embodiments, CNN 228 has nine layers with 4: 1 compression in middle layers. CNN 228 and model 229 achieve consistent and rapid segmentation of bio- morphological features in Cherenkov images 226, where the bio-morphological features include subcutaneous veins, scars, and pigmented skin. CNN 228 and model 229 allow the bio-morphological features to be extracted consistently in frame-to-frame, real-time, Cherenkov imaging (e.g., where cameras 117 provide a continuous video output). Advantageously, system 100 provides quantitative tracking of position and motion of subject 102 to improve effectiveness and outcome of radiotherapy.Reference Image

[0055] Segmentation algorithm 234 may also be used to generate a reference image 242 from one Cherenkov image 226 captured when subject 102 was correctly positioned. Reference image 242 may be generated from a first captured Cherenkov image 226 of subject 102 during an initial radiotherapy treatment. For example, subject 102 is carefully positioned and aligned to radiotherapy machine 150 using multiple techniques as known in the art. Then, during a first radiotherapy treatment of subject 102, a first Cherenkov image 226 is captured and processed by segmentation algorithm 234 to generate reference image 242. Accordingly, reference image 242 defines bio-morphological features of subject 102 when subject 102 is positioned correctly on radiotherapy machine 150.

[0056] FIG. 4 shows reference image 242 of FIG. 2 overlaid with a subsequently captured bio-morphological feature image 240 to illustrate displaced vasculature, in embodiments. In the example of FIG. 4, reference image 242 is represented as dark lines and shows reference vasculature 404, whereas bio-morphological feature image 240 is represented in grey and shows displaced vasculature 406 after rigid registration and before non-rigid registration. In the example of FIG. 4, displaced vasculature 406 of bio- morphological feature image 240 does not fully align with reference vasculature 404 of reference image 242.Registration

[0057] After generating bio-morphological feature image 240, controller 101 invokes a registration algorithm 236 that implements a rigid / non-rigid combined registration technique to determine both inter-fractional and intra-fractional positioning variations of subject 102. That is, registration algorithm 236 registers bio-morphological feature image 240 with reference image 242 to determine a transformation matrix representative of registration differences. Registration algorithm 236 implements a rigid registration technique that determines a rigid displacement 244 from the transformation matrix, where rigid displacement 244 defines a global shift between reference vasculature 404 and displaced vasculature 406. For example, registration algorithm 236 may first minimize the mean squared error between reference vasculature 404 and displaced vasculature 406 of bio- morphological feature image 240 and reference image 242, respectively. Registration algorithm 236 may then use a regular step gradient descent optimization technique to determine the transformation matrix and rigid displacement 244 through translation androtation. Rigid registration is a known technique and may be similar to using the imregister function in MATLAB.

[0058] Registration algorithm 236 implements a non-rigid registration technique that constructs a deformation map 246 that indicates locoregional deformation between bio- morphological feature image 240 and reference image 242, and thereby locoregional deformation of deformable tissue in treatment area 105. FIG. 5 shows one example grid representation 500 of deformation map 246 of FIG. 2, in embodiments. FIG. 6 is a schematic diagram representing operation of registration algorithm 236 to determine rigid displacement 244 and deformation map 246 using rigid and non-rigid registration, in embodiments. FIGs. 4, 5, and 6 are best viewed together with the following description.

[0059] For example, registration algorithm 236 uses a pixel-based B-spline grid-based registration algorithm to construct a grid of basis spline control points that control the transformation of reference image 242 to bio-morphological feature image 240 using a squared pixel distance as a similarity measure to assess discrepancy between bio- morphological feature image 240 and reference image 242. Registration algorithm 236 may use a fast optimization approach with a limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithm, a quasi-Newton method, that iteratively adjusts control points to minimize registration errors until deformation map 246 defines a best alignment between bio- morphological feature image 240 and reference image 242 is achieved.

[0060] With the combined scheme of rigid and non-rigid registration, both rigid displacement 244 (e.g., global shift) and deformation map 246 (e.g., locoregional deformation) are quantified from the two types of registration methods, respectively. In detail, rigid displacement 244 indicating the shift in x and y direction was extracted from a transformation matric generated by the rigid registration technique.

[0061] As shown in FIG. 6, a rigid registration is first applied to bio-morphological feature image 240 and reference image 242 to determine rigid displacement 244, which results in improved alignment of bio-morphological features as shown in rigid aligned image 602. Non-rigid registration is then applied to generate deformation map 246 (illustrated as deformation output image 248 - see FIG. 7) with a resulting improved alignment of bio- morphological features as shown in as shown in non-rigid aligned image 604.

[0062] FIG. 7 shows one example deformation output image 248 generated by registration algorithm 236 of FIG. 2 to simultaneously illustrate both rigid displacement 244 and deformation map 246 of tissue in treatment area 105, in embodiments. Deformation output image 248 is a two-dimensional correction guidance map that allows an operator ofradiotherapy machine 150 to easily see displacement of tissue within treatment area 105 relative to a reference position defined by reference image 242. Registration algorithm 236 generates deformation output image 248 to include Cherenkov image 226 (e.g., the Cherenkov image of the deformed tissue), rigid displacement 244 represented as arrows 702 (e.g., orthogonal vectors) positioned in a corner to indicate the global shift, and deformation map 246 represented as an array of two-dimensional vectors that represent pixel-level locoregional tissue deformation. As shown in FIG. 7, deformation output image 248 may also include digital representations 704 of rigid displacement 244 and contour lines 706 that are curves joining locoregional deformation magnitudes of the same value. The magnitude of quantified variations from the registration-based methodology is converted from units of pixels to physical dimension (e.g., millimeters) based on the optical properties and positioning of cameras 117. Deformation output image 248 may be output on console 134 (or any other device) during operation of system 100 and radiotherapy machine 150. Advantageously, system 100 and deformation output image 248 provide clear guidance to an operator of radiotherapy machine 150 during operation, allowing the operator to halt a treatment and correct patient positioning as needed. System 100 thereby also improves output of the radiotherapy treatment. System 100 works well with radiotherapy machine 150 to provide improved radiation treatment of human and animal soft tissues, including human breast tissues, and may also function with radiation treatment of abdominal tissues such as but not limited to liver, pancreas, stomach, and intestinal tissues.Summary of Results:

[0063] Testing of system 100 demonstrated an accuracy within 0.83 ± 0.49 mm for of determined displacements with simulated treatment variances. A retrospective analysis of a previously captured Cherenkov imaging dataset for ten breast cancer patients indicated an inter-fraction setup uncertainty of 3.71 ± 2.36 mm, with significant improvements observed in locoregional deformation and quantification post-application of the combined registration method.

[0064] In an embodiment, displacements determined from the rigid and non-rigid registrations are compared to limits, and treatment is stopped to permit adjustment of subject position if these displacements exceed limits. In certain embodiments, at least part of one or both of rigid displacement 244 and deformation map 246 are sent to radiotherapy machine 150, where control apparatus 152 controls patient positioning table 130 and / or positioning apparatus 132 to correct the positioning error.Method

[0065] FIG. 8 is a flowchart illustrating one example method 800 for verifying position of deformable tissue undergoing radiotherapy, in embodiments. Method 800 may be implemented at least in part by controller 101 of FIG. 1, and in particular by CNN 228 and registration algorithm 236 of software 224 when executed by processor 218, for example. Although method 800 is directed towards use of CNN 228 for segmenting, method 800 may also use segmentation algorithm 234 in place of CNN 228.

[0066] In block 802, method 800 trains a neural network to segment images of vasculature. In one example of block 802, CNN 228 is first trained for vessel segmentation using a retinal vessel dataset, forming model 229, which is then fine-tuned using a Cherenkov dataset with known annotated vasculature masks.

[0067] In block 804, method 800 captures a Cherenkov image of Cherenkov emissions from a subject receiving radiotherapy. In one example of block 804, radiotherapy machine 150 initiates radiation beam 110 to treat tumor 104 of subject 102 and controller 101 controls cameras 117 to capture Cherenkov image 226 of treatment area 105. In block 806, method 800 segments the image to isolate vasculature and form a bio-morphological feature image. In one example of block 806, software 224 inputs Cherenkov image 226 into CNN 228, which outputs bio-morphological feature image 240.

[0068] In block 808, method 800 saves the bio-morphological feature image as a reference image when no reference image exists. In one example of block 808, software 224 saves bio-morphological feature image 240 as reference image 242. Where the radiotherapy is a first fraction for subject 102 (or where a previous reference image has not been saved), bio-morphological feature image 240 is also stored as reference image 242 (e.g., in a database in association with subject 102).

[0069] In block 810, method 800 performs a rigid registration of vasculature to recorded vasculature. In one example of block 810, registration algorithm 236 performs a rigid registration of bio-morphological feature image 240 against reference image 242 to generate rigid displacement 244.

[0070] In block 812, method 800 performs a non-rigid registration of vasculature to recorded vasculature. In one example of block 812, registration algorithm 236 performs a non-rigid registration of bio-morphological feature image 240 and / or rigid displacement 244 against reference image 242 to generate deformation map 246.

[0071] In block 814, method 800 verifies the subject and / or treated region identity. In one example of block 814, registration algorithm 236 determines whether the registration of bio-morphological feature image 240 with reference image 242 was successful or not, where an unsuccessful registration indicates an incorrect subject or an incorrect treated region. Where the identity is incorrect (e.g., no registration) method 800 continues with block 816. Otherwise, method 800 continues with block 818.

[0072] In block 816, method 800 aborts treatment. In one example of block 816, registration algorithm 236 invokes an alarm generator 238 that generates an alert on console 134 (or elsewhere by sending a message or text) to indicate that one of a positioning error or a subject identity error has occurred. Advantageously, by detecting a mismatch between bio- morphological features of subject 102 and previously captured reference image 242 (e.g., stored in a database and associated with a patient identity), system 100 may prevent incorrect treatment of subject 102. Similarly, where an incorrect portion of subject 102 is positioned for treatment (e.g., the wrong breast when performing radiation treatments of breast cancer), the error is detected and further treatment is prevented. Where alarm generator 238 is invoked and an alert generated, the operator of radiotherapy machine 150 may verify the identity of the subject and area being treated.

[0073] In block 818, method 800 compares the displacement of rigid registration and non-rigid registration to limits. In one example of block 818, registration algorithm 236 compares rigid displacement 244 and deformation map 246 against a deformation limit 237. In block 820, method 800 pauses treatment to adjust the subject’s position if needed. In one example of block 820, where block 818 indicates alignment indicated by rigid displacement 244 and deformation map 246 is greater than deformation limit 237, registration algorithm 236 displays a message on console 134 (or sends a message to an operator) to indicate the alignment error and to request repositioning of subject 102.

[0074] Blocks 804 through 820 repeat for each fraction.Conclusion

[0075] Advantageously, system 100 detects imprecision in patient positioning during radiotherapy, which, when undetected, may result in ineffective dose delivery and potential damage to healthy tissue. By determining both global shifts (e.g., rigid displacement 244) and locoregional deformations (e.g., deformation map 246) accurately, system 100 enhances the precision of dose delivery to targeted tissue of subject 102.

[0076] System 100 has may advantages over existing technologies, including: (1) better treatment outcomes through enhanced precision, by quantifying both global and locoregional deformations to ensure higher precision in positioning of subject 102 as compared to prior techniques; (2) real-time monitoring and providing real-time imaging guidance by generating deformation output image 248 to inform and adjust subject positioning during radiotherapy sessions, unlike many prior technologies that only allow for pre- or post-treatment imaging; (3) non-invasiveness by using Cherenkov imaging, which is non-invasive and thereby avoids additional radiation exposure or discomfort to the subject being treated.

[0077] By focusing on the direct observation and quantification of locoregional deformations, system 100 represents a significant advancement over prior technology, which generally lacks the means to account for such detailed physiological changes during the course of treatment.

[0078] Changes may be made in the above methods and systems without departing from the scope hereof. It should thus be noted that the matter contained in the above description or shown in the accompanying drawings should be interpreted as illustrative and not in a limiting sense. The following claims are intended to cover all generic and specific features described herein, as well as all statements of the scope of the present method and system, which, as a matter of language, might be said to fall therebetween.Combination of Features

[0079] Features described above as well as those claimed below may be combined in various ways without departing from the scope hereof. The following enumerated examples illustrate some possible, non-limiting combinations:

[0080] (Al) A system for verifying positioning of deformable tissue during radiotherapy of a subject, includes: a first camera configured to capture a Cherenkov image of a target area of the subject during the radiotherapy; and a controller having a processor and a memory storing software executable by the processor to cause the controller to: segment the Cherenkov image to form a bio-morphological feature image of bio-morphological features of the deformable tissue; and register the bio-morphological feature image to a reference image of the bio-morphological features to determine (a) a rigid displacement and (b) a nonrigid displacement.

[0081] (A2) In embodiments of (Al), the memory further includes software that, when executed by the processor, cause the controller to invoke a trained neural network to segment the Cherenkov image to form the bio-morphological feature image.

[0082] (A3) In either of embodiments (Al) or (A2), the memory further includes software that, when executed by the processor, cause the controller to use an image processing algorithm and a threshold to segment the Cherenkov image to form the bio-morphological feature image.

[0083] (A4) In any of the embodiments (A1)-(A3), the rigid displacement defining a two-dimensional shift in the bio-morphological features between the bio-morphological feature image and the reference image.

[0084] (A5) In any of the embodiments (A1)-(A4), the nonrigid displacement defining a two-dimensional locoregional deformation of the bio-morphological features.

[0085] (A6) Any of the embodiments (A1)-(A5) further include a second camera configured to capture a second Cherenkov image of the target area, the memory further comprising software that, when executed by the processor, cause the controller to process the Cherenkov image and the second Cherenkov image to generate a three-dimensional (3D) rigid displacement and a 3D nonrigid displacement.

[0086] (A7) Any of the embodiments (A1)-(A6) further include a console, the memory further storing software that, when executed by the processor, causes the controller to display information of at least one of the rigid displacement and the nonrigid displacement on the console during the radiotherapy.

[0087] (A8) In any of the embodiments (A1)-(A7), the memory further includes software that, when executed by the processor, cause the controller to display an alert on the console to indicate an alignment error.

[0088] (A9) In any of the embodiments (A1)-(A8), the memory further includes software that, when executed by the processor, cause the controller to display an alert on the console to indicate an identity error.

[0089] (Bl) A method for verifying position of deformable tissue undergoing radiotherapy, including: capturing a Cherenkov image of a treatment area of a subject during the radiotherapy; segmenting the Cherenkov image to generate a bio-morphological feature image defining vasculature of the deformable tissue; performing a rigid registration of the bio- morphological feature image and a reference image to determine a rigid displacement; and performing a non-rigid registration of the bio-morphological feature image and the referenceimage to determine a deformation map indicating locoregional deformation of the deformable tissue.

[0090] (B2) Embodiments of (Bl) further include sending an alert to an operator performing the radiotherapy when registration of the bio-morphological feature image with the reference image is unsuccessful, the alert indicating the subject is not a prior subject of the reference image.

[0091] (B3) In either of embodiments (Bl) or (B2), the reference image is a prior bio- morphological feature image determined from a first Cherenkov image captured of the subject during a first fraction of the radiotherapy.

[0092] (B4) Any of embodiment (B1)-(B3) further include sending an alert to an operator performing the radiotherapy when the locoregional deformation is greater than a deformation threshold, the alert indicating the subject needs repositioning.

[0093] (B5) Any of embodiment (B1)-(B4) further include adjusting a subject positioning table to reposition the subject when the locoregional deformation is greater than a deformation threshold.

[0094] (B6) Any of embodiment (B1)-(B5) further include displaying the deformation map to an operator performing the radiotherapy.

[0095] (B7) Any of embodiment (B1)-(B6) further include invoking a neural network to generate the bio-morphological feature image from the Cherenkov image.

[0096] (B8) Any of embodiment (B1)-(B7) further include invoking a neural network to perform the non-rigid registration of the bio-morphological feature image with the reference image to generate the deformation map.

Claims

CLAIMSWhat is claimed is:

1. A system for verifying positioning of deformable tissue during radiotherapy of a subject, comprising: a first camera configured to capture a Cherenkov image of a target area of the subject during the radiotherapy; and a controller having a processor and a memory storing software executable by the processor to cause the controller to: segment the Cherenkov image to form a bio-morphological feature image of bio-morphological features of the deformable tissue; and register the bio-morphological feature image to a reference image of the bio- morphological features to determine (a) a rigid displacement and (b) a nonrigid displacement.

2. The system of claim 1, the memory further comprising software that, when executed by the processor, cause the controller to invoke a trained neural network to segment the Cherenkov image to form the bio-morphological feature image.

3. The system of claim 1, the memory further comprising software that, when executed by the processor, cause the controller to use an image processing algorithm and a threshold to segment the Cherenkov image to form the bio-morphological feature image.

4. The system of claim 1, the rigid displacement defining a two-dimensional shift in the bio-morphological features between the bio-morphological feature image and the reference image.

5. The system of claim 1, the nonrigid displacement defining a two-dimensional locoregional deformation of the bio-morphological features.

6. The system of claim 1, further comprising a second camera configured to capture a second Cherenkov image of the target area, the memory further comprising software that, when executed by the processor, cause the controller to process the Cherenkov image and the second Cherenkov image to generate a three-dimensional (3D) rigid displacement and a 3D nonrigid displacement.

7. The system of claim 1, further comprising a console, the memory further storing software that, when executed by the processor, causes the controller to display information of at least one of the rigid displacement and the nonrigid displacement on the console during the radiotherapy.

8. The system of claim 7, the memory further comprising software that, when executed by the processor, cause the controller to display an alert on the console to indicate an alignment error.

9. The system of claim 7, the memory further comprising software that, when executed by the processor, cause the controller to display an alert on the console to indicate an identity error.

10. A method for verifying position of deformable tissue undergoing radiotherapy, comprising: capturing a Cherenkov image of a treatment area of a subject during the radiotherapy; segmenting the Cherenkov image to generate a bio-morphological feature image defining vasculature of the deformable tissue; performing a rigid registration of the bio-morphological feature image and a reference image to determine a rigid displacement; and performing a non-rigid registration of the bio-morphological feature image and the reference image to determine a deformation map indicating locoregional deformation of the deformable tissue.

11. The method of claim 10, further comprising sending an alert to an operator performing the radiotherapy when registration of the bio-morphological feature image with the reference image is unsuccessful, the alert indicating the subject is not a prior subject of the reference image.

12. The method of claim 10, wherein the reference image is a prior bio-morphological feature image determined from a first Cherenkov image captured of the subject during a first fraction of the radiotherapy.

13. The method of claim 10, further comprising sending an alert to an operator performing the radiotherapy when the locoregional deformation is greater than a deformation threshold, the alert indicating the subject needs repositioning.

14. The method of claim 10, further comprising adjusting a subject positioning table to reposition the subject when the locoregional deformation is greater than a deformation threshold.

15. The method of claim 10, further comprising displaying the deformation map to an operator performing the radiotherapy.

16. The method of claim 10, further comprising invoking a neural network to generate the bio-morphological feature image from the Cherenkov image.

17. The method of claim 10, further comprising invoking a neural network to perform the non-rigid registration of the bio-morphological feature image with the reference image to generate the deformation map.

Images