Systems and methods for determining the location of a patient's gross target volume.

Abstract

This invention relates to systems and methods for determining the location of a gross target volume in a patient. In an example, the system includes one or more processors configured to acquire image data associated with a plurality of two-dimensional (2D) images. The one or more processors may be configured to determine the location of one or more lesions at a first time point and, based on the location of the one or more lesions and a trajectory representing the expected motion of the one or more lesions, determine the future location of the one or more lesions at a second time point. In an example, the one or more processors may be configured to generate control signals based on the future location of the one or more lesions to move a linear accelerator.

Description

Technical Field

[0001] This application generally relates to systems and methods for predicting the location of a gross target volume within a patient during radiotherapy, and in some embodiments relates to systems and methods for predicting the location of a gross target volume within a patient during radiotherapy by modeling the movement of one or more lesions of the gross target volume over time. Background Technology

[0002] Radiation therapy (also known as "radiotherapy" or "RT") involves delivering radiation (energy) to a target within a patient's body during treatment. For example, a linear accelerator (LINAC) can be configured to move relative to the patient at multiple control points according to an RT treatment plan and deliver energy to target tissues (such as tumors, lesions, etc.) to treat cancer located within the patient's body. This allows energy to be delivered specifically to the target tissue with the aim of reducing or eliminating the target tissue without affecting surrounding tissues.

[0003] However, traditional methods for implementing treatment plans for patients are difficult to execute in a controlled manner. For example, a patient's anatomy, tumor lesions, etc., can move during respiration due to reflexive movement and / or due to visceral movement (the natural movement of organs within the body). This can lead to displacement of the gross target volume (GTV) being targeted during RT therapy and misalignment relative to LINAC. To reduce the likelihood of this, the patient being treated can be instructed to hold their breath during energy delivery (often referred to as deep inspiratory breath-hold or "DIBH"). However, the success rate of this technique can vary drastically and cannot fully account for unconscious movement or visceral movement. Furthermore, other techniques involving monitoring patient movement based on external features (such as surrogate devices) may suffer from "drift," leading to inaccurate determination of GTV location. Therefore, the energy delivered to the patient's GTV may not be optimal, as some of the energy originally intended to target the GTV may be misaligned and delivered to different parts of the GTV or one or more organs at risk (OARs). Summary of the Invention

[0004] For the reasons mentioned above, there is a need for a system and method for predicting the location of a gross target volume within a patient during radiotherapy by modeling the movement of one or more lesions of the gross target volume over time.

[0005] The methods and systems discussed in this paper aim to address the challenge of predicting lesion location in 3D space during radiotherapy, where the lesion location changes at least to some extent due to patient insensitivity, reflexes, or visceral movement. More specifically, the currently disclosed techniques aim to model the motion of the lesion's gastrointestinal tract (GTV) over time during radiotherapy, thereby minimizing the deviation between the lesion's predetermined location (e.g., posture) in 3D space and its actual location (as they change).

[0006] In some embodiments, a system including one or more processors can be configured to acquire image data associated with multiple two-dimensional (2D) images generated by an imaging device positioned around a patient. These systems can determine the location of one or more lesions of the patient at a first time point based on the multiple 2D images. The system can then determine the future location of the one or more lesions at a later time point based on the location of the one or more lesions and trajectories representing the expected movement of the one or more lesions. In some examples, the system can then generate control signals to control the operation of one or more devices. For example, the system can generate control signals during RT treatment to control the operation of a LINAC, wherein such control signals adjust the position of one or more components of the LINAC. In other examples, the system can generate control signals to generate a beam direction view of the predicted location of the GTV over time.

[0007] Several technological advantages can be achieved by implementing some or all of the techniques described herein. First, the location of one or more lesions in 3D space can be determined more accurately compared to conventional methods. This, in turn, can improve treatment by delivering energy more precisely during RT therapy. Furthermore, systems implementing the techniques described herein can account for unpredictable uncertainties arising from involuntary patient movements, reflexive movements, or visceral movements. In addition to improved energy delivery, these advantages can reduce the consumption of computational resources. For example, systems including processors and memory can be completely preserved or retained, which would otherwise be dedicated to determining the location of the GTV (e.g., based on relative motion of surrogates). Moreover, medical devices such as LINAC can be configured to operate with less downtime between instances of energy delivery, thus delivering energy to the GTV more quickly, which can improve the effectiveness of RT therapy (e.g., targeting and destroying cells in the lesion). These advantages, through radiotherapy and through improved energy delivery, collectively promote more effective and efficient cancer treatment.

[0008] In one embodiment, a system is disclosed for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time. The system may include one or more processors configured to acquire image data associated with multiple two-dimensional (2D) images. The image data is generated at a first time point by multiple imaging devices positioned around the patient. The system can determine the location of one or more lesions at the first time point based on the multiple 2D images. The system is capable of determining the future location of one or more lesions at a second time point after the first time point based on the location of the one or more lesions and trajectories representing the expected motion of the one or more lesions in three-dimensional (3D) space. The system can generate control signals based on the future locations of the one or more lesions to move a linear accelerator (LINAC) from its current orientation at the first time point to its future orientation at the second time point, thereby adjusting the beam path of the linear accelerator.

[0009] In some aspects, the multiple 2D images may include two or more X-ray images. One or more processors, which may be configured to acquire image data, are configured to acquire two or more X-ray images from multiple devices. The X-ray images may include a two-dimensional (2D) representation of at least a portion of the patient at a first time point. In some aspects, one or more processors, which may be configured to determine the location of one or more lesions at the first time point, are configured to determine the pose of one or more lesions based on the multiple 2D images. In some aspects, one or more processors may be further configured to generate a trajectory representing the expected motion of one or more lesions based on the location of one or more previous lesions at the first time point and the future location of one or more previous lesions at a second time point. In at least one aspect, one or more processors configured to generate a trajectory representing the expected motion of one or more lesions may be configured to provide the location of one or more lesions at the first time point to an adaptive motion model, so that the adaptive motion model generates an output representing the trajectory.

[0010] In some aspects, one or more processors configured to provide the locations of one or more lesions to an adaptive motion model can be configured to: provide the locations of one or more lesions at a first time point to the adaptive motion model. The adaptive motion model can be configured to predict a set of future locations for one or more lesions between the first time point and a second time point. In at least one aspect, the system can obtain future location data associated with this set of future locations for one or more lesions. The system can determine a trajectory based on this set of future locations.

[0011] In one aspect, one or more processors may be further configured to determine a beam orientation view (BEV) of one or more lesions at a second time point based on the LINAC's attitude at a second time point and the future location of one or more lesions.

[0012] In another embodiment, the method may include: acquiring image data associated with multiple two-dimensional (2D) images via one or more processors. The image data may be generated at a first time point by multiple imaging devices positioned around the patient. The method may include: determining the location of one or more lesions at the first time point based on the multiple 2D images via one or more processors. In some aspects, the method may include: determining the future location of one or more lesions at a second time point after the first time point based on the location of one or more lesions and trajectories representing the expected motion of one or more lesions in three-dimensional (3D) space via one or more processors. The method may include: generating control signals based on the future locations of one or more lesions via one or more processors to move a linear accelerator (LINAC) from its current pose at the first time point to its future pose at the second time point, thereby adjusting the beam path of the linear accelerator.

[0013] In one aspect, multiple 2D images may include two or more X-ray images. Acquiring image data may include acquiring two or more X-ray images from multiple imaging devices via one or more processors. The X-ray images may include a two-dimensional (2D) representation of at least a portion of the patient at a first time point.

[0014] In some aspects, determining the location of one or more lesions at a first time point may include: determining the pose of one or more lesions based on multiple 2D images using one or more processors. The method may also include: generating a trajectory representing the expected motion of one or more lesions based on the locations of one or more previous lesions at the first time point and the future locations of one or more previous lesions at a second time point using one or more processors. Generating a trajectory representing the expected motion of one or more lesions may include: providing the locations of one or more lesions at the first time point to an adaptive motion model using one or more processors, so that the adaptive motion model generates an output representing the trajectory. In some aspects, providing the locations of one or more lesions to the adaptive motion model may include: providing the locations of one or more lesions at the first time point to the adaptive motion model using one or more processors. The adaptive motion model may be configured to predict a set of future locations of one or more lesions between the first and second time points. The method may include: obtaining future location data associated with this set of future locations of one or more lesions using one or more processors. The method includes determining a trajectory based on this set of future locations. In some aspects, the method may also include: determining a beam orientation view (BEV) of one or more lesions at a second time point based on the pose of the LINAC at the second time point and the future locations of one or more lesions using one or more processors.

[0015] In yet another embodiment, a non-transitory computer-readable medium may store instructions thereon that, when executed by one or more processors, cause one or more processors to acquire image data associated with multiple two-dimensional (2D) images. The image data may be generated at a first time point by multiple imaging devices positioned around the patient. These instructions may cause one or more processors to determine the location of one or more lesions at the first time point based on the multiple 2D images. These instructions may cause one or more processors to determine the future location of one or more lesions at a second time point after the first time point based on the location of the one or more lesions and trajectories representing the expected motion of the one or more lesions in three-dimensional (3D) space. These instructions may cause one or more processors to generate control signals based on the future locations of the one or more lesions to move the linear accelerator (LINAC) from its current pose at the first time point to its future pose at the second time point, thereby adjusting the beam path of the linear accelerator.

[0016] In one aspect, multiple 2D images may include two or more X-ray images. Instructions causing one or more processors to acquire image data may cause one or more processors to acquire two or more X-ray images from one or more imaging devices. The X-ray images may include a two-dimensional (2D) representation of at least a portion of the patient at a first time point. In some aspects, instructions causing one or more processors to determine the location of one or more lesions at a first time point may cause one or more processors to determine the pose of one or more lesions based on multiple 2D images. In some aspects, these instructions may also cause one or more processors to generate trajectories representing the expected motion of one or more lesions based on the location of one or more previous lesions at a first time point and the future location of one or more previous lesions at a second time point.

[0017] On the other hand, instructions that cause one or more processors to generate a expected motion trajectory representing one or more lesions can cause one or more processors to provide the positions of one or more lesions at a first time point to an adaptive motion model, thereby causing the adaptive motion model to generate an output representing the trajectory. Instructions that cause one or more processors to provide the positions of one or more lesions to the adaptive motion model can cause one or more processors to provide the positions of one or more lesions at a first time point to the adaptive motion model. The adaptive motion model can be configured to predict a set of future positions of one or more lesions between a first time point and a second time point. These instructions can be configured to cause one or more processors to obtain future position data associated with this set of future positions of one or more lesions. These instructions can cause one or more processors to determine a trajectory based on this set of future positions.

[0018] In one embodiment, a system is disclosed for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time. The system may include one or more processors configured to: acquire image data associated with a plurality of two-dimensional (2D) images generated by a plurality of imaging devices positioned around the patient and representing the location of one or more lesions within a first time period. The one or more processors may be configured to provide the location of one or more lesions to a sequential triangulation network (or sequential triangulation network) to generate an estimated trajectory representing the motion of one or more lesions within the first time period. In some aspects, the one or more processors may be configured to provide the estimated trajectory to a motion prediction network to generate a predicted trajectory representing the expected motion of one or more lesions in three-dimensional (3D) space within a second time period, and determine the future location of one or more lesions based on the location of one or more lesions and the predicted trajectory. In some aspects, the one or more processors may be configured to generate control signals based on the future location of one or more lesions to move a linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator.

[0019] In some implementations, one or more processors can be configured to determine the location of one or more lesions within a first time period based on multiple 2D images. One or more processors for providing image data to a sequence triangulation network can be configured to provide image data to the sequence triangulation network, wherein the sequence triangulation network is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients. In some aspects, the sequence triangulation network can be trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for one or more patients.

[0020] In some aspects, one or more processors configured to provide one or more lesion locations to a sequence triangulation network can be configured to condition the sequence triangulation network based on at least a portion of the locations of the one or more lesions. Multiple 2D images may include two or more X-ray images, and one or more processors configured to acquire image data can be configured to acquire two or more X-ray images from multiple imaging devices, said two or more X-ray images including a two-dimensional (2D) representation of at least a portion of the patient within a first time period. The locations of the one or more lesions within the first time period may represent the pose of the one or more lesions in 3D space. In some aspects, one or more processors can also be configured to determine a beam orientation view (BEV) of the one or more lesions based on the future locations of the one or more lesions and the pose of the LINAC at a second time point.

[0021] In another embodiment, a method is disclosed for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time. The method may include: acquiring image data associated with a plurality of two-dimensional (2D) images via one or more processors, the image data being generated by a plurality of imaging devices positioned around the patient and representing the location of one or more lesions within a first time period; in some aspects, the method may include: providing the location of one or more lesions to a sequence triangulation network via one or more processors, such that the sequence triangulation network generates an estimated trajectory representing the motion of one or more lesions within the first time period. In some aspects, the method may include: providing the estimated trajectory to a motion prediction network via one or more processors, such that the motion prediction network generates a predicted trajectory representing the expected motion of one or more lesions in three-dimensional (3D) space within a second time period. In some aspects, the method may include: determining the future location of one or more lesions based on the location of one or more lesions and the predicted trajectory via one or more processors. In some aspects, the method may include: generating control signals based on the future location of one or more lesions via one or more processors to move a linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator.

[0022] In some aspects, the method may further include: determining the location of one or more lesions within a first time period based on multiple 2D images by one or more processors. Providing image data to a sequence triangulation network may include: providing image data to the sequence triangulation network by one or more processors, wherein the sequence triangulation network is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients. In some aspects, the sequence triangulation network may be trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for one or more patients. In at least some aspects, providing the location of one or more lesions to the sequence triangulation network may include: conditionalizing the sequence triangulation network by one or more processors based on at least a portion of the location of one or more lesions. The multiple 2D images may include two or more X-ray images, and obtaining the image data may include: obtaining two or more X-ray images from multiple imaging devices by one or more processors, said two or more X-ray images comprising a two-dimensional (2D) representation of at least a portion of the patient within a first time period.

[0023] In some aspects, the position of one or more lesions during a first time period can represent the pose of one or more lesions in 3D space. In some aspects, the method may further include: determining a beam orientation view (BEV) of one or more lesions by one or more processors based on the future position of one or more lesions and the pose of LINAC at a second time point.

[0024] In yet another embodiment, a non-transitory computer-readable medium storing instructions thereon is disclosed, which, when executed by one or more processors, cause one or more processors to: acquire image data associated with a plurality of two-dimensional (2D) images generated by a plurality of imaging devices positioned around a patient and representing the location of one or more lesions within a first time period. The instructions may cause the one or more processors to provide the location of the one or more lesions to a sequence triangulation network, causing the sequence triangulation network to generate an estimated trajectory representing the motion of the one or more lesions within the first time period. In some aspects, these instructions may cause one or more processors to: provide the estimated trajectory to a motion prediction network, causing the motion prediction network to generate a predicted trajectory representing the expected motion of the one or more lesions in three-dimensional (3D) space within a second time period, and determine the future location of the one or more lesions based on the location of the one or more lesions and the predicted trajectory. In some aspects, these instructions may cause one or more processors to generate control signals based on the future location of the one or more lesions to move a linear accelerator (LINAC) from a first pose to a second pose, thereby adjusting the beam path of the linear accelerator.

[0025] In at least some aspects, these instructions can also prompt one or more processors to determine the location of one or more lesions within a first time period based on multiple 2D images. In some aspects, instructions prompting one or more processors to provide image data to a sequence triangulation network can cause the sequence triangulation network to be trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients. In some aspects, the sequence triangulation network can be trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for one or more patients. Attached Figure Description

[0026] Non-limiting embodiments of this disclosure are described by way of example and with reference to the accompanying drawings, which are schematic and not drawn to scale. Unless indicated as background art, the drawings illustrate various aspects of this disclosure.

[0027] Figure 1 A diagram is shown of a system for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment.

[0028] Figure 2 A flowchart is shown, according to an embodiment, of a process for predicting the location of a gross target volume within a patient during radiotherapy.

[0029] Figure 3 An example implementation of a process for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment, is shown.

[0030] Figure 4 An example of target motion modeling that can be established during implementation is shown according to an embodiment.

[0031] Figure 5 An accuracy diagram of the trajectory predicted using a simulated X-ray imager according to an embodiment is shown.

[0032] Figure 6 A flowchart is shown, according to an embodiment, of a process for predicting the location of a gross target volume within a patient during radiotherapy.

[0033] Figure 7 An example implementation of a process for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment, is shown.

[0034] Figure 8 An example of respiration-induced movement of a general target volume according to an embodiment is shown. Detailed Implementation

[0035] Reference is now made to the illustrative embodiments depicted in the accompanying drawings, and they are described herein using specific language. However, it should be understood that this is not intended to limit the scope of the claims or this disclosure. Changes and further modifications to the inventive features illustrated herein, as well as other applications of the subject matter principles illustrated herein (which will be apparent to those skilled in the art and are included within the scope of this disclosure), are configured to be considered within the scope of the subject matter disclosed herein. Other embodiments may be used, or other changes may be made, without departing from the spirit or scope of this disclosure. The illustrative embodiments described in the detailed description are not intended to limit the subject matter presented.

[0036] Figure 1Components of a system 100 for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment, are shown. System 100 may include an analysis server 114a, a system database 114b, a treatment planning system 111, electronic data sources 120a-d (unless otherwise stated, each electronic data source is individually referred to as electronic data source 120 and collectively as electronic data source 120), end-user devices 140a-c (unless otherwise stated, each end-user device is individually referred to as end-user device 140 and collectively as end-user device 140), an administrator computing device 150, a medical device 160, and a medical device computer 162. Figure 1 The various components depicted herein may belong to a radiation therapy clinic where patients may receive radiation therapy under certain circumstances via one or more radiation therapy machines (e.g., medical device 160) located within the clinic. System 100 is not limited to the components described herein and may include additional or other components (not shown for brevity) that may be configured to be considered within the scope of the embodiments described herein.

[0037] The aforementioned components can be interconnected via network 130. Examples of network 130 may include, but are not limited to, private or public local area networks (LANs), wireless local area networks (WLANs), metropolitan area networks (MANs), wide area networks (WANs), and the Internet. Network 130 may include wired or wireless communication according to one or more standards or via one or more transport media. Communication via network 130 may be performed according to various communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), and IEEE communication protocols. In one example, network 130 may include wireless communication according to the Bluetooth specification set or other standards or proprietary wireless communication protocols. In another example, network 130 may also include communication via cellular networks, such as GSM (Global System for Mobile Communications), CDMA (Code Division Multiple Access), and EDGE (Enhanced Data Global Grid) networks.

[0038] The analytics server 114a can be any computing device that includes a processor and non-transitory machine-readable storage capable of performing the various tasks and processes described herein. The analytics server 114a can use various processors, such as a central processing unit (CPU) and a graphics processing unit (GPU). Non-limiting examples of such a computing device may include workstation computers, laptops, server computers, etc. While system 100 includes a single analytics server 114a, the analytics server 114a can include any number of computing devices operating in a distributed computing environment, such as a cloud environment.

[0039] Analysis server 114a can generate and display an electronic platform configured to use treatment planning system 111 to receive patient information and input from users (e.g., clinicians), such as utility functions and updated utility functions as described herein, and output the execution results of treatment planning system 111. The electronic platform may include a graphical user interface (GUI) displayed by a display device of one or more electronic data sources 120, end-user device 140, medical device 160, or administrator computing device 150. Examples of electronic platforms generated and hosted by analysis server 114a may be web-based applications or websites configured to be displayed on various electronic devices (e.g., mobile devices, tablets, personal computers, etc.).

[0040] For example, the information displayed by the electronic platform may include input elements for receiving data associated with the patient being treated, synchronizing one or more sensors, and displaying predictions generated by the treatment planning system 111. For example, the analysis server 114a may execute the treatment planning system 111 (e.g., a system such as a treatment planner configured or trained to generate beam configurations that can be used to configure the medical device 160 during patient treatment, as described herein). The analysis server 114a may then display the results to a clinician or directly revise one or more operational attributes of the medical device 160.

[0041] Electronic data source 120 can be any computing device, including a processor and non-transitory machine-readable storage capable of performing the various tasks and processes described herein. For example, electronic data source 120 can represent various computing devices that contain, retrieve, or access data associated with medical device 160, such as data associated with operational information of currently or previously performed radiotherapy (e.g., electronic log files or electronic configuration files), data associated with currently or previously monitored patients (e.g., computed tomography (CT) scans, magnetic resonance imaging (MRI) scans, tumor location, deformation information, etc.), or study participants. For example, analysis server 114a can use clinic computer 120a, medical professional device 120b, server 120c (associated with a clinician or clinic), and database 120d (associated with a clinician or clinic) to retrieve / receive data associated with medical device 160. Analysis server 114a can retrieve data from end-user device 140, generate datasets, and use these datasets to configure treatment planning system 111 (e.g., models implemented by treatment planning system 111). Analysis server 114a can execute various algorithms to convert raw data received / retrieved from electronic data source 120 into machine-readable objects, which can be stored and processed by other analysis processes described herein.

[0042] End-user device 140 can be any computing device including a processor and a non-transitory machine-readable storage medium capable of performing the various tasks and processes described herein. Non-limiting examples of end-user device 140 may be a workstation computer, a laptop computer, a tablet computer, or a server computer. In operation, various users as described herein, such as clinicians, can use end-user device 140 to access a GUI operatively managed by analytics server 114a, or additionally access the execution results of treatment planning system 111. Specifically, end-user device 140 may include clinic computer 140a, clinic server 140b, and medical professional device 140c. Although referred to herein as “end-user” devices, these devices are not always operated by end-users. For example, clinic server 140b cannot be directly used by end-users. However, results stored on clinic server 140b can be used to populate various GUIs accessed by end-users via medical professional device 140c. In some embodiments, end-user device 140 may be associated with one or more clinicians associated with generating one or more treatment plans for a patient (e.g., participating in the preparation of one or more treatment plans).

[0043] The administrator computing device 150 may represent a computing device operated by a system administrator. The administrator computing device 150 may be configured to display radiotherapy attributes generated by the analysis server 114a (e.g., various analytical metrics determined during the training of one or more machine learning models or systems); monitor multiple treatment planning systems 111 utilized by the analysis server 114a, electronic data source 120, or end-user device 140; review feedback; or facilitate the training or retraining (calibration) of the treatment planning systems 111 maintained by the analysis server 114a.

[0044] In some embodiments, the medical device 160 may be a diagnostic imaging device or a therapeutic delivery device (also referred to as a radiotherapy system). For example, the medical device 160 may include one or more computed tomography (CT) scanners, such as cone-beam CT (CBCT) scanners, linear accelerators (LINAC), such as Varian® TrueBeam® linear accelerators, proton beam therapy systems that use accelerated protons to precisely irradiate tumors (referred to as proton beam systems), or other similar devices configured to deliver energy to a target tissue associated with the patient (referred to as the gross target volume) and, in some cases, measure the energy delivered to the target tissue. The medical device 160 may also include one or more sensors configured to monitor the patient being treated. That is, the medical device 160 or the analysis server 114a may communicate with a variety of sensors capable of monitoring external biosignals of the patient. Non-limiting examples of sensors may include three-dimensional (3D) surface mechanisms and optical (or other) sensors configured to monitor the patient's movement (e.g., how the patient moves or breathes). In some embodiments, the medical device 160 may receive data associated with a treatment plan from (a plurality of) medical device computers 162, which cause the medical device 160 to operate according to the treatment plan.

[0045] Treatment planning system 111 may be stored in system database 114b. Treatment planning system 111 can be trained using data received / retrieved from electronic data source 120 and can be executed using data received from end-user devices, medical devices 160, or sensors 163. In some embodiments, treatment planning system 111 may reside locally or in a clinic-specific data repository. In various embodiments, treatment planning system 111 may use one or more deep learning engines to develop treatment plans for patients receiving radiation therapy. For example, analysis server 114a may transmit patient attributes from sensor 163 and execute treatment planning system 111 accordingly. Analysis server 114a may then display the results on one or more end-user devices 140. In some embodiments, analysis server 114a may modify one or more configurations of medical device 160 based on the results predicted by treatment planning system 111.

[0046] refer to Figure 2 A flowchart of a process 200 for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment, is shown. Process 200 includes operations 202-208. However, other embodiments may include additional or alternative operations, or one or more operations may be omitted entirely. Process 200 is described as being performed by an analysis server, which can communicate with... Figure 1The analysis server 114a described herein is the same as or similar. However, one or more steps of process 200 can be performed by [the server described herein]. Figure 1 The distributed computing system described herein can run on any number of computing devices. For example, one or more computing devices can execute locally. Figure 2 Some or all of the operations described in the document.

[0047] In operation 202, the analysis server can obtain image data associated with two or more two-dimensional (2D) images generated by imaging devices. For example, the analysis server can obtain image data associated with two or more 2D images generated by one or more imaging devices positioned around the patient. These imaging devices may include X-ray machines, etc. In some embodiments, the analysis server can obtain image data from multiple imaging devices, wherein the multiple imaging devices are positioned relative to the patient in 3D space. In this example, the multiple imaging devices may be associated with a medical device (e.g., its association with...). Figure 1 The medical device 160 (identical or similar), for example, a LINAC being moved to multiple control points established by a treatment plan to deliver energy to the patient, is positioned relative to the patient. Although the present disclosure is discussed in relation to LINACs, it should be understood that different types of medical devices (e.g., proton beam systems, etc.) are considered as alternatives to or complements to LINACs.

[0048] In one example, during RT treatment, a patient can be positioned relative to a LINAC gantry supporting a magnetron or klystron that generates high-energy X-rays for delivery to the patient's lesion. The LINAC may include a multi-leaf collimator (MLC) positioned along the treatment beam path (beam path) and configured to shape the high-energy X-rays as they are directed toward the patient's lesion. In some embodiments, multiple imaging devices can be positioned such that they can target at least a portion of the patient, including the lesion targeted as part of the treatment plan. For example, during patient treatment, the LINAC may be moved to multiple control points (e.g., 3D orientation within a 3D space containing the patient, LINAC, and imaging devices) and configured to generate energy at each control point and deliver that energy to the patient. At each control point, the blades of the LINAC's multi-leaf collimator can shape the X-rays to optimize energy delivery and consistency with the patient's lesion, while minimizing energy delivery outside the lesion (e.g., to one or more organs at risk). In some embodiments, a set of control points, MLC blade configurations, and power levels for energy delivery can be established using a treatment plan generated prior to surgery. The analytics server can be configured to perform actions to control the operation of the LINAC based on the treatment plan developed for the patient, as described herein.

[0049] In some embodiments, the analysis server may acquire image data at a first time point. For example, the analysis server may acquire image data at two or more time points during a pre-treatment period or during a period when the patient is receiving RT treatment. During this period, the imaging device may be configured to generate images at multiple points. In some embodiments, the analysis server may acquire image data from multiple imaging devices, wherein the image data includes (e.g., representing) two or more X-ray images acquired by one or more imaging devices. The X-ray images may include a 2D representation of at least a portion of the patient. For example, X-ray images acquired by one or more imaging devices may represent a corresponding 2D representation of at least a portion of the patient including GTV. As described herein, the analysis server may acquire image data at two or more different time points. To enable the analysis server to acquire additional image data, one or more imaging devices may be configured to periodically or continuously generate image data and provide the image data to the analysis server.

[0050] In operation 204, the analysis server can determine the location of one or more lesions. For example, the analysis server can determine the location of one or more lesions within the patient at a first time point corresponding to the time point at which the image data was generated. In this example, the analysis server can determine the location of one or more lesions, wherein the one or more lesions are located within the patient's GTV. In some embodiments, the location of one or more lesions may represent the location and / or orientation (e.g., pose) of at least a portion of the GTV and be established relative to one or more imaging devices. The analysis server can then determine the pose of one or more lesions (e.g., included within the GTV). For example, the analysis server can determine the pose of one or more lesions in the 3D space where the patient is receiving RT treatment based on multiple 2D images acquired by imaging devices positioned relative to the patient.

[0051] In some embodiments, the analysis server can determine the location of one or more lesions by using one or more models to generate trajectories representing the expected motion of one or more lesions. For example, the analysis server can use a model (e.g., a neural network, an adaptive motion model, etc.) to generate one or more trajectories representing the expected motion of one or more lesions. In this example, the model can be configured to receive data associated with the location of one or more lesions at a first time point and generate an output representing the trajectory. The data associated with the location of one or more lesions can be represented using multiple 2D images acquired at the first time point of image data generation. In some embodiments, the analysis server can determine the trajectory based on the model's output. For example, the analysis server can prompt the model to determine (e.g., extract, etc.) the trajectory of one or more lesions over a time period starting from the time point of image data generation. The trajectory can represent the motion of one or more lesions (or at least a portion of one or more lesions) in 3D space. This motion can be represented as an offset along the X, Y, and Z axes. In some examples, the motion can also be represented by a change in velocity over a time period.

[0052] In some embodiments, the analysis server can generate trajectories using a model trained on the locations of one or more pre-existing lesions at a first time point and the locations of one or more pre-existing lesions at future time points. For example, a training dataset can be built using the locations (e.g., poses) of multiple lesions at the first time point. In an example, during, for instance, an earlier RT treatment, while an imaging device tracks one or more pre-existing lesions, the locations of one or more pre-existing lesions can be determined based on generating 2D images similar to those described above. The training dataset may also include the corresponding locations of one or more pre-existing lesions at a second time point. These locations at the second time point may be offset from the previous time point by an established time interval.

[0053] The model can then be trained using a training dataset. For example, the analytics server can use the training dataset to train a model to output a prediction in response to the location of one or more previous lesions received during RT treatment (e.g., established from two or more 2D images acquired during RT treatment). In one example, the analytics server can train the model by providing it with the location of one or more previous lesions at a first time point, causing the model to generate an output. In this example, the output could represent a trajectory indicating the predicted motion of one or more lesions. The analytics server can then compare the output (e.g., the predicted trajectory, which, when applied to the location of the GTV, causes the GTV to be repositioned to the predicted location) with the initially represented location of one or more previous lesions. In some examples, the analytics server can then determine the future location of one or more previous lesions at a second time point (e.g., based on the motion of one or more previous lesions from their first location according to the trajectory predicted by the model) and compare the future location with a known future location established from the training dataset. The analytics server can then compare the difference between the predicted location and the known future location, determine a loss based on that difference, and update one or more weights of the model to reduce the loss when the analytics server subsequently executes the model. The analytics server can iteratively repeat this process until the model converges (e.g., the model generates a prediction that leads to a future location with a corresponding loss that satisfies a threshold of acceptable accuracy in response to the model). This can allow for consistent trajectory predictions of one or more lesions during subsequent RT treatments performed by the analytics server.

[0054] In some embodiments, once trained, the analytics server can use the model to generate trajectories representing the expected motion of one or more lesions, wherein the model is trained on a dataset representing the positions of one or more lesions as they move over time. For example, the analytics server can provide the positions of one or more lesions at a first time point as input to the model. In this example, the analytics server can prompt the model to execute based on the positions of one or more lesions at the first time point to generate output. The output can represent trajectories indicating the expected motion of one or more lesions. For example, the output can represent the trajectories of one or more lesions in 3D space, the change in acceleration of one or more lesions as they move in 3D space over time, etc.

[0055] In operation 206, the analysis server can determine the future location of one or more lesions. For example, in response to the analysis server generating one or more trajectories representing the expected movement of one or more lesions, the analysis server can determine the future location of one or more lesions. As described herein, the analysis server can determine the future location of one or more lesions within a predetermined time period starting from the point in time when the image data is generated. For example, based on a predetermined time interval after the image data is generated, the analysis server can determine the future location of one or more lesions in the 3D space where the patient is located. In this example, the time interval can be associated with (e.g., correspond to) the time it takes for one or more lesions to move between locations established by a training dataset. In some embodiments, the analysis server then iteratively repeats one or more of operations 202-206 and predicts the future location of one or more lesions in 3D space during RT treatment.

[0056] In some embodiments, based on the location of one or more lesions at a first time point and a trajectory generated by a model, the analysis server can determine the future location of one or more lesions. For example, by applying the trajectory to points established for one or more lesions from image data generated at the first point within a time period, the analysis server can determine the future location. In this example, the analysis server can apply a transformation to points representing one or more lesions in 3D space based on the trajectory to model the movement of one or more lesions and use the model to predict the future location of one or more lesions.

[0057] In operation 208, the analysis server can generate control signals to move the medical device from its current position to a future orientation. For example, the analysis server can determine the orientation of the medical device at a first time point. At the first time point, based on the relative position of the patient to the 3D space in which the patient resides and / or the relative position of the medical device, the analysis server can configure the medical device to generate energy and transmit it toward the patient's GTV. In this example, the analysis server can cause the medical device to generate and transmit energy according to a treatment plan. For example, the analysis server can be configured to control the operation of the medical device according to a treatment plan established prior to RT treatment of the patient, wherein the treatment plan indicates one or more control points for moving the medical device between control points and one or more configurations of the medical device (e.g., one or more power levels for energy transmission in one or more blade configurations, etc.). In some embodiments, the analysis server can then predict the future position of the GTV during RT treatment, as described above. For example, when the medical device moves from a control point toward a control point and is activated to deliver energy to the patient's GTV, the analysis server can predict the future position of the GTV. The analysis server can then compare the predicted future location with the location established through the treatment plan and control the operation of the medical device so that the medical device is aimed at the GTV at the (predicted) future location.

[0058] In some embodiments, the analysis server may determine a beam orientation view (BEV). For example, the analysis server may determine a beam orientation view that represents the view of the GTV relative to one or more components of the medical device (e.g., the collimator of a LINAC). The beam orientation view may be represented as a 2D representation of the GTV as seen from one or more components of the LINAC. In some embodiments, the analysis server may use the beam orientation view when aiming the GTV with the medical device during the execution of an RT treatment plan.

[0059] refer to Figure 3 An example implementation 300 of a process for predicting the location of a GTV (including one or more lesions thereof) within a patient during RT treatment, according to an embodiment, is shown. Figure 3 As shown, implementation 300 involves the execution of the target motion modeling network 304. In some examples, the target motion modeling network 304 can be related to the above-mentioned... Figure 2 The described models (e.g., neural networks, adaptive motion models, etc.) are the same as or similar. In some embodiments, one or more operations described with respect to implementation 300 may be performed by an analysis server, which is similar to... Figure 1 Analysis server 114a and / or about Figure 2 The analysis servers discussed are the same or similar.

[0060] In some embodiments, implementation 300 may involve one or more operations performed by a target motion modeling network 304. For example, the target motion modeling network 304 may be configured to receive one or more past timestamps as input, corresponding to images 302a acquired during RT treatment by multiple imaging devices positioned around the patient. In the example, the target motion modeling network 304 may be configured to receive the position of the imaging device used to generate image 302a, represented as the position / orientation (e.g., pose) of the imaging device in 3D space where the patient is being treated with a medical device such as LINAC as described herein. The target motion modeling network 304 can then generate a representation of a 3D target trajectory 306 (e.g., as described above regarding...). Figure 2 One or more operations are performed when the corresponding output of the described trajectory is received. These 3D target trajectories 306 may correspond to the expected motion of one or more lesions (e.g., GTVs) within a time period that begins at a first time point (e.g., represented by a past timestamp) and a second time point (e.g., represented by a future timestamp). In some embodiments, the 3D target trajectories 306 may be projected onto a 2D X-ray image plane using a projection matrix associated with multiple imaging devices (e.g., X-ray images, virtual imaging devices, etc., as described herein) used to generate image 302a. In some embodiments, the estimated 3D locations may be projected onto the 2D X-ray image plane using the projection matrix for comparison with actual 2D observations of one or more lesions. In response to comparing the estimated 3D locations with the 2D observations, a loss may be calculated between the predicted location at the first time point established by the timestamps and the known location of one or more lesions (e.g., at future time points) established based on the 3D target trajectories 304. During training of the target motion modeling network 304, the parameters of the target motion modeling network 304 may be iteratively adjusted by comparing future timestamps and ground truth observations with the predicted locations of one or more lesions. This allows the target motion modeling network 304 to be configured to reconstruct and predict the movement of one or more lesions within the GTV in the same implementation 300.

[0061] In some embodiments, the target motion modeling network 304 may be configured (e.g., trained) based on meta-learning of implicit functions, where the implicit functions are modeled as deep neural networks whose parameters are controlled and learned by another deep neural network. In one embodiment, the implicit neural representation may be used in conjunction with periodic activation functions to model the implicit functions. Specifically, the target motion modeling network 304 may use a set of learnable sine functions as periodic activation functions, which are more flexible in modeling semi-periodic lesion motions compared to a predetermined set of basis vectors. In some embodiments, meta-learning may involve performing a model-agnostic meta-learning (MAML) framework to enable rapid adaptation of model parameters. In the example, the MAML framework may be adapted to global model parameters pre-trained using many offline cohort samples, allowing these parameters to be quickly adapted to specific samples. In the example described herein, cohort samples from multiple patients may be used to obtain a global model (e.g., the global target motion modeling network 304), thereby allowing rapid fine-tuning of the global model from short-term observations of a specific patient to reconstruct 3D trajectories and predict future target motions.

[0062] Figure 4 An example 400 of target motion modeling that can be established during the execution of implementation 300 according to an embodiment is shown. As shown, the ground truth “GT” trajectory and the estimated or predicted “PD” trajectory are shown. The first two columns 402 and 404 show the projected trajectories in the kV (X-ray) and MV (beam) coordinate systems, respectively. The third column 406 shows the 3D trajectory on three axes, where the values ​​within the normalized timestamps from -1.0 to 0.5 are reconstructed, and the values ​​between 0.5 and 1.0 are predicted. The fourth column 408 shows a magnified view of the first five predicted signals, while the fifth column 410 shows a magnified view of all predicted signals within the timestamps from 0.5 to 1.0.

[0063] Figure 5 A precision plot 500 of the predicted trajectory using a simulated X-ray imager is shown. More specifically, a precision plot 500 of the predicted trajectory is shown, where the simulated X-ray imager has a sampling rate of 5.2 Hz and a gantry movement speed of 5 degrees / second. It can be seen that for a predicted signal within 500 ms, the average error of the predicted trajectory is less than 1.0 mm, and for a predicted signal within 1000 ms, the average error of the predicted trajectory is less than 1.5 mm.

[0064] Figure 6 A flowchart is shown, according to an embodiment, of a process for predicting the location of a gross target volume within a patient during radiotherapy.

[0065] refer to Figure 6A flowchart of a process 600 for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment, is shown. Process 600 includes operations 602-610. However, other embodiments may include additional or alternative operations, or one or more operations may be omitted entirely. Process 600 is described as being performed by an analysis server, which can communicate with... Figure 1 The analysis server 114a described herein is the same as or similar. However, one or more steps of process 200 can be performed by [the server described herein]. Figure 1 The distributed computing system described herein can run any number of computing devices to execute. For example, one or more computing devices can execute locally. Figure 6 Some or all of the operations described in the document.

[0066] In operation 602, the analysis server can obtain image data associated with two or more two-dimensional (2D) images generated by the imaging devices. For example, the analysis server can obtain image data associated with two or more 2D images generated by multiple imaging devices positioned around the patient, similar to the above description. Figure 2 The process described in 200. The imaging device may include an X-ray machine, etc. In some embodiments, the analysis server may acquire image data from multiple imaging devices, wherein the multiple imaging devices are positioned relative to the patient in 3D space. In this example, the multiple imaging devices may be used with a medical device such as LINAC (e.g., with...). Figure 1 The LINAC is positioned relative to the patient along with a medical device (identical or similar) 160, and moves to multiple control points established by the treatment plan in order to deliver energy to the patient. While the schemes disclosed herein are discussed in relation to LINACs, it should be understood that different types of medical devices (e.g., proton beam systems, etc.) may be considered as alternatives to or complements to LINACs.

[0067] In one example, during RT treatment, a patient can be positioned relative to a LINAC gantry supporting a magnetron or klystron for delivering high-energy X-rays to the patient's lesion, as described herein. During patient treatment, the LINAC can be moved to multiple control points and configured to generate and deliver energy to the patient at each control point. At each control point, the blades of the LINAC's MLC can shape the X-rays to optimize energy delivery and consistency with the patient's lesion, while minimizing energy delivery outside the lesion (e.g., to one or more organs at risk). In some embodiments, a set of control points, MLC blade configurations, and power levels for energy delivery can be established using a treatment plan generated prior to the procedure. It should be understood that an analytics server can be configured to perform actions based on the treatment plan developed for the patient to control the operation of the LINAC, as described herein.

[0068] In some embodiments, the analysis server may acquire image data after generating image data within a first time period. For example, the analysis server may acquire image data at one or more time points within the first time period, either when or before the patient receives treatment. In this example, the imaging device may be configured to generate images at multiple time points during the period in which the patient is being observed (e.g., conditioned one or more models as described herein to account for movement of one or more anatomical structures due to reasons such as the patient's breathing) and / or being treated. In some embodiments, the analysis server may acquire image data from one or more imaging devices, wherein the image data includes (e.g., representing) two or more X-ray images acquired by one or more imaging devices. The X-ray images may include a 2D representation of at least a portion of the patient. For example, the X-ray images acquired by the imaging device may include a corresponding 2D representation of at least a portion of the patient (with the GTV being treated within that at least portion). To enable the analysis server to acquire additional image data, multiple imaging devices may be configured to generate image data periodically or continuously and provide the image data to the analysis server during patient observation.

[0069] In some embodiments, the analysis server can determine the location of one or more lesions represented by image data at time points within a first time period (e.g., similar to that described above for process 200). For example, the analysis server can compare representations of one or more events in 2D images at time points within the first time period and determine the location of one or more lesions in response to the comparison. In some examples, the location of one or more lesions can be represented as 2D coordinates and / or 3D coordinates. Additionally or alternatively, the location of one or more lesions can indicate the location and / or orientation (e.g., pose) of one or more lesions (and by extension, the GTV associated with one or more lesions). In an example, the analysis server can determine the pose of each of the one or more lesions at a time point within the first time period based on the representation of one or more lesions in a 2D image. As will be understood, the analysis server can determine the location of one or more lesions located within the patient's GTV, similar to that described above regarding... Figure 2 The process is as described in 200. Examples of techniques for determining the location of one or more lesions are also described in U.S. Patent Application No. 18 / 999,536, filed December 23, 2024, entitled “SYSTEMS AND METHODS FOR DETERMINING A LOCATION OF A GROSS TARGET VOLUME OF A PATIENT”, the contents of which are incorporated herein by reference in their entirety.

[0070] In operation 604, the analysis server can provide the locations of one or more lesions to a sequence triangulation network to generate an estimated trajectory. For example, the analysis server can provide the locations of one or more lesions to a sequence triangulation network (e.g., a model such as a neural network, deep neural network, transformer, or transformer-based network, configured to perform one or more of the operations described herein to model one or more implicit functions) built from two or more 2D images of the patient. The sequence triangulation network can then be configured to generate an output associated with (e.g., representing the estimated trajectory) the locations of one or more lesions built from two or more 2D images. The estimated trajectory can represent the observed motion of one or more lesions in 3D space over a period determined based on 2D images generated before or during the patient's RT treatment.

[0071] In some embodiments, the analysis server can prompt a sequence triangulation network to generate estimated trajectories of one or more lesions in a patient. For example, during RT treatment, the analysis server can acquire two or more 2D images of one or more lesions in the patient. The analysis server can then provide the two or more 2D images as input to the sequence triangulation network, causing the network to generate an output representing the estimated trajectory. The estimated trajectory can represent the relative motion of one or more lesions in the 3D space in which the patient is located.

[0072] A training dataset can be used to train a sequence triangulation network. For example, an analytics server can use the training dataset to train the sequence triangulation network to generate an estimated trajectory of the patient's lesions in response to acquiring (e.g., receiving, deriving, etc.) a 2D image of the patient (e.g., an X-ray image) and determining the location of one or more lesions (e.g., established from two or more 2D images acquired during RT treatment). In this example, the analytics server can train the sequence triangulation network by providing the location of one or more lesions, thereby prompting the sequence triangulation network to generate a corresponding output. In this example, the corresponding output can represent the estimated trajectory, indicating the movement of one or more lesions observed in the patient's 2D image. The analytics server can then compare the output (e.g., the estimated trajectory) with a known trajectory established using the training dataset. The analytics server can then determine the difference between the estimated trajectory and the known trajectory (e.g., the offset between the estimated trajectory and the known trajectory), determine a loss based on this difference, and update one or more weights of the sequence triangulation network to reduce the loss when the analytics server subsequently executes the sequence triangulation network. The analytics server can iteratively repeat this process until the sequence triangulation network converges (e.g., the model generates a prediction that leads to a future location with a corresponding loss that satisfies a threshold of acceptable accuracy in response to the sequence triangulation network).

[0073] In some embodiments, a known trajectory can be determined based on one or more four-dimensional CT (4DCT) scans. For example, a training dataset can be built based on 4DCT scans acquired from one or more previously observed patients. In this example, the analysis server can generate multiple 2D images in different gantry poses from the previously observed 4DCT scans of patients and include these 2D images in the training dataset. Similarly, the analysis server can generate multiple trajectories from the 4DCT scans. In some embodiments, the analysis server can use the training dataset built from the 4DCT scans to train a sequence triangulation network as described above.

[0074] In some embodiments, during RT treatment, the analysis server can condition the sequence triangulation network. For example, the analysis server can condition the sequence triangulation network based on the location of one or more lesions acquired by an imaging device for a specific patient receiving treatment. The analysis server can determine the location of one or more lesions based on 2D images of the patient acquired before or during RT treatment and provide these locations so that the sequence triangulation network generates an estimated trajectory. By providing the location of one or more lesions of the patient before or during treatment, the analysis server can train (e.g., update) the sequence triangulation network to model the movement of a specific lesion within that patient. This can lead to a more accurate determination of the location of one or more lesions on each patient.

[0075] In operation 606, the analysis server may provide the estimated trajectory to the motion prediction network to generate a predicted trajectory. For example, the analysis server may provide the estimated trajectory of one or more lesions to the motion prediction network (e.g., a model configured to perform one or more operations described herein to model one or more implicit functions, such as a neural network, deep neural network, transformer, or transformer-based network, etc.) based on (e.g., in response to) the estimated trajectory generated by the sequence triangulation network. The motion prediction network may be configured to generate an output associated with (e.g., representing) a predicted trajectory of expected motion of one or more lesions within the 3D space in which the patient is located during a second time period. Specifically, the analysis server may prompt the motion prediction network to generate a predicted trajectory in a time period following the time period represented by the estimated trajectory. The analysis server can then use the predicted trajectory described herein to determine the location (e.g., pose) of one or more lesions.

[0076] A training dataset can be used to train a motion prediction network. For example, an analytics server can use the training dataset to train a motion prediction network to generate predicted trajectories for patient lesions in response to obtaining (e.g., receiving, deducing, etc.) estimated trajectories of one or more lesions. In the example, the analytics server can train the motion prediction network by providing the location and / or estimated trajectory of one or more lesions of one or more patients represented in the training dataset at a certain time point (e.g., corresponding to the final time point represented by the estimated trajectory), thereby prompting the motion prediction network to generate a corresponding output. In this example, the corresponding output can represent a predicted trajectory that indicates the expected motion of one or more lesions of the corresponding patient within the 3D space in which the patient is located during a second time period (after the time period established by the estimated trajectory). The analytics server can then compare the output (e.g., the predicted trajectory) with a known trajectory established from the training dataset. In response to this comparison, the analytics server can determine the difference between the predicted trajectory and the known trajectory (e.g., the offset between the predicted trajectory and the known trajectory), and based on this difference, determine a loss and update one or more weights of the motion prediction network to reduce the loss when the analytics server subsequently executes the motion prediction network. The analysis server can iteratively repeat this process until the motion prediction network converges (e.g., the model generates a prediction that results in a trajectory with a corresponding loss that satisfies a threshold of acceptable accuracy in response to the motion prediction network).

[0077] In operation 608, the analysis server can determine the future location of one or more lesions. For example, the analysis server can determine the future location of one or more lesions in response to generating a predicted trajectory representing the expected movement of one or more lesions. In this example, the analysis server can determine the future location of one or more lesions based on the location of one or more lesions represented by image data at the start of the second time period and the predicted trajectory. In some embodiments, based on the predicted trajectory, the analysis server can determine the future location (e.g., future pose) of one or more lesions based on the transformation applied to the location of one or more lesions at the start of the second time period. It will be understood that the analysis server then iteratively repeats one or more of operations 602-606 and predicts the future location of one or more lesions in 3D space during RT treatment.

[0078] In operation 610, the analysis server can generate control signals to move the medical device from its current position to a future orientation. For example, the analysis server can determine the orientation of the medical device at a first time point (as described above, at the start of the second time period). At the first time point, based on the relative position of the patient relative to the 3D space in which the patient is located and / or the relative position of the medical device, the analysis server can configure the medical device to generate energy and transmit that energy to the patient's GTV. In this example, the analysis server can cause the medical device to generate and transmit energy according to a treatment plan similar to that described in process 200. In some embodiments, the analysis server can predict the future position of the GTV during the RT treatment process as described above. For example, when the medical device moves from a control point toward a control point and is activated to deliver energy to the patient's GTV, the analysis server can predict the future position of the GTV. The analysis server can then compare the predicted future position with the position established by the treatment plan and control the operation of the medical device such that the medical device aims at the GTV at the (predicted) future position.

[0079] In some embodiments, the analysis server may determine a beam direction view. For example, the analysis server may determine a beam direction view that represents the view of the GTV relative to one or more components of the medical device (e.g., the collimator of the LINAC). The beam direction view may be represented as a 2D representation of the GTV from the perspective of one or more components of the LINAC. In some embodiments, the analysis server may use the beam direction view when aiming the GTV with the medical device during the execution of an RT treatment plan.

[0080] Figure 7 An example implementation 700 of a process for predicting the location of a gross target volume within a patient during radiotherapy, according to an embodiment, is shown. (As in...) Figure 7 As shown, implementation 700 includes executing a sequence triangulation network 702 and a motion prediction network 704. In some examples, the sequence triangulation network 702 and / or the motion prediction network 704 may be related to the above-mentioned... Figure 6 The described models are the same or similar. In some embodiments, one or more operations described with respect to implementation 700 may be performed by an analysis server, which is similar to... Figure 1 Analysis server 114a and / or about Figure 2 The analysis servers discussed are the same or similar.

[0081] In some embodiments, implementation 700 may include generating multiple 2D projected trajectories using a 2D trajectory generator 702c. For example, one or more 4DCT images 702b may be obtained and used to determine (e.g., derive) 3D trajectories. The 2D trajectory generator 702c can then determine various angles and timestamps from the 3D trajectories to build a training dataset and allow training of a sequence triangulation network 702 (similar to that described above) to configure the sequence triangulation network 702 to reconstruct 3D trajectories (referred to as estimated trajectories) from sequential 2D X-ray images (also referred to as 2D observations) from different angles during a first time period.

[0082] Based on 2D images generated by an imaging device directed at the patient during RT treatment, the trained sequence triangulation network 702' can be further updated for a specific patient. The estimated trajectory can be provided as input to the motion prediction network 704 to predict 3D trajectories over future timeframes. Both the sequence triangulation network 702' and the motion prediction network 704 can be modeled using the implicit functions described herein. In some embodiments, an external surrogate signal can optionally be used as input to one or more of the sequence triangulation networks 702, 702', or the motion prediction network 704.

[0083] Figure 8 An example of movement of a gross target volume (GTV) 800 according to an embodiment is shown, the movement being caused by respiration. In some embodiments, the GTV 800 may be the same as or similar to the GTV described above. In a first state 800a, the GTV 800 may remain stationary; and during respiration 800b, the GTV 800 may move from a first position to a second position. As shown, whether the GTV 800 is moving or not, the GTV may be at least partially surrounded by a clinical target volume (CTV). Similarly, the CTV may also be at least partially surrounded by a planned target volume (PTV).

[0084] When moving (e.g., from a first position to a second position during an individual's respiration), the GTV 800 and CTV 801 can move in coordination with the PTV. For example, during respiration, the GTV 800, representing one or more lesions, may be significantly displaced due to movement of surrounding tissues and organs. This movement, especially relative to the PTV, can be modeled using the techniques described herein. In the example, the CTV 801 includes not only the GTV 800 but also any subclinical lesions that may be present, while the internal target volume (ITV) is based on the CTV to address uncertainties caused by organ movement, and the PTV may include additional margins to account for uncertainties in treatment delivery, such as patient positioning and setup errors. As shown in 800b, during expiration, the GTV 800 and CTV 801 can move upward relative to the ITV and PTV; and during inspiration, the GTV 800 and CTV 801 can move downward relative to the ITV and PTV. It will be understood that although the movement is shown in one dimension, the GTV 800 can move in various directions in three-dimensional space.

[0085] The following examples will help you better understand the currently available technologies: 1. A system for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time, the system comprising: one or more processors configured to: acquire image data associated with a plurality of two-dimensional (2D) images generated by one or more imaging devices positioned around the patient and representing the location of one or more lesions within a first time period; provide the location of one or more lesions to a sequence triangulation network to generate an estimated trajectory representing the motion of one or more lesions within the first time period; provide the estimated trajectory to a motion prediction network to generate a predicted trajectory representing the expected motion of one or more lesions in a three-dimensional (3D) space within a second time period; determine the future location of one or more lesions based on the location of one or more lesions and the predicted trajectory; and generate control signals based on the future location of one or more lesions to move a linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator. 2. The system according to any of the above examples further includes: determining the location of one or more lesions within a first time period based on multiple 2D images. 3. A system according to any of the foregoing examples, wherein one or more processors for providing image data to a sequence triangulation network are configured to: provide image data to the sequence triangulation network, wherein the sequence triangulation network is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients. 4. A system according to any of the preceding examples, wherein the sequence triangulation network is trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for one or more patients. 5. A system according to any of the foregoing examples, wherein one or more processors configured to provide the location of one or more lesions to the sequence triangulation network are configured to conditionally condition the sequence triangulation network based on at least a portion of the location of one or more lesions. 6. A system according to any of the foregoing examples, wherein the plurality of 2D images include a plurality of X-ray images, and wherein one or more processors configured to acquire image data are configured to: acquire the plurality of X-ray images from one or more imaging devices, the plurality of X-ray images including a two-dimensional (2D) representation of at least a portion of the patient during a first time period. 7. A system according to any of the preceding examples, wherein the position of one or more lesions during a first time period represents the pose of one or more lesions in 3D space. 8. A system according to any of the foregoing examples, wherein one or more processors are further configured to: determine a beam direction view (BEV) of one or more lesions based on the future location of one or more lesions and the orientation of LINAC at a second time point. 9. A method for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time, the method comprising: acquiring image data associated with a plurality of two-dimensional (2D) images via one or more processors, the image data being generated by one or more imaging devices positioned around the patient and representing the location of one or more lesions within a first time period; providing the location of one or more lesions to a sequence triangulation network via one or more processors to generate an estimated trajectory representing the motion of one or more lesions within the first time period; providing the estimated trajectory to a motion prediction network via one or more processors to generate a predicted trajectory representing the expected motion of one or more lesions in a three-dimensional (3D) space within a second time period; determining the future location of one or more lesions based on the location of one or more lesions and the predicted trajectory via one or more processors; and generating control signals based on the future location of one or more lesions via one or more processors to move a linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator. 10. The method according to any of the above examples further includes: determining the location of one or more lesions within a first time period based on multiple 2D images using one or more processors. 11. The method according to any of the foregoing examples, wherein providing image data to the sequence triangulation network comprises: providing image data to the sequence triangulation network via one or more processors, wherein the sequence triangulation network is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients. 12. The method according to any of the preceding examples, wherein the sequence triangular network is trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for one or more patients. 13. The method according to any of the foregoing examples, wherein providing the location of one or more lesions to the sequence triangulation network comprises: conditionalizing the sequence triangulation network by one or more processors based on at least a portion of the location of one or more lesions. 14. The method according to any of the foregoing examples, wherein the plurality of 2D images comprises one or more X-ray images, and wherein obtaining image data comprises: obtaining the plurality of X-ray images from a plurality of imaging devices via one or more processors, the plurality of X-ray images comprising a two-dimensional (2D) representation of at least a portion of the patient during a first time period. 15. The method according to any of the preceding examples, wherein the position of one or more lesions during a first time period represents the pose of one or more lesions in 3D space. 16. The method according to any of the above examples further includes: determining a beam direction view (BEV) of one or more lesions by one or more processors based on the future location of one or more lesions and the pose of LINAC at a second time point. 17. A computer program storing instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of: acquiring image data associated with a plurality of two-dimensional (2D) images, the image data being generated by one or more imaging devices positioned around a patient and representing the location of one or more lesions in a first time period; providing the location of the one or more lesions to a sequence triangulation network to cause the sequence triangulation network to generate an estimated trajectory, the estimated trajectory representing the motion of the one or more lesions in the first time period; providing the estimated trajectory to a motion prediction network to cause the motion prediction network to generate a predicted trajectory, the predicted trajectory representing the expected motion of the one or more lesions in a three-dimensional (3D) space in a second time period; determining the future location of the one or more lesions based on the location of the one or more lesions and the predicted trajectory; and generating control signals based on the future location of the one or more lesions to move a linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator. 18. A computer program according to any of the foregoing examples, wherein the instructions further cause one or more processors to perform the step of: determining the location of one or more lesions within a first time period based on multiple 2D images. 19. A computer program according to any of the preceding examples, wherein instructions causing one or more processors to provide image data to a sequence triangulation network cause one or more processors to perform the step of: providing image data to the sequence triangulation network, wherein the sequence triangulation network is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients. 20. A computer program according to any of the preceding examples, wherein the sequence triangular network is trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for one or more patients.

[0086] The various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above in terms of their functionality. Whether this functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of this disclosure or the claims.

[0087] Implementations in computer software (e.g., computer programs, computer program products, etc.) can be implemented using software, firmware, middleware, microcode, hardware description languages, or any combination thereof. Code segments or machine-executable instructions can represent procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments can be coupled to another code segment or hardware circuit by passing or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted through any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0088] The actual software code or dedicated control hardware used to implement these systems and methods does not limit the claimed features or this disclosure. Therefore, the operation and behavior of the systems and / or methods are described without reference to specific software code, and it should be understood that the software and control hardware can be designed to implement the systems and methods based on the description herein.

[0089] When implemented in software, these functions can be stored as one or more instructions or code on a non-transitory computer-readable or processor-readable storage medium. The steps of the methods or algorithms disclosed herein can be embodied in a processor-executable software module, which can reside on a computer-readable or processor-readable storage medium. Non-transitory computer-readable or processor-readable media include computer storage media and tangible storage media used to transfer computer programs from one place to another. Non-transitory processor-readable storage media can be any available medium accessible to a computer. For example (but not limited to), such non-transitory processor-readable media can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other tangible storage medium that can be used to store desired program code in the form of instructions or data structures and is accessible to a computer or processor. As used herein, disks and optical discs include compact optical discs (CDs), laser optical discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer-readable media. Additionally, the operation of a method or algorithm may be located as one or any combination or group of code or instructions on a non-transitory processor-readable medium or computer-readable medium that may be included in a computer program product.

[0090] The description of the disclosed embodiments provided is intended to enable those skilled in the art to make or use the embodiments herein and variations thereof. Various modifications to these embodiments will be apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the spirit or scope of the subject matter disclosed herein. Therefore, this disclosure is not intended to be limited to the embodiments shown herein, but should be accorded the widest scope consistent with the appended claims and the principles and novel features disclosed herein.

[0091] While various aspects and embodiments have been disclosed, other aspects and embodiments are contemplated. The various aspects and embodiments disclosed are for illustrative purposes only and are not intended to be limiting; the true scope and spirit are indicated by the appended claims.

Claims

1. A system for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time, the system comprising: One or more processors are configured to: Image data associated with multiple two-dimensional (2D) images, generated by one or more imaging devices positioned around the patient and representing the location of one or more lesions during a first time period; The location of the one or more lesions is provided to a sequence triangulation network so that the sequence triangulation network generates an estimated trajectory representing the movement of the one or more lesions during the first time period; The estimated trajectory is provided to a motion prediction network to generate a predicted trajectory that represents the expected movement of the one or more lesions in a three-dimensional (3D) space during a second time period. Based on the location of the one or more lesions and the predicted trajectory, determine the future location of the one or more lesions; and Based on the future location of the one or more lesions, a control signal is generated to move the linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator.

2. The system according to claim 1, further comprising: The location of one or more lesions within the first time period is determined based on the multiple 2D images.

3. The system of claim 1, wherein, The one or more processors used to provide the image data to the sequence triangulation network are configured to: The image data is provided to the sequence triangulation network, which is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients.

4. The system of claim 3, wherein, The sequence triangular network is trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for the one or more patients.

5. The system according to claim 1, wherein, The one or more processors configured to provide the location of the one or more lesions to the sequence triangulation network are configured to: The sequence triangular network is conditionalized based on at least a portion of the location of the one or more lesions.

6. The system according to claim 1, wherein, The plurality of 2D images include a plurality of X-ray images, and The one or more processors configured to obtain the image data are configured to: Multiple X-ray images are obtained from the one or more imaging devices, the multiple X-ray images including a two-dimensional (2D) representation of at least a portion of the patient during the first time period.

7. The system according to claim 1, wherein, The position of the one or more lesions during the first time period represents the orientation of the one or more lesions in the 3D space.

8. The system according to claim 1, wherein, The one or more processors are further configured to: Based on the future location of the one or more lesions and the orientation of the LINAC at a second time point, a beam direction view (BEV) of the one or more lesions is determined.

9. A method for predicting the location of a gross target volume within a patient during radiotherapy by modeling the motion of one or more lesions of a gross target volume over time, the method comprising: Image data associated with multiple two-dimensional (2D) images is obtained by one or more processors, the image data being generated by one or more imaging devices positioned around the patient and representing the location of one or more lesions within a first time period; The location of the one or more lesions is provided to the sequence triangulation network by the one or more processors, so that the sequence triangulation network generates an estimated trajectory representing the movement of the one or more lesions during the first time period; The estimated trajectory is provided to the motion prediction network by the one or more processors, so that the motion prediction network generates a predicted trajectory representing the expected motion of the one or more lesions in three-dimensional (3D) space during a second time period; The one or more processors determine the future location of the one or more lesions based on their current location and the predicted trajectory; and The one or more processors generate control signals based on the future location of the one or more lesions to move the linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator.

10. The method of claim 9, further comprising: The location of the one or more lesions within the first time period is determined based on the plurality of 2D images using the one or more processors.

11. The method according to claim 9, wherein, Providing the image data to the sequence triangulation network includes: The image data is provided to the sequence triangulation network by the one or more processors, wherein the sequence triangulation network is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients.

12. The method according to claim 11, wherein, The sequence triangular network is trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for the one or more patients.

13. The method according to claim 9, wherein, Providing the location of the one or more lesions to the sequence triangulation network includes: The sequence triangular network is conditionalized by the one or more processors based on at least a portion of the location of the one or more lesions.

14. The method according to claim 9, wherein, The plurality of 2D images includes one or more X-ray images, and The acquisition of the image data includes: Multiple X-ray images are obtained from the plurality of imaging devices by one or more processors, the plurality of X-ray images including a two-dimensional (2D) representation of at least a portion of the patient during the first time period.

15. The method according to claim 9, wherein, The position of the one or more lesions during the first time period represents the orientation of the one or more lesions in the 3D space.

16. The method of claim 9, further comprising: The one or more processors determine the beam direction view (BEV) of the one or more lesions based on the future location of the one or more lesions and the orientation of the LINAC at a second time point.

17. A non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to: Image data associated with multiple two-dimensional (2D) images, generated by one or more imaging devices positioned around the patient and representing the location of one or more lesions during a first time period; The location of the one or more lesions is provided to a sequence triangulation network so that the sequence triangulation network generates an estimated trajectory representing the movement of the one or more lesions during the first time period; The estimated trajectory is provided to the motion prediction network so that the motion prediction network generates a predicted trajectory representing the expected motion of the one or more lesions in three-dimensional (3D) space during a second time period; Based on the location of the one or more lesions and the predicted trajectory, determine the future location of the one or more lesions; and Based on the future location of the one or more lesions, a control signal is generated to move the linear accelerator (LINAC) from a first attitude to a second attitude, thereby adjusting the beam path of the linear accelerator.

18. The non-transitory computer-readable medium according to claim 17, wherein, The instruction also causes the one or more processors to: The location of one or more lesions within the first time period is determined based on the multiple 2D images.

19. The non-transitory computer-readable medium according to claim 17, wherein, Instructions that cause the one or more processors to provide the image data to the sequence triangulation network cause the one or more processors to: The image data is provided to the sequence triangulation network, which is trained based on multiple four-dimensional computed tomography (CT) scans corresponding to multiple previously observed patients.

20. The non-transitory computer-readable medium according to claim 19, wherein, The sequence triangular network is trained based on multiple estimated 2D images extracted from four-dimensional CT scans generated for the one or more patients.

Images